Talk:Train the Trainers consultation

Welcome to the discussion about Train The Trainers. Do you have opinions on the programme? Were you on one of the courses? Were you trained by someone who was? How do you think it can develop? Please let me know so that I can compile a report for the Board and plan the future programme. Jon Davies (WMUK) (talk) 15:51, 11 June 2013 (UTC)

Weekends

I was on one of the courses last year and was pretty impressed. One observation though, is it true that we pay a premium to have it run at the weekend? If so, and if we run more than one event a year, you might consider doing one in the week, a lot of the attendees would happily have attended a mid week session especially if it was saving the charity money. Jonathan Cardy (WMUK) (talk) 10:06, 19 June 2013 (UTC)

Yes, I have the same recollection. Midweek would have been just as good for me. Edwardx (talk) 22:06, 21 July 2013 (UTC)

Accreditation

This word worries me as it implies that you need to go on the course to be an accreditted trainer. This can be an expensive use of resources. As I understand it the course is intended to teach Wikipedians to teach and not to train Teachers about how to teach Wikipedia. We should assume that people who already have a degree level teaching qualification have already been taught to teach. The SWAT analysis overleaf does not mention the accreditation apart from noting it exists. It should be a strength but the course needs to recognise its limitations. If we compare an average accreditted trainer (from the WMUK course) with a WMF person who trains people in the USA and is visiting the country or Tony Santi before I met him at EduWiki (who was teaching Wiki without WMUK guidance) then how many of these are accreditted trainers? I think the correct answer is three. If its "one" then we belittle the training offered here. --176.227.135.245 10:14, 19 June 2013 (UTC)

Thanks for taking the time to comment - very much appreciated and I will be building everyone's comments into the review. You are quite right that the primary aim was to show active wikimedians how to teach, especially newbies. I might argue with your assertion that those with a teaching qualification (I have one) would always know how to teach editing but you are generally right. The TtT weekend does contain more than just, pace, differentiation etc it also includes exercises on writing modules and use teaching skills to interpret how to edit.

We have 30+ accredited trainers now. I don't know if the Foundation have anything like our scheme, I have not heard of it but may be wrong. Nobody HAS to be on the course to train but to be accredited you DO and the process is taken very seriously. That way we can offer an assurance to those being trained that their commitment is taken seriously. Jon Davies (WMUK) (talk) 09:14, 28 June 2013 (UTC)

To the anonymous questioner: yes, I would expect that in getting trainers for an event, we take into account more of their background than attendance at the TfT course. However, we need to be careful because people can have lists of qualifications as long as your arm and yet be weak on the skills that make a good workshop. I disagree that a degree level teaching qualification should be treated as equivalent: teaching is simply not the same thing as training. It's possible to be, say, an experienced university lecturer and still a complete beginner to the sort of skills needed for running the training workshops we need. In recent years universities have introduced a lot more active learning into their training so the overlap is getting stronger.

I agree that paying consultants every year is not sustainable: Wikimedia UK has to develop its own training programme in the longer term, but an essential step towards that is developing and auditing the skills we already have, by a credible process. If the Lead Trainers (accredited by the TfT process) decide someone is suitable to train on behalf of WMUK, then they should be able to "accredit" them, but bear in mind that these are early days and we don't yet have a formal process for this. We need some quality standard, so I don't think we should be afraid of the word "accreditation", but you're right that we need to apply the label with common sense. Pete Forsyth (ex-WMF) gives outstandingly good training, and Toni Sant has a great track record getting his students to improve articles: we're not blind to that.

Jon, in North America there is the Wikipedia Campus Ambassador network which identifies suitable people and prepares them to deliver a specific two day workshop: this is focused on the particular task of campus training to support a course, not general trainer-training like we have been doing, and it has tough entrance requirements so it basically requires entrants to already have the presentation/training skills. MartinPoulter (talk) 09:50, 28 June 2013 (UTC)

Maintaining a profile

Not sure whether this actually ties in with accreditation directly, so I've started a new topic.

From my perspective as Education Organiser, it would be really useful if every active trainer had a separate public profile as a trainer (perhaps similar to staff pages) to ensure visibility as well as record interest and expertise along with maintaining a public record of accreditation in a standard and useful way. This will also help us circulate a more professional profile of trainers ahead of training events, so that attendees (especially new editors) know who they're going to meet, and how they can contact their trainer before/after the event. I'll gladly elaborate on this, if required. --Toni Sant (WMUK) (talk) 18:58, 1 July 2013 (UTC)

• Great idea! Llywelyn2000 (talk) 11:46, 2 July 2013 (UTC)
• Excellent suggestion. As a new trainer, I would welcome having such a page. We need to make it easier for attendees to interact with trainers both before and after events. Edwardx (talk) 22:04, 21 July 2013 (UTC)
• Nice idea, Toni. I put one together for me as an example of how I see them; please see User:Deskana/Profile. I borrowed heavily from the existing Template:user info. In particular, to ensure some uniformity in the "What I do" section, I created a template (currently stored in my userspace at User:Deskana/Accreditation) where people can put in the level of their accreditation for each area, and then the template populates the entire section for them. Something similar could be done with the "Contact me" section if uniformity is wanted there as well. --Deskana (talk) (email) 22:35, 21 July 2013 (UTC)

I rather like the layout of the template, and having some uniformity strikes me as a good idea. It might be useful for trainers to note somewhere which events they have helped at; the office has these details, but it might be handy as a personal reference. Richard Nevell (WMUK) (talk) 10:17, 23 July 2013 (UTC)

Thanks for the suggestion. I've added the list of events I've attended (all one of them) in the "What I do" section. I think that one could theoretically make a template that accepts an arbitrary number of parameters to ensure uniformity there too, but in this case it may be easier to ask anyone that writes a trainer profile to copy my template. --Deskana (talk) (email) 15:30, 23 July 2013 (UTC)

I'm really liking how this is developing! Thanks. --Toni Sant (WMUK) (talk) 15:46, 23 July 2013 (UTC)

(unindent) It's turned out so much better than my userpage that I just moved it there. We can continue developing it, but really I don't know what to add to it at this stage. I consider it relatively complete. Thoughts? --Deskana (talk) (email) 08:19, 25 July 2013 (UTC)

As I noted just now on the UK mailing list, I'm not happy with the accreditation process. It was handed down from above, as far as I'm concerned, and I think it requires more thought. Toni can be forgiven for not being up to speed on past discussion, but it seems to me that these pages risk setting in stone a system that is not particularly well adapted to needs (as far as I can see). Charles Matthews (talk) 11:48, 25 July 2013 (UTC)

You raise a good point that we need to be careful not to create a divide between accredited trainers and those who are not accredited. The accreditation is not compulsory to run or be involved in training events, and we do need to make this clear. --Deskana (talk) (email) 12:34, 25 July 2013 (UTC)

Well, more than that, I distinctly remember being told, back in the day, that the point was to have a "whitelist". My concern would be that if this information is considered an integral part of a potential trainer's public profile, the absence of accreditation will be quite noticeable. Charles Matthews (talk) 15:06, 25 July 2013 (UTC)

The relevant diff. Charles Matthews (talk) 15:16, 25 July 2013 (UTC)

And, on the other hand, if I am to be discouraged from mentioning it in my profile, then I question the very existence of the training process. I am conscious of your concerns, but I do not feel that the only way to address them is by completely avoiding any mention of the accreditation on the profiles/userpages. --Deskana (talk) (email) 15:27, 25 July 2013 (UTC)

I actually have concerns of two kinds. The first is about mission creep: I don't see that this "outcome" is what was originally suggested. Others may see it a different way. (One way to address that concern is indeed to allow everyone do as they please with the data.)

The second is that I don't think the accreditation as currently given is particularly what the system needs, and as far as I'm concerned it was imposed. Since the whole business is now under review, this would be the time to mention that concern also. (E.g. the process takes no account of actual technical knowledge, but that is not apparent.) I imagine there may be people who think it is exactly what is required, or at least good enough.

Given that the training as a whole is staffed by volunteers, while up to now the approach driving it has been top-down to a fault, it would be good to air these matters. Charles Matthews (talk) 16:25, 25 July 2013 (UTC)

Understood. If these templates are found to be unwanted by a review, then we can get rid of them. I'm kind of having fun brushing up on my template syntax anyway (although this might be pointless due to Scribunto!). --Deskana (talk) (email) 16:33, 25 July 2013 (UTC)

Consultation

Re what is said on the page, I have not in fact been interviewed. Charles Matthews (talk) 11:45, 25 July 2013 (UTC)

Quite right we have not interviewed you or Andy. I will email you. It fell off the list pre-Wikimania. Jon Davies (WMUK) (talk) 10:11, 14 August 2013 (UTC)

Situation remedied - thanks Charles and Andy for your time yesterday. Jon Davies (WMUK) (talk) 08:33, 16 August 2013 (UTC)

Lead trainers

Who are the four lead trainers? Andy Mabbett (User:Pigsonthewing); Andy's talk; Andy's edits 13:53, 15 August 2013 (UTC)

The four people accredited as lead trainers are Doug, myself, Andrew Gray and Fae. Whether Fae is continuing is uncertain. MartinPoulter (talk) 11:16, 20 August 2013 (UTC)

Recommendations for future

Any thoughts in particular about these? Jon Davies (WMUK) (talk) 13:59, 15 August 2013 (UTC)

Overall I believe that we are on the right track and it is my opinion that the project should continue with the following recommendations. The training has been of a high quality and the benefits are beginning to be felt. As a charity we need to find more opportunities to use our trainers and keep them engaged in the meanwhile.

1. That the WMUK community is approached to find more opportunities to use our trainers.
2. That the fourth Train the Trainers is held in the Autumn outside London. Possibly in Wales to link in with the Living Paths Project or failing that in Scotland or an area where we currently need trainees.
3. That £20,000 is put into the budget for 2014-15 to facilitate two more sessions.
4. That as part of the 2014-15 programme we consider the possibility of including some refresher support for previous participants.
5. That a tender for a further two years is opened in February 2014.
6. That the accreditation system can function without the agreement of all lead trainers. Any two should hold the authority to approve Midas’s recommendations.
7. That the Virtual Learning Environment is extended to include the best material from our wider community with work undertaken to increase its readability and accessibility.
8. That the recommendations of the May 2013 Education Committee are considered particularly with a view to establishing reliable measurements of the course and its impact.
9. That one volunteer or failing that a member of staff is designated Train the Trainers liaison with the task of keeping in touch with the trainers, with quarterly email news and making sure they are given the opportunity to use their skills.
10. That a map on office wiki is established showing the approximate area where the trainers are willing to run sessions.
11. That a chart of sessions delivered is created and maintained on office.
12. That emphasis is given to measuring satisfaction from those receiving training from our trainers.
13. That regular monitoring of delivery is made by lead trainers or others to ensure standards are maintained.
14. The retention of editors will be vital to the success of our whole programme and we need to pay special attention to supporting new editors through follow-up contact and support.
15. The comprehensive review be shared with Education Committee and staff for lessons learned.
16. That, following a volunteer suggestion, we create short profiles of our trainers for people to see that will emphasise the quality of the programme.
17. That we consider holding courses on weekdays.

No. 8 - what recommendations? Link, please! Andy Mabbett (User:Pigsonthewing); Andy's talk; Andy's edits 14:24, 15 August 2013 (UTC) Done. Thanks Andy Jon Davies (WMUK) (talk) 15:21, 15 August 2013 (UTC)

Economics of the programme

This section currently includes the sentence "Realistically I would expect our volunteer trainers to be able to offer around 8 sessions a year and maintain involvement for three years as a minimum."

This sounds very unrealistic to me. If there sufficient opportunities to train, the most keen trainers will achieve that or possibly even slightly higher in a tiny number of cases. If we are being realistic, we have to admit that some trainers won't train at all and others would probably like to do three or four sessions a year if the opportunities are there. If we put additional effort into organising more opportunities to train we might get an average of three or four sessions per year, per person trained, with the numbers for individuals ranging from 0 to 10.

Remember: we are talking about volunteers.

And of course, because we are talking about volunteers, it will still be A LOT cheaper than if we had a small number of paid trainers. A naïve look at the cost break-down would have us say "why are we spending so much on training these people?" but the economics of it still work out nicely.

Yaris678 (talk) 17:36, 15 August 2013 (UTC)

Thanks Yaris678, good points. Perhaps 'an average' would be better. I based this on what is happening now without pushing too hard. One of the key bits of feedback is that the trainers are really keen to do more which is marvellous. There was a lot of interest in what we are doing at Wikimania. I cannot take much credit for TtT but those who fought for it, Steve V, Martin P etc should be very proud. Jon Davies (WMUK) (talk)

How much does this cost per head and how much do the outcomes cost?

I previously calculated that the cost of the training programme was £800 per head. Could the charity please publish a figure and track how this changes, rather than leaving it to deduced figures such as mine, particularly in the light of a commitment to an open tender in 2014?

With regard to the outcomes, I have not noticed any tracking of the key claimed outcome of the training programme of how many new (established, being characterized by more than X edits over Y months) editors the resulting training courses and workshops generate. Again, measuring the cost of the charity's investment per new long term editor/contributor head in all Wikimedia projects, would be a convincing rationale as to the value, or ongoing trend in value, of this programme. --Fæ (talk) 10:27, 16 August 2013 (UTC)

If "new editors" is the key outcome, it at least has the merit that editing activity is measurable. Choosing how to measure the quality and quantity of such editing is another matter, but the data will always be there, so we can aim to improve our statistcal analysis over time. Of course, there are many other less tangible possible benefits to editathons and training sessions.

For me, what should matter is the editing that we might not have had but for that trainee having attended the editathon. So, we need to measure activty before (if any), probably ignore the day itself, and activity afterwards. That seems better than some arbitrary judgment about what might constitute a "regular editor". For example, ten people each making ten edits a year is probably better than one person making 100 in a year, even if that person one person is the only one who qualifies as a "regualr editor". I'm not a statistician, but if we knew all the attendeee usernames, someone ought to be able to write some scripts to measure this activity. Edwardx (talk) 13:13, 16 August 2013 (UTC)

We are working hard to create measurements for what we achieve (see our FDC reporting which has made a start at this) but this is a challenge in any volunteer community and an organisation that has grown so quickly. We need to be clear that although page edits, user stats etc. are important, especially as proxy indicators, they are not the be all and end all of what we are achieving. To explain this, e.g. influencing institutions or public opinion, we have to be able to 'tell our story' as the FDC say. In parallel there is work being done for automatic tracking and measuring of editors participating in Wikimedia events and training. It was previewed in Hong Kong but I am guessing it will not be ready for a little while yet.

The reasonable estimate of £800 per person is not really expensive in terms of good quality training and the feedback from participants indicates overwhelmingly that Midas have offered training of a very high quality. Next year we need to prove that it has borne fruit in terms of sessions delivered. As Martin says we need to look to be self-sustaining in the future. And then there is the question of quantity vs quality.... This could be a metric doomed to being skewed in line with some of the NHS targets that led to unintended and perverse consequences. Jon Davies (WMUK) (talk) 09:46, 29 August 2013 (UTC)

Recommendation 2 & location

Recommendation 2 reads "That the fourth Train the Trainers is held in the Autumn outside London. Possibly in Wales to link in with the Living Paths Project or failing that in Scotland or an area where we currently need trainees." Train the Trainers really isn't suitable for new volunteers, though, as a lot of prior knowledge about the Wikimedia projects is assumed (both during the event, and also in order to give training later on). So it should really be run in places that already have a reasonable number of active volunteers - somewhere with a current wikimeet/group of people would be the best location. Wales could work if Wici Cymru is involved, but identifying the right place in Wales to hold the event could be tricky. Cambridge or Coventry could work well. Scotland definitely wouldn't, though, as there's a shortage of suitable volunteers to train there (I think there's currently 3?), and I don't see that changing sufficiently in the next few months.

In general, the office (or possibly the education committee, depending on who's driving this program) need to put more effort into figuring out what the best target audience is for this sort of event, which then feeds into figuring out where there is a critical mass of suitable volunteers, and would then have a direct impact on the economics of the programme. Thanks. Mike Peel (talk) 16:50, 16 August 2013 (UTC)

Not written in stone but if it was in Cardiff it would be easily reached from Wales and the West Country for instance. Jon Davies (WMUK) (talk) 09:48, 29 August 2013 (UTC)

Measurements

Thanks for all the work that's gone into this - and also to the many people that are commenting! Good to see lots of community engagement. I see that some metrics have been identified here:

• How many training days delivered?
• How many trainees kept editing?
• How many edits did they make, how many pages created?
• How we support the newbies after their training?

  (presumably "newbies" here means new editors or new trainers? I don't particularly like the word...)

However I can't see answers about what the metrics for these are to date. I would naturally expect a relatively high attrition of trainers and of new editors but I do think we need to know what these numbers are! The Land (talk) 09:15, 18 August 2013 (UTC)

Newbies can refer to others we will need to be clearer though. As I say earlier up there is an app in preparation for tracking editors and pages. We are working on integrating the data and would like to be able to use CiviCRM to create a single place to record the data. The support is important and need not be too onerous. A regular email or phone call would help encourage a lot of people. Jon Davies (WMUK) (talk) 09:52, 29 August 2013 (UTC)

What sort of "tracking editors and pages" are you talking abut? A lot of people might be very uncomfortable at the idea that their edits were being tracked by a third party. Harry Mitchell | Penny for your thoughts? 15:00, 29 August 2013 (UTC)

But Harry, how would I know you were standing for WikiProject Military history Co-ordinator if I didn't keep an eye on your Contributions page?Leutha (talk) 22:52, 14 September 2013 (UTC)

Feeding in

I'd really like to feed into this at length, but I've been at Wikimania, then on holiday, and now back in day-job stuff. Is this consultation still going until, say Wednesday next week? MartinPoulter (talk) 11:14, 20 August 2013 (UTC)

Final comments please

Thanks for the comments so far - I think we are heading in the same direction. Final comments taken until Friday 30th. Jon Davies (WMUK) (talk) 10:22, 28 August 2013 (UTC)

Comments from Harry Mitchell

• I generally agree with people like Charles that the accreditation is meaningless, and it creates a risk of over-formalising things. This is a volunteer-led movement and people don't necessarily want formality.
• What are "lead trainers"? Who are they? What's the point of them? Who decided we would have them? How were they chosen (I was under the impression it was to do with position in a pseudo-hierarchy, but I've been told it was to with ability)? Do we need to have a training rank structure?
• "No metrics for following up on edits"; be careful not to assume that an event was a failure if none of the attendees have edited since – not everybody who comes to one of our workshops wants to become a long-term Wikipedian. A lot of them just want to know more about Wikipedia or have one edit they want to make. That said, our track record when it comes to following up with attendees and trying to keep them enthused is abysmal. Perhaps in future we should aim to have a series of events with partners rather than just a single introductory event?
• "After the session trainers know how to train but can lack information as for how to run Wikipedia workshops"; it's not fair to include this as a weakness of the current programme because it's not what the programme was intended for; we knew that when we appointed Midas if we didn't know it before.
• "I would suggest that we need to ensure quality control by having lead trainers or others observe sessions on a regular basis." Bear in mind that the lead trainers are volunteers too. Also, this shouldn't be pitched as "higher-ups" supervising sessions; it would be nice if it was normal practice to have an observer at training sessions – they can offer feedback from a different perspective and the observer will benefit from watching different training styles.
• I sense a move towards "accreditation" from these courses becoming a prerequisite to running workshops in the name of WMUK. I doubt it's intentional, but contacting only people who have been "accredited" means that people who haven't been on the course but would like to deliver training aren't being given opportunities. Perhaps having a mechanism for such people to express an interest might mitigate this.
• This review mentions the lack of use WMUK is getting out of its accredited trainers, but doesn't see this as a symptom of WMUK's lack of engagement with its volunteers. It's not nearly as easy to get involved with WMUK activities as it should be—even of the few who know the chapter exists, hardly any know how they can get more involved or how WMUK can support them. It's not the first time I've said this (and if you go to one of the established meetups outside London, it's obvious), but I'll mention it again in the hope rather than expectation that it won't be kicked into the long grass.
• The review mentions trainers no taking the initiative as one reason that they are being under-used. This is true, but the skills required to run a workshop that's already set up and those required to make an initial contact, get agreement, and sort the logistics of the event are very different. Besides which, volunteers don't know they can, much less that they're expected to (see above). Many won't want to make the initial contact, but we could invite them to make suggestions for organisations or groups they'd like to work with, and staff could then make the initial approach.
• Finally, and I apologise for the length of these comments, especially so late in the day, but the review either underestimates or under-values the effect of these courses on the volunteers who attend them. I've had the privilege to get to know and to work with a lot of those people, and it's had a genuine impact no just on their involvement with WMUK, but on other aspects of their lives as well. The courses have given some of them a huge confidence boost, faith in their own abilities, bonds with their fellow volunteers, and a host of unmeasurable but significant transferable skills. Although we have a way to go before we realise the full potential of the program, we shouldn't underestimate the impact it has had on the individuals.

—Harry Mitchell | Penny for your thoughts? 15:47, 29 August 2013 (UTC)