WMUK membership survey - suggestions and comments

Hello everyone.

Wikimedia UK is in the process of designing and developing a survey of our members. A sensible approach is for us to speak with you all about the things that you are interested in, rather than what we think you might be interested in.

To this end, we'd like to hear your suggestions and comments. What would you like to tell us about? What kind of information would you like to share? About which areas of our work and our organisation do you have strong opinions?

Talk to us here. Thanks for your time. --Stevie Benton (WMUK) (talk) 16:33, 10 September 2012 (UTC)

Two surveys

I would like to suggest to cut the survey in two parts (not sure how that should happen technically, but it should come down to it that it is totally fine if you leave after part 1 and you can simply click 'i dont want to take part 2'.). The first part should be the default, easy to answer part. What do you think about the color of the walls in our office on a scale from 1 to 5, what would you feel if the chair would grow a moustache on a scale from 1 to 5 etc and what is your general satisfaction as a) a member 1-5, b) as a community member 1-5 and c) as a member of the general audience 1-5. Just a bunch of scaling questions where the main outcome is the average scores and distributions.

Part 2 would be more input questions. Questions that are designed to get intelligent and constructive feedback, things where one single answer is more important than the weighed average or even distribution. Where answers make you think, and realize that while he's the only one saying it, he has a darn good point (how would you like us to communicate with you, how can we make the impact we have on your community activities bigger, what are qualities you're currently missing in our staff and/or the board or how could we persuade you to candidate for the board).

Just a thought :) (oh, and some of the example questions above might actually be serious suggestions ;)). Lodewijk (talk) 00:03, 11 September 2012 (UTC)

Hi Lodewijk - It is pretty easy to do this with the survey tool we use (survey monkey) but I wondered why you felt a split approach was needed - was it so we could collect bog standard attitudinal and demographic data as 'required questions' and allow people to opt out of the more nuanced feedback questions? I would have felt that a survey with piped questions based on previous answers was fine, especially as we don't have to make any of the questions 'required to answer' unless we want to! Thanks for the suggested questions too :) Katherine Bavage (WMUK) (talk) 08:53, 11 October 2012 (UTC)

Could we ask two questions

1. Their experiences as a newbie and what was good/could be better.

2. If they feel excluded for any reason - mobility, gender, geography, sexuality, race... and if so what we should do about it

Draft survey and Membership Strategy

Hi everyone! I'm going to send a reminder round the Wikimedia-uk list today about this, but given the feedback so far I've got some ideas to get started with writing the survey! I'll put a table in here with some suggestions and I'd be really grateful for feedback on the talk page, and suggestions for further questions here.

I'd like to run the survey for two weeks and have it close at the end of the month or early November. This is so the results can be collated and shared here (with raw data too for interested parties who want it), and take any further feedback those generate to the Board. This is in order for them to make some decisions about what benefits and opportunities we pursue for members, alongside how we keep members updated and well informed about the Chapter, board and staff.

To complement this I'm also starting an on-wiki dissuasion at 2012 Membership strategy consultation where I would welcome input from members and non members alike about the direction our membership strategy could take in terms of recruitment, benefits, development and overall goals. Please go have a look and put some thoughts there too! Katherine Bavage (WMUK) (talk) 09:10, 11 October 2012 (UTC)

Draft survey text

Please feel free to amend questions and insert new ones - I'll comment on the talk page if I have any concerns we can chat about them there :) Katherine Bavage (WMUK) (talk) 14:38, 22 October 2012 (UTC)

The survey will be emailed to members with a personalised link. It will therefore not be anonymous - but all questions are optional i.e. recipients do not have to answer anything they do not wish to. The email including the link will state this clearly, and invite members who wish to provide anonymous feedback to either contact me to discuss in a discrete phone call or email to the general membership@ email address from a non identifiable address.

This was because I felt that on balance although this could cause people to be more guarded with their responses, it would mean only members could answer the survey. This I hope means that people who are invested in the organisation in this way are given a specific forum for their voices to be heard. It also prevents any individual or individuals using the survey as a method of making unconstructive criticism under the protection of anonymity - skewing the results.

This is important as over time we want to track how these change amongst members only, and a weblink has no way of verifying membership - whereas an email link can only be used once, and by the recipient of the email.

I should add that it is anticipated that donors, and possibly volunteers and editors will be surveyed separately in future as staff capacity allows (or volunteer capacity - if anyone is interested in administering a survey of volunteers or editors then please get in touch to discuss!)

Demographic

Question	Answer Format	Follow up Question	Follow-up Format
1. How do you identity your gender?	Multiple choice (one answer)	-	-
2. How do you identify your sexual orientation?	Multiple choice (one answer)	-	-
4. Which category below includes your age?	Multiple choice - from 13 or younger to 80 or older (one answer)	-	-
5. What is the highest level of education you have completed	Multiple choice with descriptors (one answer)	-	-
6. What is your approximate household income?	Multiple choice - from £24,999 and below to £150,00 and above (one answer)	-	-
7. What is your relationship status?	Multiple choice (one answer)	-	-
8. If you are a a parent or carer, how many children currently live in your household? (If you are not, simply skip this question)	Matrix - multiple answers	-	-

Benefits

Question	Answer Format	Follow up Question	Follow-up Format
1. Have you ever attended a volunteer meet up?	Y/N - (If Y > Q5, If N > (Q2))	-	-
2. Is there a volunteer meet up near to where you live or work?	Y/N/DK (If Y > Q3, If N/DK > (Q4))	3. What reasons have prevented you from attending a meet up?	Multiple choice (multiple answers)
4. Would you like to be involved in establishing a volunteer meetup near to where you live?	Y/N	-	-
5. Would you attend events held specifically for members in your region?	Y/N	-	-
6. Do you think membership should be offered for free to editors based on a set number of edits made in a given period of time (e.g. 300 edits prior to a calling notice for an AGM)?	Y/N/DK	-	-
7. Do you think it would be appropriate to offer discounted memberships to editors or editor groups?	Matrix of choices (Multiple answers as Y/N/DK) - choices including 'Any editor with an account' 'Women', 'Minority UK language groups' etc	-	-
8. As a member, would you like to have access to an online catalog of members' materials to support recruiting members and partnership work with institutions local to you?	Y/N/DK	-	-
9. Following are some suggestions for potential benefits to be offered to members. Please rate from 1 - 5 these benefits in terms of the importance you would like to see Wikimedia UK place on securing them for members in the next 12 months (with 1 as most important and 5 as least important). Please note each option has a cost implication roughly indicated by the numbers 1 - 4, with (1) being potentially highly expensive to Wikimedia UK, which you may chose to take into account in answering.	Matrix of options - following suggests here role;_member development	-	-

Communications

Question	Answer Format	Follow up Question	Follow-up Format
1. Have you found the process of applying for and renewing membership easy or straightforward?	Y/N/DK (If Y > Q3, If N/DK > Q2	2. Please could you comment on specific issues and any suggestions for improvement	Comment box
3. Are you aware that Wikimedia UK provides regular updates about its work on Facebook, Twitter, via mailing lists and a blog?	Y/N/DK (If Y > Q4, If N/DK > Q6	-	-
4. Which of these news sources were you aware of?	Multiple Choice (Twitter, Facebook, Blog, Mailing lists)	5. How regularly would you read news about Wikimedia UK from any of these sources?	Multiple choice (Daily, Weekly, One or two times a month, One or two times a quarter)
6. Would you be happy to receive an e-newsletter mailing sent to you on a regular basis from Wikimedia UK?	Y/N/DK (If Y > Q7, If N/DK > Q8)	7. How often would you like to receive an e-newsletter?	Multiple choice (Monthly, Bi-Monthly, Quarterly, Twice yearly)
8. What topics would you like to hear about as a member of Wikimedia UK	Multiple choice (Multiple answer, inc 'Other' with a comment box)	-	-
9. This is the link to the membership area on the UK Wiki (http://uk.wikimedia.org/wiki/Membership) Do you think this has enough information about membership processes and benefits?	Y/N/DK (If Y > Q11, If N/DK > Q10	10. Here are suggested some changes that could be made - please rank in terms of importance. Feel free to add additional ideas	Ranking scale of staments plus comment box
11. Trustees and Staff members have Wikimedia UK accounts on the UK Wiki and can be contacted on their talk pages, on articles and on discussion pages. Have you contacted a member of staff or Trustee in this way as a member before?	Y/N/DK (If Y > Q12, If N/DK > Q13	12. Did you receive a satisfactory response within a reasonable timescale? If not please feel free to leave specific feedback in this comment box	Y/N and Comment box
13. Do you feel Trustees are responsive to you as a member?	DK (If Y/DK > Q15, If N > Q14)	14. Were you aware of the following ways of contacting Trustees or being apprised of their work?	Matrix of choices for Y/N/DK
15. Do you feel Staff members are responsive to you as a member?	DK (If Y/DK > next section, If N > Q16)	16. Were you aware of the following ways of contacting staff members or being apprised of their work?	Matrix of choices for Y/N/DK

Chapter and Community

Question	Answer Format	Follow up Question	Follow-up Format
1. How would you rate your experience as a new member? Please rate against the following areas with 1 being 'This was excellent' and 5 being 'This was poor'	Matrix of choices to include 'Processing of membership fee', 'Introduction and contact with other members', 'Guidance on role in WMUK governance processes', 'Introduction and contact with Trustees' etc	-	-
2. Have you ever felt excluded from fully participating as a member on any of the following grounds?	Multiple answer (Multiple responses based on Equality Act protected criteria) including 'None of the above' - If any selected > Q3	3. What measures do you feel could be put in place to prevent this reoccurring?	Comment box
3. Do you feel you have a clear understanding of the role members play in the organisation of Wikimedia UK?	Y > Q4, If N/DK > Q5	4. Please rank in order of most relevant the following statements about the role members play in the organisation of Wikimedia UK	Ranking order
5. Do you feel have a clear understanding of the role members play in relation to the wider Wikimedia UK and Wikimedia communities?	Y > Q6, If N/DK > Q7	6. Please rank in order of most relevant the following statements about the role members play in relation to the wider Wikimedia UK and Wikimedia communities	Ranking order
7. All members are eligible to stand to be elected as trustees of Wikimedia UK. Have you ever stood to be a Trustee before?	Y/N - Y > Q8, If N > Q9	What was your primary motivation to stand?	Multiple choice (one answer, including 'Other')
9. What have been the considerations that have prevented you from standing to be a Trustee in the past?	Multiple choice (multiple response) Including 'Lack of time', 'New member', 'Not interested in the role', 'Inexperience', 'Large responsibility' etc	-	-
10. Currently Trustees are elected by an approval voting system. A proposal has been made to change this to a standard transferrable vote system (STV) - details can be seen here {https://uk.wikimedia.org/wiki/User:LondonStatto/Proposed_EGM_Motion_on_Voting_System]. Would changing to STV make you more or less likely to vote in Trustee elections?	More/Less/No difference	-	-
11. Would you be interested in participating in a members workshop or workshops (online or offline) themed around policy and governance aimed at ensuring more members are engaged with the executive leadership and oversight of the Chapter	Y/N/DK	-	-

Election rules change

Not the most important issue, I understand, but if you are doing a membership survey it would be useful to ask a question to see whether the proposed change in election system [1] would make people less likely to vote (as some people have said might) AndrewRT (talk) 19:34, 11 October 2012 (UTC)

Great idea - have popped in as question 10 in the fourth section - let me know if you think this could be phrased better or anything else? Katherine Bavage (WMUK) (talk) 14:54, 22 October 2012 (UTC)

Concerns about purpose of questions and data storage

On 31st October Lodewijk raised a series of concerns about the demographic questions asked in the membership survey. In summary these were that to ask these types of questions could be considered insulting or invasive, and should not have been asked in surveys that were not anonymous, even if the questions were non compulsory to answer. He asked: 'Explain your policy regarding the collection, access to and storage of this highly private data with respect to the relevant EU privacy directives.' and then later broke this down into more detailed queries. I've split these out and tried to answer by section, and I've offered to post this up here to allay any concerns anyone may have, get feedback, and make sure that future surveys are administered better! (Always a good aim) Katherine Bavage (WMUK) (talk) 10:24, 1 November 2012 (UTC)

What is the specific purpose of the survey in general (in the context of the Data Protection Act 1998)

I've outlined the purpose of the survey as a whole in this page already, so I'll perhaps answer a slightly different question: 'What is the purpose of this section of the survey.

The short answer is - monitoring equality of opportunity, which according to ICO guidance requires us to ensure the appropriate safeguards are in place for the rights of individuals.

More specifically, what is the specific purpose of the (non-anonymous) questions regarding sexual orientation, income and relationship status

The purpose of the whole survey being non anonymous was to prevent the inclusion of results from respondents without a valid membership. We don't want or need to know anyone's sexuality or gender on an individual basis. However, as a cohort, I wanted to know about the following characteristics for the following reasons:

• Sexual orientation

I wanted to be able to get a sense of the specific proportion of our membership community that identifies LGBT to see:

• Whether there are grounds to offer targeted support - there is a section later in the survey where respondents are asked if they feel they have experienced disadvantage based on these grounds.
• Whether there are grounds to offer targeted outreach - we are starting to offer outreach based on encouraging/supporting editors from groups that are a minority in the movement, such as women and black and minority ethnic groups, so I thought it would be another area the Chapter might want to consider in a strategy to recruit and retaining members.
• Whether our membership as a group is similar to the UK population in terms of certain characteristics - gender, age and sexuality are covered by the survey questions - and how it compares to its board of elected representatives.

• Income

I wanted to be able to get a sense of the specific proportion of our membership community that may come from lower or higher income households to identify:

• A potential context for considering any proposed change to membership fees
• Useful information to justify a budget line to support membership activities (i.e. subsiding costs covering travel to events or accommodation)

• Relationship status

This and the question about children in the household help build a demographic picture of members, but other than helping paint a picture of who they are it's hard to say whether it would inform specific decisions in relation to membership. I included it as a standard question and received no objections before the survey went out. I'd be happy to disregard/remove this question from future surveys if people are concerned about it (but do read on)

To what extent will this personal data be stored and for how long?

The data is being collected and stored by Survey Monkey. Once I've checked respondents are members (and removed any that weren't) I can collate results into a report that will go to the board which will provide the headline results about the cohort. Then I will follow up on specific responses to two questions (Would you like to set up a meet up in your area, and Would you be interested in a workshop on policy and governance) by logging said interest on the respondents records. Then I can delete the responses - no need to keep the raw data when we have the report. This has to be done before the 7th November, which is the deadline of submission for reports to the board.

Could you point me to the location of your policies that are relevant to the processing of this data ?

I think our Data Protection Policy and Confidentiality Policy are the relevant ones.

Could you point me to the location of your policies that are relevant to the storage & access to this data?

Not yet. There are draft policies that were created in response to our obligations as outlined in the Fundraiser agreement, which are still draft and not approved by the Board. I had hoped to get these up on the UK wiki for consultation some weeks ago, but have had to prioritise other aspects of my workload. At the moment the intention is for the Board to review and approve them this month, but I'm uncomfortable with that not leaving enough time for consultation. I will request that the deadline for this is pushed back for the January meeting, so we all have plenty of time to scrutinise and develop them before seeking board approval.

I will circulate these for comment in the same way I did the survey ideas and questions - via the Wiki, mailing list, and on the water cooler.

Could you explain why this information was not included in the invitation/email? (Donor privacy policy is not relevant to the membership)

I didn't include a detailed explanation as it's conventional not to, however, linking to one is! The Donor privacy policy isn't about member data, but it's the nearest we have that enshrines the rights of a respondent, rather than our obligations, and elicits the same level of commitment we have to protecting and processing any personal data. Perhaps it would have been better to link to the data protection policy.

Its good that responding to you has highlighted a need to improve this. I will make sure that a page is placed on the UK Wiki eliciting intention and use in collecting survey data in more detail, and link this to future surveys, along with the data protection policy. Perhaps we do need a members version of the Donor privacy policy?

Is Wikimedia UK registered with the Information Commissioner's Office?

Yes, the certificate is linked in our Data Protection Policy

Have you considered at all whether collecting this data is legal in relation to the Data Protection Act 1998 and the EU data protection directive of 1995?

I believe it is. However, responding to you has made me keen to make sure in future I improve how communicating our intention of use and intention of collection and storage of data (as best practice, rather than because I believe that the current arrangements are not legal)

Respondents have been informed twice (in email, and in the survey intro) that all questions are optional, and that responses are not anonymous.

They have also been asked to email if they would like to give anonymous feedback. This can be achieved in three ways:

• By changing the settings of the survey before it is closed (so that email addresses are not stored against results) and emailing those who have requested this and are members a link with a password required for log in, with a warning that responses to questions about member meet ups and governance training cannot be followed up.
• By emailing thoughts directly to membership@wikimedia.org.uk using a non identifiable email address.
• By calling me for a confidential discussion, the content of which will be fed back in a section of the report.

Is any data collected from incomplete surveys?

No. The survey is set up so respondents can go back to previous pages in the survey and update existing responses until the survey is finished or until they have exited the survey. After the survey is finished, the respondent will not be able to re-enter the survey. So, I'd define that as a complete survey, whether or not all questions are answered.

Follow up

1. Feel free to respond here - on page, or on discussion page
2. Call me if a chat is helpful - 0207 065 0949
3. Emailing me - membership[@]wikimedia.org.uk