How to extract the right information from key stakeholders.

You’re embarking on a large website redevelopment project. You will realise the need for effective content curation. You also understand that involvement, and buy-in, from all stakeholders and users is critical to the success of the project. Through information gathering at the start, both of these objectives can be managed through the same process. But how do you ensure the process is effective?

First of all, congratulations. You are one of the few people who have realised that successful digital projects understand the needs of all stakeholders. This includes both internal stakeholders—executives, board members—and external stakeholders like end users, media, and connected organisations.

The next hurdle is extracting the information you need in a way that gives you confidence to push your project to the next stage. There are several methods to consider. The one you choose can dictate the success of the project—and the quality of the information you collect. This piece will look at three common data collection methods to help you assess the right process for your project:

1. Online surveys
2. One-on-one interviews and
3. Workshops.

If you’re running long-form workshops or interview sessions, a flash of green can help keep things fresh.

1. Online surveys.

If you need to extract specific information from a large number of people, but don’t have the ability or the resources (read: time and money) to contact each respondent individually, online surveys are a good option.

The questionnaire you use to gather information via the survey could be quantitative, qualitative, or both.

Quantitative surveys involve ‘closed’ questions that allow people to respond from a fixed set of answers. Think ‘yes/no’, dropdown boxes, or Likert scales.

Qualitative surveys feature ‘open’ questions, involving free-typed text boxes asking for a respondent’s opinion.

So should you use a survey? Let’s look at some advantages and disadvantages:

Advantages of online surveys

• They are great for extracting information at scale. If you need to get information from dozens, hundreds or even thousands of people, you might struggle to do this through interviews or workshops. Online surveys are great for situations where you need to hear from lots of people, giving you a much better chance of getting a representative sample of your target population.
• They are less time consuming for both respondents and researchers. Interviews and workshops can be incredibly time-consuming. Online surveys—on the other hand—are cheap and easy to set up, and easy to fill in.
• They can be completed at the respondent’s leisure. Interviews and workshops will always mean trying to coordinate times between researchers and participants. With online surveys, respondents can complete the online survey at any time they like. This is particularly useful in situations where a respondent will need to go away and get information from someone or somewhere else, which may not be possible in an interview or a workshop.
• They facilitate comparability, analysis and visualisation. Especially in the case of quantitative surveys, when everyone answers exactly the same questions and this is done at scale, you can compare everyone’s answers, perform statistical analysis on the data and then chart the results.
• Answers are less influenced by the researcher. While online surveys still have the potential for researcher bias (based on how questions are worded), the impact of this is much lower than interviews and workshops, where leading questions can often creep in. Anonymity leads to candid responses. Online surveys, if kept anonymous, can lead to respondents feeling safer to be more honest with their answers.

Disadvantages of online surveys

• People may provide incorrect or incomplete information. The researcher has no way of knowing whether people’s answers to the survey are true and correct, as the respondent may have just filled in the bare minimum of information just to say they’ve completed the survey.
• Respondents may not understand a question. If they don’t understand it, they aren’t able to answer it.
• Answers cannot be probed. Unlike interviews and workshops, there is an element of rigidity in online surveys. You may want to learn more about a respondent’s answer, but unfortunately this is not possible.
• Small sample size. As mentioned above, one of the main benefits of online surveys is scale. However, one of the hardest parts of online surveys is getting enough people to contribute, meaning you may not have enough respondents to make it meaningful. It’s hard to convey feelings or emotions. In interviews and workshops, the researcher can read a participant’s non-verbal cues and body language. This is not possible with online surveys.
• Lack of accessibility. Online surveys are often designed in a way that makes them harder to fill in for people with visual or cognitive disabilities.

When to consider an online survey for your digital project

Basically, it’s any time you want simple information at scale.

For example, you want to learn from multiple users about the functionality on your current site and what people would like to see on a new site. Polling a large number of users for this information would be best done using an online survey.

Tips on running a successful online survey

• Think about the structure of questions. I sometimes think about the form you fill in when you sign up at a new doctor’s clinic. The question ‘Do you smoke?’ almost always has a ‘yes/no’ response. Which is reductive, because what if I say ‘no’, but I previously smoked for 20 years and I only quit last month? The doctor probably needs to know that information. Think about this when structuring your questions.
• Keep questions specific and understandable. The simpler the better.
• Consider using visuals. If you’re asking for feedback on a certain part of your website, consider adding a screenshot to support the question. This makes it easier for people to understand, and means they don’t have to navigate to your site to find the thing you’re asking them about.
• Consider coupling together closed and open questions. For example, ‘Do you feel the current site is easy to navigate? Yes/No’ – is a closed question. Follow up with an open question: ‘Why did you select this response?’ The combination of the two questions helps you to get quantitative data along with the meaning behind the data: the best of both worlds.

2. One-on-one interviews.

One-on-one interviews can be conducted in a few different ways. Our preference is face-to-face: the researcher sitting in the same room as the interviewee. If that’s not possible, consider video conference interviews or, our least favourite (but sometimes necessary), phone interviews.

Snacks can help interview participants feel more comfortable throughout their contribution.

Let’s look at some advantages and disadvantages of one-on-one interviews.

Advantages of one-on-one interviews

• Answers can be probed. Like surveys, one-on-one interviews usually start with a questionnaire of some sort. With interviews, however, the list of questions is best considered as a ‘seed’ list. You aren’t just looking to play question-answer tennis with the interviewee. Instead, look to expand on each answer with a follow-up question. This is almost impossible to do with surveys, but easy with interviews.
• You can capture non-verbal cues and emotions. When someone answers a question, you can use their body language and emotions to assess the intensity of their answer.
• Focused time with no distractions. An interview involves time blocked out specifically for the task of getting information. The interviewee can’t be distracted by social media, a crying baby or a knock at the door.
• Low time commitment for participants. While it is very time-consuming for researchers, interviews are usually a fairly low time commitment for participants… not quite as quick as online surveys, but usually much less than workshops.
• They are good for sensitive or confidential data. Some people may be unwilling to give away information via an online survey (through fear of that electronic data being insecure) or in a workshop (because they don’t want other workshop participants to have access to that sensitive data). One-on-one interviews tend to be a better setting for this type of information.

Disadvantages of one-on-one interviews

• They are costly. Given that there is an almost direct linear relationship between the number of participants and the cost of the research, interviews are almost always the most expensive form of research, especially as the number of interviews grows.
• They are time consuming. Each interview needs to be done separately, so from a researcher’s point of view, they can be incredibly time-consuming.
• You often have a limited sample size. Because of the two points mentioned above, it’s sometimes difficult to source participants. This means there is a lower chance the results of the research are truly representative of a wider cohort.
• The quality of the responses is dependent on the skill of the interviewer. Because interviews are more about probing (rather than just asking a set list of questions), the interviewer has to be highly proficient in this skill. And it is a skill: something that can be learned over time, but not something that everyone possesses innately. Inexperienced researchers may struggle to think of follow-up questions on the spot, and may fall into the trap of asking closed questions that cause conversations to ‘dry up’, or leading questions that negatively affect the quality of responses.
• They aren’t good for collecting quantitative data. If you want to measure something, online surveys are a much better option.

When to consider one-on-one interviews for your digital project

One-on-one interviews are good for situations when you need deep insight into a topic, but that topic needs to be treated with sensitivity.

For example, you want to hear from every department in your company about the troubles they face with a current website. You won’t get deep insight from an online survey, but people may not be willing to truly open up with their feelings in a workshop situation. One-on-one interviews will glean the insights you need.

Tips on running successful one-on-one interviews

• Allow people time to prepare. Never let someone go into an interview without any knowledge of what the interview is about. You don’t need to send them the specific list of questions, but you should give participants a heads up on what topics you’d like to discuss in the interview so they have time to think about those topics and bring more detailed answers to the interview.
• Ensure complete confidentiality. Reassure participants that whatever they tell you will be treated with complete confidentiality. Doing this will put people at ease to open up and elicit more candid responses.
• Ensure questions are mostly open-ended, and always non-leading. Your job as a researcher is to get people to open up about their opinions on a topic, so avoid closed questions with limited responses. But be wary of leading questions – ones where the answer is contained in the question, or where you subconsciously direct an interviewee to respond in a certain way.
• Have some good prompts up your sleeve. Like, ‘tell me more’, ‘what happened next?’, ‘how do you feel about that?’ Be careful though. Questions worded incorrectly, like ‘why did you do it that way?’, can sometimes make a participant defensive.
• Silence is your friend. When someone answers a question, don’t feel the need to jump straight to the next question. Instead, pause for a few seconds. Silence is uncomfortable, so the participant will often want to fill that silence with more information.
• Don’t lose focus, and keep an eye on the time. While one-on-one interviews are great for digging deeper into a topic, conversations can often go off on tangents and time can get away from you. Be respectful of people’s time. If you feel like you’ve spent too long on a topic, say so and move on. When you’ve reached your allotted time, give the participant the option to finish up on time. Only keep going if they are happy to do so.
• Consider getting out of the office. Sitting face-to-face across a desk can be intimidating for some people. Sitting side-by-side on a park bench on a sunny day could be a much better setting. Remember though, if you need to take notes, make sure it’s easy to do this. And if you need to record the conversation’s audio (with the participant’s permission, of course), make sure you don’t go somewhere too noisy.

3. Workshops.

Workshops often involve a group of people in the same room, working together to achieve some predetermined goal. Unlike online surveys and one-on-one interviews, workshops are often not solely an information gathering exercise. Participants are often also involved in helping develop a solution.

One or two team members should facilitate the conversation between contributing stakeholders.

In a workshop, your job is less of an ‘interviewer’ and more of a ‘facilitator’. Like one-one-one interviews, you may still have some ‘seed’ questions, but those seed questions are likely to be less specific and used solely to help you facilitate; to keep conversations moving, rather than looking for specific answers.

In workshops, you may also be joined by your coworkers. Here, you act as a group of researchers working together towards the same common goal. Workshops almost always involve some sort of tactile activity to help participants ‘see’ the information they are providing, such as answers written on a whiteboard or butcher’s paper, or post-it notes or index cards stuck to the wall.

Keep a visual record as stakeholders make progress throughout the workshop.

Let’s look at some advantages and disadvantages of the workshop format.

Advantages of workshops

• They encourage creativity. Workshops are useful when you need participants to ‘feed off each other’. When one workshop participant says something, it triggers an idea in someone else’s mind, who then adds their contribution to the conversation.
• You can have different levels of knowledge and experience in the same room. Further to the above point, participants can learn from one another. This is not possible with online surveys and one-on-one interviews.
• You can sometimes resolve issues there and then. With multiple people in the room, it is sometimes possible to solve a problem immediately, as opposed to the researcher going away, reporting on the findings and then acting on the data (which can sometimes take weeks).
• Ownership of results. Because workshop participants are usually there to develop a solution of some sort, there is often a greater sense of ‘buy-in’ for the outcome of the project.
• Helps ensure a shared understanding. When everyone is in the same room, everyone gets access to the same information. And while you can never guarantee that everyone understands everything in the same way, workshops are useful in achieving a shared understanding of a topic.

Disadvantages of workshops

• Dominant characters can take over. We’ve all been in meetings where one person does 80% of the talking. This is a real risk with workshops, where one person decides to take the superior role, jumping to answer each question first and shooting down other people’s answers. It’s critical these dominant characters are managed.
• A risk of ‘groupthink’. Groupthink is a psychological phenomenon that occurs when a group’s desire for harmony causes them to make poor decisions, with individuals avoiding controversial issues or agreeing with more dominant characters because they fear how their colleagues may perceive them. Groupthink leads to everyone just agreeing with everyone else (because it feels good to come to an agreement on something), rather than hunting for the right solution by tackling challenging topics.
• They involve a large time commitment from participants. The largest of our three information gathering methods, workshops can often involve participants in a room together for several hours. Some can even span several days. Workshops can be exhausting, both mentally and physically.

When you might consider a workshop for your digital project

Workshops are great when you are looking for the collective knowledge of a group of people, rather than just individual opinions. In successful workshops, the value of the group is greater than the sum of its individual parts. Workshops are great for situations when you need everyone to collectively ‘build’ something. This could include:

• a common understanding of the project.
• a solution to a complex problem or aspect of your project.
• a detailed roadmap of website features, functionality and deliverables.
• a timeline of activity.

Tips on running successful workshops

• Allow people time to prepare. Just like interviews, people will contribute better if they have some time to think about the topic. Don’t just send a meeting request and expect people to be at their best if they don’t know what is going to be discussed.
• No tech. Apart from any tech necessary to the workshop (projector, TV screen etc.), consider banning all technology from the session. Workshops can be long and it is difficult to stay focused. So, eliminate anything that can become an unnecessary distraction. Mobile phones turned off. Laptops closed.
• Consider ‘controlled reveal’. This a technique that sees participants write down their answers to a question in private, and then each person ‘reveals’ their answer. This simple technique is great for preventing one or two people from dominating a conversation and for avoiding groupthink.
• Take turns. Again, a technique for avoiding dominance. Get everyone to take turns speaking on a topic.
• Ask open-ended questions. Just like interviews, workshops are about getting people’s opinions; you’ll struggle to get people to open up with closed questions.
• Make it visual. Since workshops are often about a group of people building something together, the group needs to be able to see what they are building. Write stuff on post-its, draw stuff on whiteboards, stick stuff to the wall. Don’t just ‘talk’ about a topic. That’s not a workshop, it’s just a meeting.
• Get participants to contribute more than just words. They should be involved in any physical aspects of the workshop: the writing, drawing, or sticking I mentioned above. You shouldn’t be the only person doing this. Get the participants involved too.
• Capture the outcome. All those things you stuck to the wall should be photographed and sent to the workshop participants. This will help with recall, and give participants an even greater sense of ownership.

***

So, which form of research is best? Well, it turns out that they all have their own uses, and a combination of the three is probably best. If you’re running a digital project, consider using online surveys, one-on-one interviews and workshops together. They all have their benefits, and any disadvantages presented with one form of research can be overcome by another.