User testing is essential to "seeking understanding" about our community.
I've seen multiple people fail at user testing at Experiment. This is a guide of the foundations, so that you don't have to learn this from scratch.
The ideal scenario for the product designer is to observe is as a fly on the wall with access to the subjects thoughts. This is impossible, so we opt for the next best thing. You watch the user use the product in person. The next best thing and probably the most efficient is watch the person over screen share using software such as Skype.
The more user tests you do the better you will get. Start with your best friend or start with your brother. Don't weight your best friend or your brother's opinion too heavily. Use it as practice to get comfortable asking inquisitive questions.
Remember your goal is to get in, get the information you need, and get out. Your single goal is to get 30 minutes with a valuable user. Do whatever it takes to earn those precious 30 min.
Let's start with four. If your goal is to get four 30 minute meetings in 24 hours this breaks down the problem. If it takes you more than 48 hours to get four 30 minute meetings you're doing it wrong. Come talk to me.
Pick your first victim and write them a succinct but personal email. Make them feel special. Making someone feel special never hurts when trying to get someone to do you a favor.
Tell them who you are.
Tell them the problem you're trying to solve.
Tell them why their opinion is valuable.
Tell them when and where and indicate flexibility.
Hit send.
Perfect, you have your template now. Copy and paste that template for your other victims.
For four meetings you're going to need to calculate how many users you need to contact based on an expected conversion rate. If you're new at this send 10 emails in 30 min. Done.
Start your first meeting at noon and work hours up and down. We have users in multiple time zones keep that in mind. If someone can only talk at 8pm, your user interview will be at 8pm.
You will want the flexibility to extend an interview if it is going well. You also never want to be rushed going into a new interview.
Keep it short and succinct. In the last two weeks I've had a lot of success with "a favor".
Now go get a snack or drink a beer and come back in 4 hours. The emails will either come back or they won't.
In 4 hours hopefully one person will have agreed to speak with you. Some will suggest other times. If you have 4 you're golden. Schedule calendar events and gather Skype usernames and go home for the night.
If you're not quite at 4, tweak your copy or just email 10 more users. At this point it is okay to double book for existing invitations already out there. You don't sleep until you get 4.
Ok. Go to sleep. Wake up and do the interviews.
We're going to assume you're doing user interviews over Skype screen share.
Each victim will have a calendar event from you marked for 30 minutes. The event will be called: Dan (UW) <> Cindy (Experiment). The description will read: Cindy to Skype Dan at "danjaffee"
The event will be scheduled in the victim's time zone. Use World Time Buddy to calculate time zones. Always schedule the event in the victims time zone.
When it is time for the interview, add them on Skype. Then call them. People love to talk about themselves. Spend the first 2 minutes asking them about their day. Then, briefly introduce who you are and what the goal of this is.
Ensure the user that there are no right answers to this test, your feelings will not be hurt if they don't like the product, the most valuable thing they can do is be honest and verbalize what they are thinking and feeling. Ask them to speak their thoughts out loud.
Ask them to share their screen. This may take novice computer users some time to figure out. If Skype is not working always have a back up plan such as appear.in or google hangouts.
If the network is having troubles the worst thing you can do is panic. Stay calm and ensure them that together you will find a solution. Whatever you do do not come off as frantic or rushed.
Once they have the screen shared ask them to use your product.Let the user explore. If they do something you didn't intend, ask why? Why did you do that specific action? When a user is about to click a button you may ask them to predict what the button will do before they press it. Your ideal candidate will verbalized everything they are thinking out loud. People not used to doing user studies will need more coaxing. The more you make them feel safe the more they will reveal to you.
At the end of the interview, genuinely thank them for their time. Let them know how much you appreciate their time (even if you don't).
There is one pro tip I learned early on from a friend. If you watch detective movies you may have noticed that right before a detective leaves the office he'll start to close the door behind him and then reopen the door, peek his head in and say... "I just have one more question..." You can use this strategy to your advantage. No one will be annoyed by this.
It is nice if you can record the interview or have someone next to you taking notes. This isn't always possible, so jot down the key points and learn from this interview how you want to conduct the next one.
You're interviewing multiple people because you're looking for patterns. The opinion of one user is largely useless, unless you want to build a product for a user base of 1. You want to test to see if other users actions support your assumptions. Even if all four people show the pattern remember it is still possible that they are an extremely small population within a large pool. When in doubt get another opinion. If the pattern prevails it will show it in the numbers.
If it helps you may want to write down the user story from your interview. Add in your observations. If the person seemed tired ask them if they are tired. Don't assume anything without confirming with the victim.
Currently there is no one on the team that is an expert on user testing. We've never had someone on the team with this expertise. I will always argue that the best person to do the user test is the builder herself.
To reply you need to sign in.