NewYork2008:User testing processes and getting info from users

Discussion

Why is everyone here?

Interest in how to build testing into the project cycle.

• Functional Testing - does it do what you want it to do?
• Usability Testing - can average people use it?

Functionality Testing

One/two sentence stories Story - Describes how a user would interact with the system to obtain value

"A user comes to the site and enters a search term here and gets results here"

You must design your functionality testing plan at the very beginning

Gunner doesn't do a lengthy requirements document up from - replaced with a documentation layer that describes what the site should do at the story level.

Debugging with Staging server before Production server.

Tester goes through stories and checks for bugs - a single set of clicks for each user story (not checking what happens in a form each time that a field is left blank).

Story Examples

• A user can create a toolbox
• A user can tag a toolbox
• A user can add to the toolbox
• etc...

Where does functional testing fit into the project plan?

• Functional testing at each phase - then roll out to users for usability testing
• Agile development tries to fix issues immediately as users find them
• "Asynchronous Agile" - users send feedback and it gets put to the top of the deliverables priority list for developers
• Ideally, it would be good to have the phrase "We won't consider this finished until _____ " and having some kind of document to fill in the blank.

Usability Testing

Who do you get to test?

• Sanity Checking
  • Test for Mom.
  • Barter with peers (other consultants who can test your stuff and you'll test their stuff)
• If doing testing with actual organization members/potential users, at the beginning, tell the client that you'd like access to their users
• Write the "find users" communication with the knowledge that it could be forwarded
• Ask for a mailing list of users who might care about the project
• If it's close, just do a push and ask users for feedback on new features
• Ask for volunteers
• You have to get users that care about the project (e.g. if you're testing a system that)

What are you asking them to do?

• Gunner - "Ad Hoc" either:
  • Task oriented - "Go to this website and try to create a toolbox and tell me what you think". - Ask them to complete a user story
  • Random click oriented

What do you do with feedback?

1. It's feature to be coded (high)
2. It's a bug (high/low) (highs are show-stoppers and lows are small web bugs)
3. It's a feature request (high/low) - meta data is to document who requested it so that we can tell her when it's done (low means that you'll probably never do it)

Usability Sprints

Aspiration does sprints around usability of Open Source projects. Some strategies:

• Just stand behind someone and ask them to user your website
• You don't need a lot of users
• Try to adapt to feedback quickly
• Setting up a WebEx/Yugma for the user to show you their screen
• Yugma is a free screen sharing program
• One person calls on the phone and asks user to perform tasks
• Everyone watches the user use the site
• Another person writes "AH-HA's" on the whiteboard

Can you just factor in usability at the end? No. Get user feedback with the first build.

Card Sorts

• Write terms based on the information architecture - different terms that would be expected to be found on the site
• Ask users to choose terms
• Ask users where those terms would go
• Ask users if they were to "click" on the card, what they would expect to find

Paper Navigation

• Have the basic functionality of the site on paper (wireframes)
• Show the user the "home page" print out and ask them where they'd click to do _____
• Then show them the piece of paper representing the page where they would go to if they clicked and ask them to continue the task

Sometimes it's great to just call instead of trying to get the user input via email.

Discussion About Scenarios/Stories

• Greg mentions a Scenario/Workflow/Detail document he saw for a Kabissa project through CiviSpace - * link
• extremeprogramming.org
• Boxes and Arrows

General Griping About Site Usability

• NYTimes.com is terrible!

Take-aways

• Simple one-sentence plain language user stories that start with "A user can..." can help drive development and usability testing from project beginning to end.
• Screen Sharing software like WebEx or Yugma
• Paper Navigation (see above)