Assessing the Usability of Your Online Community (Part 2)

22 Feb

In the first part of this blog post, I talked about creating personas for your community. Although the title of that post included the word usability, it turns out that I didn’t really have much to say about usability per se in that post. That was intentional. People often think that usability is an absolute measure; either a site is usable or it is not. But that is false. Usability is contextual. You cannot properly assess usability without knowing who the user is and what they are trying to achieve. Personas and the goals/tasks that are associated with them, are a means to capture that contextual information and that’s why I focused on them in the first post.

Now, however, let’s assume that you have your personas in hand.  What do you do with them? There are actually two somewhat different scenarios you could find yourself in. If you already have a community site in place, then usability testing is a means to determine what usability issues exist with the site, and how seriously they impair peoples’ participation in the community. If you don’t have a community site already, then you can carry out usability tests against prototypes or existing platforms as a means to help determine what your site should be like.

I’m going to deal primarily with the first type and leave it as an exercise for the reader to sort out any of the salient differences for the second.

Usability testing all about having real users interact with your site in realistic ways, and then observing how they go about doing that. If at all possible, you want to create a video recording of these sessions because it will be incredibly useful to come back and review the sessions later on.

For each person you bring in, you want to give them a set of tasks to accomplish based on the persona that they most closely match. You need to make sure that these tasks are something that the user would try to do in real life, otherwise their tactics in carrying out these tasks may not be reflective of what will happen outside of the testing environment. Typically, you do not want to assist/interfere with the user while they are carrying out the tasks — in real life, your users won’t have someone sitting beside to lend a hand when they get stuck. Finally, you want to encourage your testers to talk about what they are doing. This is a little unnatural for most people — after all most of us have learned to keep our interior monologues actually interior. Nonetheless, this is a situation in which you really want to know why people are interacting with the system in the particular ways that they have chosen, and having them describe that process is probably the most straightforward way to get this information.

This sounds like a pretty intensive process, so a perfectly reasonable question is: “How many people do I need to use in order to get meaningful results?” Again, you’ll find a lot of consideration given to this in the literature. My own experience is that you can start to see issues after watching 3 or 4 people. By the time you get up to 7 or more, some issues seem blindingly obvious. That is, once you’ve seen everyone stumble while carrying out some particular task or get stuck on some particular screen, you know you’ve got problems. If those tasks are part of the primary mission for your site, then you know you’ve got to fix those things or risk having the site fail.

Other issues will be more subtle. They may only affect certain people, or they may be functionality that you don’t consider critical. As with almost anything of this nature, the reasonable response is to record it all, prioritize it, and figure out what you have the resources to deal with.

Of course, I’ve just scratched the surface of usability testing. There are a huge number of books on the topic, and I would definitely recommend picking one or two up and giving them a read. One of my favourites is Rocket Surgery Made Easy, but your mileage may vary.


3 Responses to “Assessing the Usability of Your Online Community (Part 2)”

  1. Serena Snoad (@serenasnoad) February 23, 2012 at 7:42 am #

    Hi Blaise, this is really interesting.

    I work for a charity so I have a slightly different perspective. For charities in particular, it’s important to think about the ability of your target membership when considering the usability of an online community, especially from an accessibility perspective.

    So, from an accessibility perspective, when asking: “(1) What benefit does the user feel they are getting from participating? (2) How difficult is it for them to participate?” it’s worth considering 2) before 1)

    A potential member may perceive a very high potential benefit but may experience barriers to participaion due to accessibility/usability issues. Worst case scenario: this could rob a potential member of the opportunity to take part and benefit from the community.

    • Terry Coatta February 23, 2012 at 9:58 am #

      I think this is exactly the sort of scenario that should motivate everyone to at least consider usability testing. Although the balance between “value delivered” and “barriers to access” may be more easily perceived in some cases, it is always present. Some people are “lucky” enough to host communities where the intrinsic value is so high that they can choose to ignore all but the most grievous usability issues. I think that communities hosted by game producers often fall into this category because the producer is the source of much sought after information about the game. But my experiences with associations/non-profits is that such a degree of intrinsic value is rare, so usability is something you ignore at your peril.


  1. Online Community Management Links Roundup 24/02/12 - Community Management Links | Blaise Grimes-Viort - February 27, 2012

    […] Assessing the Usability of Your Online Community (Part 2) […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: