Archive | February, 2012

Assessing the Usability of Your Online Community (Part 2)

22 Feb

In the first part of this blog post, I talked about creating personas for your community. Although the title of that post included the word usability, it turns out that I didn’t really have much to say about usability per se in that post. That was intentional. People often think that usability is an absolute measure; either a site is usable or it is not. But that is false. Usability is contextual. You cannot properly assess usability without knowing who the user is and what they are trying to achieve. Personas and the goals/tasks that are associated with them, are a means to capture that contextual information and that’s why I focused on them in the first post.

Now, however, let’s assume that you have your personas in hand.  What do you do with them? There are actually two somewhat different scenarios you could find yourself in. If you already have a community site in place, then usability testing is a means to determine what usability issues exist with the site, and how seriously they impair peoples’ participation in the community. If you don’t have a community site already, then you can carry out usability tests against prototypes or existing platforms as a means to help determine what your site should be like.

I’m going to deal primarily with the first type and leave it as an exercise for the reader to sort out any of the salient differences for the second.

Usability testing all about having real users interact with your site in realistic ways, and then observing how they go about doing that. If at all possible, you want to create a video recording of these sessions because it will be incredibly useful to come back and review the sessions later on.

For each person you bring in, you want to give them a set of tasks to accomplish based on the persona that they most closely match. You need to make sure that these tasks are something that the user would try to do in real life, otherwise their tactics in carrying out these tasks may not be reflective of what will happen outside of the testing environment. Typically, you do not want to assist/interfere with the user while they are carrying out the tasks — in real life, your users won’t have someone sitting beside to lend a hand when they get stuck. Finally, you want to encourage your testers to talk about what they are doing. This is a little unnatural for most people — after all most of us have learned to keep our interior monologues actually interior. Nonetheless, this is a situation in which you really want to know why people are interacting with the system in the particular ways that they have chosen, and having them describe that process is probably the most straightforward way to get this information.

This sounds like a pretty intensive process, so a perfectly reasonable question is: “How many people do I need to use in order to get meaningful results?” Again, you’ll find a lot of consideration given to this in the literature. My own experience is that you can start to see issues after watching 3 or 4 people. By the time you get up to 7 or more, some issues seem blindingly obvious. That is, once you’ve seen everyone stumble while carrying out some particular task or get stuck on some particular screen, you know you’ve got problems. If those tasks are part of the primary mission for your site, then you know you’ve got to fix those things or risk having the site fail.

Other issues will be more subtle. They may only affect certain people, or they may be functionality that you don’t consider critical. As with almost anything of this nature, the reasonable response is to record it all, prioritize it, and figure out what you have the resources to deal with.

Of course, I’ve just scratched the surface of usability testing. There are a huge number of books on the topic, and I would definitely recommend picking one or two up and giving them a read. One of my favourites is Rocket Surgery Made Easy, but your mileage may vary.


Assessing the Usability of Your Online Community

17 Feb

Can poor usability kill an online community? I think it can. In any human activity, there is a trade-off that individuals make whenever they undertake some task: Is the benefit that I am receiving from this worth the effort that I am putting into it? We may not always consciously frame our actions that way, but that balancing act often underlies our gut sense of whether we like something or not.

So, for your community, there are two things to consider: (1) What benefit does the user feel they are getting from participating? (2) How difficult is it for them to participate?

If you are lucky, the benefit that people feel they are getting is really high. And please take note of the phrase “feel they are getting” because it’s not about what benefit *you* think the community provides, its about what your members perceive. Nonetheless, if you fall into this camp, you have a lot less to worry about because people will tolerate poor usability. They won’t like it, and you shouldn’t feel good about it, but at least it doesn’t threaten the existence of your community.

Unfortunately, particularly in the association space, my experience is that members don’t immediately see the value of an online community. It’s also my experience that a thriving community becomes a significant member value, but the Catch 22 is that you don’t get a thriving community without getting a bunch of people involved which is challenging when the initial perception of value is low.

The upshot of all this is that I think that usability really does matter for most association online communities. Now the interesting thing about usability is that its not just a matter of asking “Is this site easy to use?” Real analysis of usability is always framed in terms of the tasks that a user wants to carry out. You can’t even begin to think about usability until you have a reasonably good grasp on the tasks your members want to accomplish via the online community.

Thus, the first step in creating an excellent user experience is understanding your members. Based on my experience in the software industry, I find that the best way to develop this understanding is to actually talk with people. Find some of your members who are already using Facebook or LinkedIn, or find members who indicate that they are interested in participating in an online community. Ask them what they would hope to be able to get from an online community specifically for the association. Are they interested in an online book club? Would they help to build out a virtual library of reference materials that all members should be aware of? Would they be willing to participate in a virtual Q&A session on their professional experiences?

From this you can start to build up a set of personas. Each persona is associated with certain characteristics, goals they want to accomplish, level of experience, etc. For each persona, there will be a set of primary tasks or activities that they want to accomplish via the community. It is very important to keep this focused on the point of view of the user. Again, it’s not what goals *you* want to achieve, it’s what *they* want to achieve. There are different ways of capturing this information. For example, some people like to represent personas as actual individuals, so they will give them names, and supply back-story such as where they work and what their position is. I’m not religious about this — do what works for you. What is important is a ruthless focus on the user.

So, the first step in looking at usability is documenting these personas. In my next post, I’ll look at what you actually do with them.

Association Communication Evolution

15 Feb

Association communication has evolved over time as new technologies have become available. I call these eras – Meet, Mail, Magazine and Moderation.


A History of Associations identifies the earliest formal associations as medieval craft guilds and merchant trading groups. They communicated face to face because there were few other options.. The first scientific society the Academia Secretorum of Naples, was established in 1560 and the members met at Giambattista della Porta’s home. The Academy of the Mysteries of Nature was required by the Pope to close eighteen years later under suspicion of sorcery but the Royal Society of London which first met in 1660 has lasted a little longer. In 1662 it was first permitted to publish and it is still doing so.


Publishing technology allowed mass printing of documents. The availability of mail service enabled distribution. Members of associations no longer had to meet face to face to communicate. They could stay informed of association news and be part of the association without the need for travel. That in turn allowed associations to expand in size and geographical scope. Charles I of Great Britain established The Royal Mail in 1635 but it took a while to catch on. For the first two hundred years until 1840 it was considered expensive, confusing and corrupt. However, once established, it did allow most associations the capability of mass communication.  Many associations still use mass mail for raising funds and attracting members.


The next step in association communication evolution was the magazine. Magazines were really a further development of printing technology but the reason I am calling it a milestone in the evolution of communications was the ability to advertize. Glossy, with pictures; magazines were a revenue producing mass media for larger associations. You did need a sufficiently large association and/or enough advertisers to be able to justify the editorial staff, pay writers and put out a magazine, but now there was a way to communicate that members considered a benefit of membership and magazines generated revenue.


My fondness for alliteration leads me to name the new era of association communication “Moderation”. By moderation I mean moderation of online conversations. The term does capture the essence of the switch from top-down, one to many mass communication technologies such as publishing a magazine or newsletter, to the anyone can communicate, many to many communication possible with forums, wikis, blogs and online communities.

This is a fundamental switch but it is not an immediate switch. Just as mail and other communication technologies took time to be adopted so to it has taken time for people to have and use online access.

It is also not a complete switch. People will still meet face to face and some will meet virtually. People will still read their monthly newsletter or magazine and some will do so online. It is an additional channel of communication.

The bigger switch is in the culture of participation. We, of a certain age such as me, are used to being informed, not informing. That is why there is initially some reluctance to putting oneself out there by posting a comment or asking a question online. Many were initially a little reticent with Facebook, even when simply sharing vacation pictures or making some social comment on a favourite team or news item.

Making a comment about something important may take a little more courage. Private association communities where the comments are to colleagues feel a little safer but that will be the topic for another posting. Most important is that associations now have the tools to harness the power of the association’s biggest asset, its members and their knowledge. The new association communication credo may be “We know more than any of us.”

What is Next?

Associations are adopting the new communication technologies. For example, the Royal Society has adapted to the times. The 44 fellows (members) that are elected per year can be proposed for fellowship online via e-Lect.  Fellows no longer have to meet face to face; they can meet in the eFellow’s room. The Royal Society has also embraced other social media; anyone can respond to a Royal Society blog after logging in with Facebook or Google.

The challenge is to figure out what software tools to use to communicate and how to use them. Should you use public communities such as Facebook or have your own private community or both? That too will be the topic of a future blog.

This blog is available as a .pdf for download here.

Your Community: What are its Vital Signs?

8 Feb
When you’re community is healthy, its usually pretty obvious. New content is being posted, discussions are ongoing, new members are signing up. But when your community is not healthy or when you are just starting up a new community, it can be a little more difficult to assess how things are going because there is much less visible activity. Nonetheless, you need to know whether the community is growing, and if not what you can do about it.

Here are some metrics that we use to assess the health of communities that are not particularly active and the tactics that we use if these vital signs move into the warning zone.

Proportion of  active users – This is the fraction of your site’s membership that has visited the site over the past seven days. You could use a longer or shorter time frame, but we find that a week works well. We like to see this value above 50%.

When this value drops below 50%, there are two things we typically do: (1) Make sure that there is sufficient new content being added to the community. This requires authoring new content, finding new content from elsewhere, or working with members to generate new content. (2) Make sure that notifications are enabled so that all community members are aware of the new content that is available.

A lot of people are lurkers, and you may find that your community does not look particularly active, but has a relatively high fraction of active users. In this case, we would start to engage with individuals within the community to encourage more active participation.

Average user page views – The average unique page views per user over the last seven days provides a measure of how much content/interaction on your site is of interest to your members. You want to measure this over a time period like a week rather than per session because a user may visit your site several times, looking at different information each time. The higher this number the better, but we start to get worried if its less than 2 because it means that users are hitting your site, but not finding enough of interest to keep them there. When this happens there is a good chance that they will drop out of the community.

When this is the case, there are a couple of potential problems. First, you may not have enough new content, so that users are checking in but not seeing anything new. Second, it may indicate that there are usability issues with your site that are acting as barriers to the user. For example, if the discussion area on your site is reasonably active, but the home page does not provide any obvious way to easily go to new discussion threads or replies, you may have the right content, but not be making it easy enough for your members to find.

Net community growth – This is the number of new users over the past seven days minus the number of users who have left the community. If you do not require explicit registration for your community, then you will need to use some sort of heuristic to both identify users and also to effectively remove them from the community if they have been inactive for a sufficient period of time.

Community growth is usually a function of community size; bigger communities grow more quickly. Of course, there is an upper limit in the total addressable audience for any community, and should you be lucky enough to approach it, growth will obviously slow. Initially, however, most communities are small and grow by word of mouth. We like to see communities growing at the rate of at least 1 new member for every 20 or so existing members. If you aren’t achieving this, then you can explicitly enlist the help of your members. Get them to invite the friends or colleagues into the community. As the community grows larger, you may find traditional SEO techniques will help draw new users in. You can also look into other social media platforms such as Twitter.

Changing Your Organization

3 Feb
I notice a lot of energy around the notion of changing our organizations these days. To make them more innovative, to make them more collaborative, to make them more open, etc. But I think sometimes the emphasis is misplaced. Because the most important factor in achieving this is not the organization, its processes, its vision, etc. The most important factor is people.An organization is not open to change — people are. To have change occur, I have to be looking for opportunities to do things differently. My boss has to be open to change, and his boss has to be open to change. Rules and processes can’t make this happen. They can hinder it, or they can support it, but fundamentally it arises from the personalities of the people I work with.

And while it is possible for people to change, it’s not an easy undertaking. So the most effective strategy for change in an organization is to hire people who have the personalities you are looking for, and promote the people who reflect those values. Unfortunately, at least in my experience, this isn’t a science.

For example, I’ve come to realize that cultural fit is a make or break issue with new hires. I’ve seen numerous situations in which people had all the technical skills required for a job, but in which cultural issues made it impossible for them to contribute effectively. I think that I have gotten better at navigating these waters, but I can’t put that knowledge in a box and pass it on to others. Maybe that’s just a limitation that I have, but I don’t think so.

So what do we do? The most important take-away for me is to keep the focus on people. Organizational structures and processes can’t make this happen. In hiring, don’t just focus on technical skills — use your intuition to see whether someone has the personality traits that you need. And since culture is vital, involve as many people as you can in helping you assess whether someone will fit in.

The Annalist

2 Feb
My bedside reading for the past few months has been the “Black Company” series by Glen Cook. It’s a kind of gritty fantasy about a band of mercenaries with an 800-year history. One of the reasons that they are able to maintain continuity over that kind of time period is that one of the roles within the company is “annalist.” The annalist is basically the company’s historian. He documents where they are, who is employing them, and what they are doing.

But its not enough to just have a history. You have to be able to apply it in a meaningful way to your current situation. And the Black Company does this. The leader of the group, generally referred to by his title as “The Captain”, often reads the selections of the annals to the company in the evenings. And he doesn’t just read them, he tries to draw meaning and inspiration from the experiences of the past.

Assuming you have made it this far, you are likely wondering what the point of this particular ramble is. Well, I think that online communities, and in particular, social curation afford us the opportunity to draw inspiration from our collective histories and wisdom. Unlike, the Black Company, we don’t have to rely on a single individual to collect our experiences. Community platforms give us all the means to contribute our observations, and to discuss the experiences of others. Like the Black Company, we can use these capabilities to build up a body of knowledge about ourselves, our interests, and our professions. And by continually revisiting it, discussing it, and applying to our current situations, we keep that knowledge alive — maybe even for 800 years!

And just in case you hadn’t noticed, I’ve outed myself as a complete geek by refering to fantasy novels (this post), Star Trek (Jan 17), and Dr. Who (Jan 4) in my recent blog posts.