Friday, July 08, 2011

The Wrong Users: Targeting, Testing, and Design

This blog post is causing a lot of hand-wringing and discussion in the product design world.

To summarize, the author was doing a test on tabbed browsing usage at a mall, and he found a man named Joe, aged 60, who claimed he'd never used a computer before. Unsurprisingly, Joe had a terrible time using Internet Explorer, Firefox, and Chrome. He was confused by the most basic operations and did not have a grasp of the fundamental metaphors in use.

The original blog post does a decent job of teasing obvious conclusions out of the experience, but all of the commentary around it and all of the coverage I've read is of the "oh my god, computers are terrible, people have failed to make good designs, can't we make them easier, the first person who figures out how to make computing culturally relevant will be a billionaire" and so forth.

I think this is all misguided and silly.

In my opinion, the author made a huge mistake and wasted 3 hours of his time working with Joe in the first place. I'm sure it was interesting from an anthropological perspective, but it didn't really help him with the problem he was trying to investigate. Joe was the wrong user to test.

One would no more expect useful feedback on tabbed browsing from someone like Joe than one would get asking questions about dashboard layout for someone who's never driven a car. And in 2011, finding someone who's never used a computer is a bit like finding someone who's never been in a car before.

I'm not saying there's absolutely nothing to be learned - there's always something to be learned - but this is a terribly inefficient way to gather data. Makes for a good blog post but not a much better product (perhaps like my own scribblings!)

The design community is also overblowing the state of computer interfaces. Let's set aside all of the tedious anecdotes about how toddlers can manipulate the iPad and other touch interfaces (toddlers can also turn the stove on, but that doesn't mean they know how to cook). Toddlers aren't the right users for these devices, either. Point is, computer interfaces have made tremendous strides in a fairly short period of time.

Computers are still not "easy" to use without prior context and education, but no sophisticated tool is. I would argue most people don't know how to properly use a hammer or knives, and training would benefit them. Clock radios are still so confusing that hotels have to put "1-2-3" instructions on them, and most guests still either use their own or arrange a wake-up call. Hell, we have to keep reminding people to lift with their knees, not their backs. Some degree of education is always important.

More importantly, when doing user testing, you must pick the right users to test. Someone who's never used a computer before is generally the wrong user to be testing. Someone fundamentally uninterested in your product is the wrong user. Even the wrong demographic is not worth talking to or considering. You're wasting your time (at best) and drawing the wrong conclusions which leads to bad design decisions (at worst).

Knowing who your users are and who you want them to be is critically important for your business and for your design. If you or your company does not have a good idea of who your desired users are, you need to figure that out or you will flail and fail. You must target and test with the "right" users.

Saying things like "it's everybody" is a sign of naivete and/or hubris - tragic flaws. It's not everybody. If you're doing software in 2011, it's people who've at least used a computer before (or whatever your product is). Probably have to have broadband or other "always-on" connectivity. Have to be interested in what your product is about.

I've worked on a number of streaming products in the music space. I always write out a list of assumptions at the start of every product describing the potential users. For example:

  • Desire to listen to music beyond current solution
  • Broadband connections where they listen to music
  • Smartphone owner (Android or iPhone)
  • Willing to pay for music
  • Enough disposable income to pay regularly
  • Not afraid of technology
This list may not include specific demographic information, but it's already starting to filter out significantly from "everybody". 10 years ago, the "broadband connection" item was a big filter. 3 years ago, the "smartphone owner" was a big filter. The other items are also important filters to consider.

These items can be used to start building or inferring a demographic profile. They may seem stupidly obvious, but it is extremely important to understand these to combat the "it's everybody" argument. It's not everybody. Focus on the right users.

You can make reasonable assumptions about users based on their existing knowledge and experience. If your target user is an iPhone user, it is probably safe to assume they know the basics of operating an iPhone. This also means if you hew to most existing iPhone conventions, you don't have to worry about the user understanding them.

However, it is not safe to assume they know to experiment with every arcane gesture or mystery UI element. Unlike you, the designer, the user does not spend all day experimenting with every new, trendy, weird app. They'll know how to scroll and tap. If you make your app look like the other apps the user works with, they might pick up things like pinch. But in my own testing, I've found very few users even know about "swipe to enable delete". Android users seldom hit the "menu" button unless really stuck, and they typically expect the menu options to be the same through most of the app.

Know who you're going after. Don't chase down Joe. Don't run after the wrong users.

No comments: