What is "user testing"?

We get a lot of references to doing "user testing", and there seems to be a vague notion of what that means. At a basic level, there are two types, and they are very different. Here's how I explained it in a recent correspondence with a project sponsor:

...there are two types of testing that should be required of our customers: usability testing and user acceptance testing. They have different purposes, and different processes.

Usability testing is a one-on-one test with a facilitator and a participant....The goal is to identify barriers to users completing what they need to do in the system. This testing usually happens before the design is finalized....It is scripted and structured to quantify specific usability issues that will negatively impact implementation success and user adoption.

User Acceptance Testing is a test phase, usually after QA testing by the internal test team, where project sponsors and users are given access to the test system ,and are asked to do their normal tasks using the system and report back any functional problems. The goal is to identify what would prevent us from launching. Usually this happens well after the system is built, just before "go-live". It is usually not moderated but may be scripted.

So, if you do usability testing, it should identify major issues up front, which is much less costly to mitigate. Then UAT will likely only identify minor issues that would prevent launch. If you don't do usability testing, UAT will uncover more severe issues requiring more expensive mitigation, such as code refactoring.

Certainly usability issues may be uncovered--and should be addressed!--during UAT. But UAT is not a replacement for usability testing early in the process. Early, you are focused almost exclusively on the usability and accessibility of the system BEFORE the code is carved in stone.

Different and equally valid goals, and neither should be short-changed.

Mixed blessing

Got another call today from someone who wants to have me "look at their new web site design" and help them make it usable. I am so glad that the awareness is there, but now I have to explain how asking for one person's opinion doesn't allow you to check the box next to "usability" on your project plan.

Best Alert I've Seen This Month

From Google Calendar:

At some point, sites may have to stop support for outdated browsers, and this is a humorous and graceful way to do so.

Seriously. NOT Helpful.

People: enterprise systems need as much usability and user-centered design work as custom systems. Case in point: an ordering system in use in a large corporate environment. It is so difficult to use without training, that seeing Help is a relief. Unfortunately, there are two Help links:

Selecting the first Help link pops up a window full of technical system-level help topics. Not what the end user wants. Selecting the lower Help link pops up the following message:

Why did someone even bother to leave those links intact? They do not contain valuable information for the majority of our users, and introduce frustration in an already difficult system.

Easy fix

If you have an action button by an entry, there should be no need to select that entry. Here's an example. I selected "Register" next to one of the items. A message popped up as shown. (A further issue with this interface is that the checkboxes are in the header and don't look as much like list items - could be confused for a graphic element.)

My assumption is that they wanted to keep the Register button close to the items, rather than scrolling down the page, but it backfired a bit.

Testing is Good for Usability

The scenario: I selected "unsubscribe" on an unsolicited e-mail. A page loaded with the form below. After selecting "no" on both options and choosing "Submit", I got this error.

Did it again; same error. Testing is good for usability.

Über-usability

So, if your remote usability testers can't understand the logistical aspects of connecting to your remote usability study (technical or otherwise), will they really be in the right frame of mind to participate in your study?

Pay attention to the usability of the logistics and communication around your study.

Rock, Paper, Scissors

Who wins at Roshambo? DUH.

So, here's the one that trumps everyone, in case it's not obvious: if the user can't (or doesn't) use the system, the sponsor can provide no resources, and the designer has no work. DUH.

To All You Designers Who Hate Liquid Layouts

I was recently asked to provide some good examples of liquid/hybrid web page layouts, to defend my position that they are superior. While I believe, fundamentally, that they are a great idea, I had a hard time coming up with "off-the-cuff" examples of sites that I use regularly.


So, I did a little soul-searching (okay, very little) to defend my position. Why do I believe, fundamentally, that liquid (or hybrid) layouts are great idea? Accessibility is the first thing that comes to mind. A liquid display doesn't care what your monitor resolution is. It doesn't require someone with a small screen to scroll to the right to see stuff. It gives a similar user experience to everyone.


Mobile devices is the second thing that comes to mind. How can anyone possibly keep up with all the mobile web devices out there, and design specifically for each resolution?


So, why wouldn't you use liquid displays? Well, it's easier to design, build and test. Just like the argument for only supporting One Browser. And you can maintain the integrity of your visual design without worrying about things "flying around".

Needing more ammunition, I resorted to the most popular research tool: Google. I turned up  http://cssliquid.com, dedicated to the promotion of liquid/hybrid displays, which had some great examples (see for yourself). The examples there aren't all great, but some are quite effective. Like any other usability guideline: "Where it makes sense."

There are many places where you can do elegant, liquid things, Recently, I designed a div-based form that displays in one column on narrow displays and two columns on wider displays. All we did was chunk the form into two equal divs so that the right column would wrap below the left column on a narrow display. Not rocket science--but it was a place where it made sense.

All you designers: be open to liquid displays. It doesn't challenge the integrity of your design; it challenges the integrity of your design ability.

Support Minneapolis Lock and Key

Yesterday, as I was architecting the communication between my Skype and Google Voice accounts, so that people could call or text a local number and get me anywhere, I thought how interesting it would be that I could create an account with a different area code than mine and let people make local calls to reach me. Brilliant! I could pretend I was from somewhere exotic, like Duluth or Boise.

Then, coincidentally, this morning, there was an article on the local TV news about a nasty company who has done just that to scam people: created a business with a local number here and a business name ("Company Scam", for our purposes) that is similar enough to a well-established local business ("Company Legit") that it generates confusion among customers. They've managed to get higher Google rankings as well, so they're the first click for a lot of people.

So, here's the scene:

1. Customer finds Company Scam through Google
2. Company Scam charges 3X for bad work
3. Customer contacts the BBB or another Google search to complain
4. Customer calls Company Legit (thinking they are Company Scam) to complain

So, Company Legit is getting a reputation for bad work.

Search Google for minneapolis locksmith. Which would you choose? It's hard, isn't it?

Lessons to be learned:

• Just because it's on Google doesn't mean it's good.
• Just because it has a local number doesn't mean it's local. 
• A high ranking doesn't mean it's the best business. 
• Do your research. 
• And do business with reputable, local companies where you can, like Minneapolis Lock and Key.

User Research: Discriminating Age?

I participate in a taste test program where they qualify you for certain studies and then pay you (and your selected K-12 institution) a modest sum to eat stuff and tell them what you think. It's fun, mostly because I like to observe their methods. And I like food. And to give my opinion.


As long as I've been a member, participation was limited to those under age 65. Just this week, I got an e-mail that they recently extended the upper age range to 85, so I sent my mom over (though she's not yet 80), because she buys food! And beverages! And she has time to go to studies in the middle of the work day! And she purchases food for my school-age children! And she has an opinion!


I always thought it was odd that they didn't care about the buying power of people who can afford to blow their childrens' inheritance at the casino.

The 11th Web Usability Top Ten Peeve

Stated more eloquently by Elliott Kember than I could. Just because you have the tools to do a job doesn't mean you're good at it.

http://blog.elliottkember.com/posts/2-OnUsability

New Music

So, this evening, I agreed to accompany my son while he practiced violin, though my piano skills are only slightly good enough for most of the entry-level Suzuki piano parts.

Having spent several years learning to sing challenging vocal music in which one has to hold one's own part regardless of what else is going on, I thought it would be fun to try with my 7-year-old. I asked him if he could play the piece again while I played something completely unrelated and weird on the piano. Not only did he do it, afterward, he said, "That actually sounded pretty good! I really mean it!", meaning our odd, aleatoric composition was somehow pleasing to his particular musical ear.

I totally didn't expect that. (Sigh.) His uncles would be proud.