Quick & dirty usability testing: a case study

We’re web developers. We always thought of usability testing as something that one could only do in fancy labs with two-way mirrors and eye-tracking software. But a few years ago, we were inspired to start incorporating usability testing into some of our projects after attending usability guru Christine Perfetti’s Usability Bootcamp. She impressed upon us that even a few quick-and-dirty tests with a handful of users is far better than nothing at all. Every time we observe testers using the sites we’ve built, it’s a wonderful combination of nerve-wracking and eye-opening that goes a long way toward helping clients achieve their goals.

Why bother making my site more usable?

I can’t sum this up better than a recent tester who helped us review our new design for the New England Foundation for the Arts (NEFA). He said, “I really appreciate the simplified menu and clean design. I have an aversion to clutter—it makes me move through a site faster because I can’t wait to leave.”

Which would you prefer: a customer (or donor, or prospective student) to scurry through your site and leave in a huff of frustration? Or linger lovingly over every page? An easy-to-use site just feels better. It creates a favorable impression of your organization. It subtly reinforces your brand.

If that's not enough, consider these benefits.

Usability testing can help. . .

Catch technical glitches. On NEFA’s site, we didn’t question the "Search Grant Recipients" menu that had always been there, and planned to leave it in the new site. A tester tried to do a search and discovered that choosing a top-level category did not return any results—editors had only been putting their content into sub-categories, and the system was treating sub-categories as distinct searches. This glitch might have gone unnoticed had we not done the usability study.

It was also helpful to discover that same menu seriously hampered testers’ efforts to find related grant recipients. Which menu would you rather use?


Challenge your assumptions about what’s obvious. Roughly half of the testers missed the large, blue donate button at the top—even when they were told, “You are interested in making a $250 check to NEFA. Figure out how.” Instead, they sifted through the drop-down menus on the main navigation bar. If half the testers, actively looking for a donate button, could miss it, we knew we had a problem! We easily addressed the issue by adding Donate to the About Us menu, which is where the other half of the testers expected to find it.

Seems obvious, right? But half our testers missed the big blue Donate button.
Test your content. We’d gone back and forth with NEFA about whether to include blog entries and blurbs describing each grant on the Dance, Music, Theater, and other discipline landing pages. In the interest of promoting exploration while reducing clutter, we agreed to keep the blog posts, but remove the blurbs. Wrong move! When we set testers the task of figuring out which grants were right for them, they either ignored or were noticeably distracted by blog posts, could not understand the grants from the title alone, and got confused. When we removed the blog posts and added blurbs for the second day of testing, users completed the task much more easily. 
Before: Users got distracted by the blog entries, and couldn’t figure out what a grant was based on its title alone.
After: When we removed blog entries and added short blurbs, users much more easily found what they wanted.

Solve internal debates. NEFA had a lot of internal debate about whether to use "Grants & Programs" or "Grants & Opportunities" as one of their main tabs. So on the first day of testing we tried "Grants & Programs," and the second day we used "Grants & Opportunities." We asked each tester for their preference. Guess what? It turned out not to matter. All the testers easily completed the task either way, and the testers were pretty evenly split about which they preferred. This information freed NEFA up to stop worrying about which was better and just pick one.

Engage your constituents. We didn’t even consider this benefit, but NEFA told us that the usability study was also helpful from a relationship-building perspective. Turns out the folks they asked to participate really appreciated having an opportunity to share their opinions and help one of their favorite non-profits build a successful website. Everyone wins!

And the obvious—make it easy on your visitors. In one early NEFA test, a tester was asked to be a grant recipient and find instructions for filing her final report. After struggling to find the information for about 45 seconds, she said, “By now I would have just picked up the phone to call someone.” Are you appropriately sobered by the fact even a user who is contractually obligated to do something on your site might give up after only 45 seconds? While NEFA does offer phone support, their grant recipients are happier if they can just find what they need online.

Excited yet?

We hope you’ve caught our contagious enthusiasm for quick & dirty usability testing, and that you better understand the benefits of adding usability testing to your next web project. Feel free to get in touch if you have questions!

Share the Love