LULABS’ usability testing methods are similar to those used by leading
companies in the computer industry. In addition, we have leveraged our vast
product exposure and testing experience to develop several special evaluation
techniques. With our unique testing we are able to measure key metrics and make
concrete recommendations to three key audiences: technology buyers who are faced
with crucial purchase decisions; product designers who strive to make their
product easy to use; and marketing professionals who strive to find the right
placement for their ads.
LULABS
has defined three crucial areas to
evaluate the usability of a product:
Productivity
This objective measurement is the
percentage of tasks that testers successfully complete. Testers generally fail
to complete tasks because they are unable to finish in the time allotted or
because they abandon the task as too difficult to do.
User
Satisfaction Upon completing each task,
testers rate their satisfaction on a scale of one to five (five being the best).
The overall product satisfaction score is an average of all the tester ratings.
In measuring satisfaction, we use a scoring system similar to the Olympic
method: we discard the high and low score for each product on each task. A
satisfaction score is then produced for each task and then for the entire test.
Help
Each time the documentation or online
help system is used, we log whether it actually assisted the tester. This
objective measurement reflects the accessibility of the product's features,
which is represented by the percentage of documentation and online help that
actually aided the testers. Because today's hardware and software products are
so feature-rich, it is natural for users to rely on product documentation and
help systems.
LULABS
Testing Facilities
What looks like a single lab
is actually three identical test centers. A 20-foot-long control console packed
with state-of-the-art technology has space for three technicians, who monitor
testers in the lab. Rooms are acoustically rated for near-soundproof operation,
and a one-way mirror separates the control room from the test room, allowing
technicians to candidly observe the testers.
The control station uses the
latest professional audio and video equipment. From their console, usability
technicians control the remote cameras, video recorders, wireless microphones,
and mixers that document every test. A dozen monitors capture all the details so
that test supervisors and casual observers can see the big picture on twin
27-inch monitors through the external observation window.
These test stations are
capable of handling a wide variety of products. Featuring the most up-to-date
technology, test rooms are also set up with standard office furniture, plants,
and bookcases. The ergonomically adjustable workstations are equipped with
state-of-the-art systems, printers, and scanners.
To properly document each
usability test, we monitor every angle of the test with three separate video
inputs: a direct-video feed from each computer screen, plus two cameras on each
tester. One camera is placed overhead to observe the keyboard, mouse, and
documentation, and the other provides a clear picture of the tester's face to
capture reactions during the test. Technicians can pan, tilt, and zoom each
camera independently with a remote control. All three images are fed into the
mixer and combined with an audio recording of the entire test. The composite
result is preserved on high-resolution videotape.
High-tech surroundings aside,
it is important that testers feel comfortable. For this reason, testers face
away from the mirrored glass to reduce the feeling that they are being watched.
They generally become comfortable with their surroundings and begin to relax and
work efficiently in around 15 minutes.
