our blognotes from san francisco: testing 1‑2‑3
6 May 2009, 06:39
This second blog posting about the openPrinting summit in San Francisco covers the usability testing of the printing dialog. First, a few words about usability testing.
I have worked with people who saw a usability test as a kind of final school exam: better send something into the test that will not rock the boat. This kind of fear surely stamps out any kind of interaction design innovation. Under those circumstances I say: why bother doing this project at all?
hard knocks
Experienced software developers appreciate working with a dedicated software test team. Yes, these testers aim to take everything apart, but the resulting software will be a lot better when it hits the streets. Similarly, I enjoy working with a usability team.
The usability test delivers hard‐hitting facts, but these flow straight into debugging and adding depth to my interaction designs. And when the test says it works, then my work is validated and that marks the end of doubt and discussion.
before the test is already a test
The usability test was performed by Jan Mühlig of relevantive, who also headed up the first involvement of openUsability with openPrinting in Atlanta, 2006. Although I took over the project at the Lexington summit in the same year, Jan has been closely involved at different phases of this project.
Straight away the preparation meeting was a hoot. For interaction architects, having a frank, uncomplicated discussion with professionals who have empathy and usability experience is both a pleasure and highly valuable. Jan’ feedback triggered a redesign that Kate, my associate at m+mi works, and I performed during our preparation of the test material.
agile like paper
The test was performed using a paper prototype. I played ‘computer’ in most of the test sessions, driving the dialog response to users’ input. Working this way allowed us to do three rounds of testing—with in between two further rounds of redesign—in exactly six days.
Being a usability researcher, Jan was not going to take for granted that any of our big, innovative concepts—no matter how sound they looked—was simply going to work. ‘Just print’; two levels of detail in the dialog; always a preview; quick presets; parameter tagging: he gave all of them a real workout during the tests. They had to prove to be a real improvement over the current state of printing dialogs.
the results are in
Walking you through the results, I will illustrate them with the latest iteration of the design, the one that went into the third test.
‘just print’
Printing still does not exist. At the start of test the participants were interviewed about what printing means to them and how they experience it. Again and again the answer is that the act of printing is not interesting, only the result counts. Here are some quotes:
‘What I think about is hard‐copies, really; a piece of paper I can look at.’
‘…but most often, I’ll just press print.’
‘I just click on print and… —[Jan:] the print appears by magic? —Yes.’
Although ‘printing does not exist’ is our number one guiding principle, it is striking to experience in the test how strong this force is. Our solution for supporting this is ‘just print,’ bypassing the print dialog altogether. Below you see the jury‐rigged file menu we used in the test:
I had put ‘Print one copy’ as a label in there for ‘Just print’ in the first test, but that did not work out. So, if you want to learn something, take some risk and use the test results. From the second test on the menu was as above. It fared better, but more testing is needed here before setting this label in stone.
preview
If ‘printing does not exist’ is a strong force, the print preview is huge. We are not talking here about participants getting along nicely with the preview and they knowing how to flip trough it. We are talking about seeing that for 100% of the participants the preview addresses a fundamental need.
Although it was my vision right from the beginning to design the print dialog around the preview, it is still amazing for me to see in the tests how deep it has an effect on users. So deep, that participants of the ‘just print’ persuasion started going through the print dialog more after seeing it—and the preview—only once. It is a huge effect to have the voodoo taken out of printing.
Now as an interaction architect I expect that long‐term—after the printing dialog has built a new level of trust in the print process through the preview and other factors—users will start taking up the offer to ‘just print’ for boring, predictable print jobs. But at any moment: if there is a hint of doubt, then the preview rules. Below we see the preview in the compact, level 2 dialog:
We added to this level the only universal printing parameter in the world: number of copies. If there is one thing we learned from this whole test cycle is that the way to the print is through the preview. The preview is the last thing to check and the Print button is logically located just below it.
tagging
Because of a lucky labelling flaw in our test set‑up, all the test participants went through the situation where the first tag they chose did not deliver the printing parameter they were looking for. After selecting a second tag they did not get the familiar situation of seeing different parameters, they got more of them (all parameters referenced by both tags). Then in 100% of the tests the conversation was the following:
Jan: Is this surprising?
Participant: Yes.
J: is it negative or positive?
P: I think it is positive, it is good that these are shown together because […].
Every participant formulated the last answer in a different way, citing a different reason why it was an improvement. But it was striking how the pattern returned in every test. Based on that Jan declared that he does not have to see dozens of tests more to know whether it works, it simply does. Below we see the full‐fledged level 3 dialog:
The whole tags + parameter section functions as an extension of the level 2 dialog. What is new is that the tags are now sectioned, along the lines of old‐fashioned functional vs. new user needs. Twelve of them in a single grid was simply too much. Next up for us is a parameters on the left vs. on the right design exercise and further work on the individual controls.
trouble
Not everything went that smooth from the beginning. I will illustrate that with this level 2 dialog that went into the first test (thus, now an obsolete design):
The test showed the following problems:
- participants had trouble identifying what the quick presets (on the left in the dialog) really were: a preset controlling all parameters or a single parameter settings?
- participants had trouble getting to level 3 in the dialog (via the More control button);
- participants were looking to ‘open up’ a quick preset to show what the actual implications (parameter settings) of it were;
- participants were looking to ‘open up’ a quick preset to show the printing parameters.
To find out what was going on we did a full‐risk redesign for the second test. That fared even worse. But now we knew what was going on:
- we were relying on the actual labelling of the quick presets to communicate what they actually were;
- our labelling was not good enough and since in production the real labels will be supplied by the printer manufacturer, they will never be perfect;
- the preview does not contain enough information to show what a quick preset completely entails;
- the information structure in the dialog was not clear: we were not showing that a quick preset encapsulates a full set of parameter values and thus the whole print.
into the light
To see how we solved this puzzle, let’s have a look again at the level 2 dialog that went into the third test:
The quick preset—now called ‘Current setting’ for users—now serves as a banner for everything but the printer. Under it, what is an essential print setting and not ascertainable from the preview is listed in short‐hand for a more complete overview what a quick preset will accomplish.
Furthermore, a change that meant the end of a principle that I had long cherished for the dialog: visibility and 1‑click selection of quick presets. With the limited reliability of quick preset labelling, the visible listing of them has limited value. Making the choice of quick preset a pop‑up enables to run it as a banner, closing the circle.
This design tested well in the third test. This means the big concept bugs have been debugged and fixed. For the actual quick preset pop‑up I have some innovative ideas in mind that I want to see explored. If they ever make it into the design, they sure will need usability testing, of course.
encore: labelling insights
While preparing for the first test, we spend some time straightening out the tagging used in the dialog. There we discovered some subtle factors in the labelling. Some of the tags had labels that looked like an end‑goal (‘fit to paper’), instead of a range where users can find their own optimum (‘paper fit’). The former gives the impression of a quick preset and is thus incorrect.
Similarly we have seen that quick presets can give the impression to be a single parameter setting. Although users have every right to store such ‘single issue’ presets, factory‐shipped presets should target broader goals and be labelled as such. ‘duplex: on’ is a very bad preset label, ‘good enough for jazz’ is a good label.
That’s it for dialog usability testing. Next up: the trouble with infrastructure.
Labels: design stage, openPrinting, practical
1 comments · post a comment
- at 19 June, 2009 20:02,
Auria commented - http://ls.berkeley.edu/lscr/advice/using/printdialog-macosx1.gif
Main dialog looks kinda familiar ;)
I like the design, but it bothers me a bit that which pages to print wouldn't be readily available. If it remembers my wishes (or, even better, if there's a way to change which options are visible by default) then it sounds great
If you like to ask Peter one burning question and talk about it for ten minutes, then check out his available officehours.
Peter Sikking is an
experienced
interaction architect. He unabashedly puts
shipping great products at the centre of his design practice. He is known for
his work for Google, Nokia, the
Metapolator &
GIMP
open source projects,
and startups.
What is Peter up to? See his /now page.
- info@mmiworks.net
- +49 (0)30 345 06 197
- imprint
