Why Doing User Observations First is Wrong - 23 August 2006
by Don Norman
Read this article in Chinese (translated by Kyle Chan, proofread by Christina Li).
Article published with the permission from the author and ACM.
How many times have you had to fight hard for the ability to do field studies and other observations at the very start of the project? How many times have you patiently explained that taking time now would be rewarded by faster time to market overall? And how many times were you successful? The HCI community has long complained about product processes that do not allow time to start with good observations.
The more I examine this issue, the more I think that it is we, the HCI community, who are wrong. This includes me, for I have long championed the “study first, design second” approach. Well, I now suggest that for many projects the order is design, then study.
Let’s face it: Once a project is announced, it is too late to study what it should be – that’s what the announcement was about. If you want to do creative study, you have to do it before the project’s launch. You have to be on the team that decides what projects to do in the first place – which means you have to be part of the management team. (HCI bug one: Not enough HCIers are executives.)
Most projects are enhancements of preexisting projects. Why do we have to start studying the users all over again? Haven’t we already learned a lot about them? Shouldn’t we be studying them all throughout the adoption period? Once a project starts, it is too late. Think about it.
But note this contradiction, too: All of us usability theorists have long argued for iterative design, trying to get rid of the lengthy, inflexible linear project schedules that stymie flexibility and change, that slow up projects. Instead, we have championed iterative design, with frequent, rapid prototyping and frequent, rapid tests.
But wait a minute. Our continual plea for up-front user studies, field observations, and the discovery of true user needs is a step backwards: This is a linear, inflexible process inserted prior to the design and coding stages. We are advocating a waterfall method for us, even as we deny it for others. Yes, folks. By saying we need time to do field studies, observations, rapid paper prototypes and the like, we are contradicting the very methods that we claim to be promoting.
The programming community has long struggled with similar issues. They too are trying to eliminate the lengthy, inflexible, linear schedules that slow up projects. They are experimenting with a variety of new methods, for example Agile, XP (Extreme Programming), and other rapid, iterative programming methods. Hurrah for them.
The linear project procedures (also called “waterfall methods”), with lengthy setting of objectives, followed by design, coding, and then test, are dead. Thank goodness. The new programming styles practice iterative design and promulgate multidisciplinary teams: everything we should be striving for. Now it is time for the UI community to follow their lead – to do what we ourselves have been preaching.
Field studies, user observations, contextual analyses, and all procedures that aim to determine true human needs are still just as important as ever – but they should all be done outside of the product process. This is the information needed to determine what product to build, which projects to fund. Do not insist on gathering this information after the project has begun. Then it is too late; then you are holding everyone back.
Once the project is underway, it is going to be severely constrained by time and resources. That’s a fact of life: All product teams feel these constraints. Our goal is to work in multidisciplinary teams to produce effective, pleasurable designs rapidly and efficiently. Design first, on day one, if you can. Review, test, and redesign. Let the programming and marketing teams know how the product will look and behave at the very start of the project. Have faith in our ability to design well-crafted, understandable interfaces and procedures rapidly, without the need for (lengthy) research. Become an essential part of the team, so our input comes simultaneously with everyone else’s. Fortunately, that’s a component of the new programming methods.
One correspondent to a debate on this topic on a CHI mailing list said: “Nothing I have seen or read suggests that agile programmers are the least interested in slowing down their scrums (a method for project management) meeting waiting for usability testing to happen. There is no utilization of user research at all in so-called agile usability practices – it’s just a game of guessing what users need.”
This correspondent is confused. Usability testing is like Beta testing of software. It should never be used to determine “what users need.” It is for catching bugs, and so this kind of usability testing still fits the new, iterative programming models, just as Beta testing for software bugs fits the models. I have long maintained that any company proud of its usability testing is a company in trouble, just as a company proud of its Beta testing is in trouble. UI and Beta testing are meant simply to find bugs, not to redesign.
So let’s separate the field and observational studies, the conceptual design work, and the needs analyses from the actual product project. We need to discover what users need before the project starts; for once started, the direction has already been determined. We need to embrace rapid, iterative methods. We need to fit the new procedures used by the programming teams, and we need to become team players. What’s especially nice about these new methods is that they have made room for us: They explicitly acknowledge the importance of HCI design. Everyone wants us on the team, but only if we won’t slow down the work. More power to them.
Note: Scrums is “a method for project management.” See Wikipedia or your favorite online information source.
Don Norman wears many hats, including co-founder of the Nielsen Norman group, Professor at Northwestern University, and author, his latest book being “Emotional Design.” He lives at www.jnd.org.
Buy books by this author from amazon.co.uk
Possible Related Articles: