Usability Evaluation Considered Harmful

Usability Evaluation Considered Harmful

Saul Greenberg, Bill Buxton

Comments:
Comment #1
Comment #2
Comment #3


Summary:
It has been almost required by the CHI community to have usability evaluations on any project and the results of these evaluations are used to judge and rate any novel ideas presented by a project. The authors of this paper suggest that not only are these evaluations unnecessary at times, but it can even be detrimental to a project.

If an idea receives a bad evaluation in a usability study, it is pretty much dead in the water. This can kill a project before any novel ideas have really been fleshed out. There is no need to evaluate an idea before it has been fully developed. It has not reached its fully potential yet and, if the evaluations are bad, it will never reach that full potential.

There is also the chance that an idea, even if it is an improvement over the standard conventions can be poorly rated against the standard conventions in a usability evaluation because the users are already comfortable with the standard conventions. This will also kill a potentially great project because a usability evaluation was used.


Discussion:
I enjoyed this paper and fully understood what the authors were saying. If a project is directed by the numbers that are given to it in a usability evaluation, the developers of that project are prone to give up on it and call it a loss. This can be very detrimental when you consider the fact that, if the developers had spent a few more hours, days, or even weeks enhancing and polishing their novel ideas, it could have been a very successful project. This is not to say that usability evaluations are always harmful, sometimes they are still necessary. It will take discernment on the developers' part to properly decide whether or not an evaluation will be necessary or beneficial.