Design evaluation, usability, and accessibility

Evaluation is a cycle, not a sign-off

Once a working interface prototype exists it has to be evaluated against the people it was built for. The cycle is short and repeats: the design model produces prototype n, a user works with it, the designer studies what happened, modifications are made, and prototype n+1 appears. It ends when there is nothing left worth changing — not when the calendar says so.

Evaluation spans a formality spectrum. At the informal end a user takes the prototype for a test drive and tells you what they thought. At the formal end you run a designed study over a population of users and analyse the questionnaires statistically. Both are legitimate; they buy different things, and cost differently.

You can also evaluate before a prototype exists, which is much cheaper. Four early indicators come out of the design model itself. The length and complexity of the specification of the system and its interface indicates how much learning you are asking of users. The number of tasks, and the average number of actions per task, indicates interaction time and overall efficiency. The number of actions, tasks and system states indicates the memory load you are imposing. And the interface style, help facilities and error-handling protocol indicate the overall complexity, and how likely the thing is to be accepted. None of these is proof. All of them are available on a Tuesday afternoon with no users in the room.

Once there is a prototype, two kinds of data are worth collecting. Qualitative data comes from questionnaires and rating sheets — what users thought. Quantitative data comes from watching: tasks correctly completed per unit time, frequency and sequence of actions, time spent looking at the display, number and type of errors, error recovery time, time spent in help and number of help lookups. The quantitative data is what lets you say a change worked.

Usability

Usability is a measure of how well a system helps people learn it, helps them remember what they learned, reduces the likelihood of errors, lets them work efficiently, and leaves them satisfied. Note what is absent from that list: beauty, cleverness, and state-of-the-art interaction. Usability arises when the structure of the interface fits the needs of the people using it, and the only way to know whether it exists is to watch people use the thing and ask questions like: can they work without continual help, do the interaction rules let a knowledgeable user move quickly, is the user always aware of the system state, does the interface anticipate errors and help correct them, is it tolerant of the errors people make.

Accessibility

Accessibility asks a blunt question: can somebody who is blind, deaf, hard of hearing, limited in mobility, affected by a cognitive impairment, or simply older, actually use this? The WCAG guidelines answer it with four properties, remembered as POUR — content must be perceivable through more than one sense, the interface operable by more than one input method, both of them understandable rather than merely present, and the whole thing robust enough to survive the assistive technology the person already relies on. It is an obligation for ethical, legal and commercial reasons, and it is judged against those published guideline sets rather than against a designer's taste.

Four recurring design issues turn into accessibility problems if they are left late, and they degrade the experience for everyone, not only for users with disabilities. System response time has two characteristics — length, and variability; variability is the more damaging of the two, because an unpredictable system prevents the user from establishing any rhythm at all. Help facilities must answer a question without making the user leave what they are doing. Error handling: every message should describe the problem in the user's language, offer constructive advice for recovering, state any consequences so the user can check for them, be accompanied by an audible or visible cue, and never blame the user. Menu and command labelling must be self-explanatory in context and consistent across the product family. Internationalisation belongs in the same bracket: build a generic core and localise it, rather than designing for one locale and retrofitting the rest.

The distinction that carries the marks

Usability asks: can a competent user achieve their goal efficiently and be satisfied doing it? Accessibility asks: can a person with a disability use this system at all? They are not the same question, they are not measured by the same instruments, and their fixes are different — usability fixes tend to remove steps, accessibility fixes tend to add correct structure and alternatives. A product can be superb on one and fail the other completely.

Worked example — BorrowBox at the locker bank

The collection screen scored well in usability testing: median time from app launch to open door was eleven seconds, and eight of ten first-time members completed collection without help. Then a member who is blind tried it. The locker number was rendered as an image of a seven-segment display with no text alternative, so the screen reader announced nothing. The 'open' control was a coloured region with no accessible name. The QR camera view gave a purely visual aiming reticule. Every one of those is invisible to a usability metric measured on sighted members, and every one of them makes the app unusable rather than slow.

The fixes were different in kind from usability fixes: give the locker number a text alternative and announce it, give every control a real accessible name and a target large enough to hit without fine motor control, add an audio cue when the QR code is in frame, and — because the environment matters too — make sure contrast survives direct sunlight and that nothing depends on hearing a click in a noisy street. Note that the last two help every member. That is typical, and it is also why the two programmes get confused. They are still separate programmes with separate evidence.