How can a team test usability beyond a vendor demo?

Discussion

• Author
  Posts
• 1st August 2026 at 7:16 pm #3243

  NNick Oscar

  Editor

  A guided demo follows an ideal path. Daily work involves normal permissions, imperfect data, interruptions, and mistakes. What should the evaluation team test itself on?

  1st August 2026 at 7:38 pm #3244

  NNick Oscar

  Editor

  Accepted editorial answer

  Put representative users in control and give them ordinary tasks to complete without coaching from the presenter. Use realistic roles, permissions, and data. Include less-polished work, such as correcting a record, finding an older item, recovering from an error, exporting data, and finishing a task after an interruption.

  Record completion, time, errors, assistance, and points of confusion. Note whether the test used a production-like setup or a simplified demo account. Digital.gov recommends planned scenarios, observation, and a rolling issues log. NIST’s usability definition focuses on effectiveness, efficiency, and satisfaction in a specific context.

  Report functional availability and observed usability separately. A feature can exist and still be difficult to find or use. Also list any important workflows that the team did not test.

• Author
  Posts