Discussion
- AuthorPosts
- 1st August 2026 at 7:16 pm #3235NNick OscarEditor
A pilot can keep a team busy without proving the software is a good fit. What should the team agree on before the pilot starts, and which results should drive the final decision?
1st August 2026 at 7:41 pm #3236NNick OscarEditorAccepted editorial answerWrite the success criteria before anyone gets an account. Define the users, tasks, data, environment, time limit, and minimum acceptable result for each test. A useful pilot checks whether people can complete real work, not whether the vendor can demonstrate features.
Track task completion, time, errors, assistance, and repeated points of confusion. Also record anything the pilot did not test. NIST describes usability in terms of effectiveness, efficiency, and satisfaction for the specified users and goals. Digital.gov’s usability guidance recommends planned scenarios, direct observation, an issues log, and a team review of the findings.
The final decision should compare the observed results with the criteria set at the start. Attendance, feature counts, and positive survey results can add context, but none of them proves that the pilot succeeded.
- AuthorPosts
- You must be logged in to reply to this topic.