How can we avoid bias in the testers of a proposed systems? How can we assess functionality when many of our users have preferences for the form of one system over another? We devised a blind taste test – as close as we could reasonably achieve. As we approached the selection of the school’s main education technology platform, portal, intranet and communications system, we knew that many staff would have loyalties, preferences or apprehension toward particular candidate systems. We engaged in substantial process to bring the right candidates forward: gathering criteria from a wide range of stakeholders, weighting criteria according to stakeholder priorities, and grading software on each criteria to establish a well-justified shortlist. We then established three test sandboxes, populating them with similar test data including both sample content and user accounts, and “branding” them similarly with school colours and styling. We wrote test scripts to guide unfamiliar users through the testing of each function that had been prioritised by stakeholders. Our testers were selected by managers in areas across the school and included teachers, support staff, parents, multilingual users and students. In this presentation we’ll describe the delivery and execution of the testing plan, key results, and reflections on the process.
