Testing Triumphs: Planning and Executing an Alpha Test
Building on the success of our previous efforts to revamp the entire Reading Horizons experience in K-3, I recognized the need to expand our testing initiatives to ensure the scalability and effectiveness of our product. Referencing the earlier case study, which outlined our journey to this pivotal moment, we acknowledged the importance of understanding how our solution would integrate into educators' daily routines and impact student learning outcomes. As the leader driving this initiative forward, my role was to define clear objectives, recruit diverse participants, set expectations, launch the test, provide support, and oversee evaluation and iteration efforts.
1
Defining Objectives
Working with multiple disciplines within Reading Horizons, we defined the following objectives for this alpha test:
2
Recruiting the Users
I spearheaded the recruitment process, working closely with our customer-facing teams to identify suitable candidates. From the 15 districts targeted, we selected 100 participants spanning four diverse districts. Our selection criteria prioritized:
These criteria facilitated the selection of a diverse group of educators, essential for comprehensive product evaluation.
Enable teachers to deliver engaging, effective reading instruction and differentiation tailored to students' needs within diverse settings.
Provide a user-friendly online portal serving as a single destination for all Reading Horizons needs.
Create an interactive experience for students that facilitates practice, reinforcement, and mastery-based progression.
Accurately measure student understanding of curriculum content and learning objectives through comprehensive assessments.
3
Setting Expectations
With a focus on transparency and communication, I developed comprehensive documentation outlining the objectives, timeline, and responsibilities for participants. Internally, our teams led frequent presentations and communications to generate excitement and alignment across all stakeholders. Externally, I fostered open channels of communication, creating a supportive environment for educators to voice their concerns and provide feedback.
4
Alpha Launch
As the anticipated launch date approached, internal and external announcements to generate excitement and awareness were crafted. In-person training sessions were organized to equip teachers with the necessary skills and resources to integrate the new platform into their classrooms effectively. These sessions also served as valuable opportunities for feedback and refinement, ensuring we were quickly identifying initial hurdles.
5
Support
Throughout the testing period, I led a group of cross-departmental experts dedicated to providing ongoing support to participants, addressing any obstacles or uncertainties they encountered. Live Q&A sessions, dedicated customer success managers, and integrated feedback channels were instrumental in facilitating communication and problem-solving. By leveraging a combination of proactive communication and responsive support, I ensured that educators felt supported and empowered to succeed.
6
Evaluation and Iteration
Continual evaluation and iteration drove product refinement, informed by user feedback and data analytics. Weekly meetings convened to scrutinize analytics data, present customer insights, and discuss observations, leading to key refinements. Notably, enhancements were made to lesson formats and user-facing reporting, addressing usability inconsistencies and improvement opportunities identified during testing. In-person engagements with customers offered invaluable insights, fostering stronger relationships and deeper understanding of user needs.
Lessons Learned
Upon conducting a retrospective, I gained valuable insights that will help me improve for future tests. Better establishing more focused intervals for evaluation and iteration with an intentionally concise scope and sequence will allow for shorter, more effective iteration phases. Moreover, I highlighted the critical role of trust-building and robust communication in navigating significant organizational and market changes.
Final Thoughts
Leading this initiative required a collaborative effort, commitment to detailed planning, and a focus on relationship building and expectation setting. Through effective leadership and thorough communication, I fostered confidence in our solution and empowered our users to become our biggest champions.
For a different perspective and examples of our relationship with customers, please refer to the attached blog post.
