SELECTED PROJECT:
Tailored Student Assessments
@ NWEA | 2021-23
Ability to “test the tests” delivers a tailored curriculum for every student.
The Need
We partnered with NWEA to build a learning platform to create tailored academic assessments for students pre-K-12.
Services Provided:
Consulting and Advisory Services to Align
Product Strategy with Business Case and Commercial Models
Product Definition, Discovery, and Research
Design and Development of pilot in just 9 months
Validate with User Research and Testing
Long-term Roadmap and Investment Strategy
NWEA assessments are currently used by over 9,500 schools and districts in 145 countries.
The Work.
Product Vision: A short-cycle assessment tool that uses student data to power personalized learning solutions — this product is named Ontos.
On average, teachers have more than 25 students per class. Each of those students has their own learning needs. Ontos not only shows what a student’s needs are, it highlights the strengths that teachers can leverage to foster their growth.
Together, we can create a personalized learning ecosystem that gives students and teachers the best resources at the right time, empowering them to thrive.
If an Ontos-powered hybrid ecosystem existed today, 34M students could benefit.
Since 2018, Tyton Partners has continually validated a shrinking TAM for assessment only and curriculum only solutions and a growing market for an assessment solution, integrated with a curriculum partner.
A conjoint Ontos pricing study found that 60% of the TAM will buy a hybrid ecosystem solution from a curriculum publisher.
Discovery & Delivery Process
After a short initial discovery effort collaborating closely with NWEA’s Research team, we began work on Delivering the Lab Platform and in-class Assessment products of Ontos. The Lab Platform allowed NWEA Researchers (and other third-party organizations) to create assessments based on a proprietary ontology and knowledge base driven by NWEA Map results as well as continued inputs from the Ontos system, school districts and machine learning. The in-class Assessment tool would be utilized by teachers and students in class to aid in more individualized and custom learning for students over time.
How we work & why it matters
We have cross-teamed between learning scientists, measurement scientists, engineers, and designers to create an instructionally relevant map of student knowledge development.
We move from nascent exploratory research to strongly confirmatory research in real classrooms to justify the claims of our assessments.
We iteratively design and test based on current students who help determine if there is meaning in our data.
An explicit model of cognition
The ontology represents an instructionally relevant model of how student thinking evolves as they learn.
Unlike content standards, which vary in grain-size and have no structure, the ontology’s value is its specificity of knowledge and how that knowledge is connected.
It makes explicit the essential knowledge critical for day-to-day learning and how that knowledge progresses, converges, and diverges.
The structure of the ontology
The ontology will drive our test design, personalization, reporting, and recommendations.
Because the ontology makes explicit how knowledge builds, it enables an asset-based approach that goes beyond identifying what students can’t do; Ontos identifies the knowledge students have to help prioritize interventions that will maximize their learning.
Methods & Activities
50 interviews and 2,000 field tests led to our measurement model ready for initial validation.
Released test presenter app proven with the Google classroom demo, designed to integrate assessments into any platform.
Validated initial end-to-end workflow with teachers and interventionists across five states.
Engineered a Proof of concept for demonstrating LTI interoperability of Ontos assessments.
Designed prototypes of core product solutions to be further tested with potential users.
The team did continuous discovery and delivery throughout the project.
Released an app for conducting interviews and field tests with students.
We engineered technical proofs of concept to integrate with actual software including Google Classroom
The Outcome.
NWEA and the team have validated both products, the lab assessment platform and the test presenter, directly with researchers, teachers, and students. NWEA has engaged third parties to conduct research around product desirability, price elasticity, and business viability.
The Ontos team is positioned to establish an overall product strategy that maximizes improvements to student learning outcomes and yields profound business opportunities.
Value proposition and pricing models are validated
At each horizon of innovation for Ontos, NWEA has data confirming the per-seat cost that districts would be willing to pay.
Brand is not a key factor
District buyers indicated that they would be willing to pay for capability, with no regard for which company would provide it.
Development should align to state adoption cycles
Recruitment, research agenda, and development should align to the buyer needs in priority states.