The education service industry is awash in marketing materials that tell stories of impact. Most of these materials come in the form of “success stories.” Curriculum publishers and edtech organizations like to disseminate success stories because they are quick and cheap to produce. This can be a useful approach for early-stage organizations who don’t yet have the resources to invest in formal research – which, when done correctly, should cost hundreds of thousands of dollars over a multi-year period.
As we have seen, success stories often include:
- A narrow, focused set of results from an individual district, school, grade level, content team, or classroom.
- A presentation of quantitative and qualitative results that are descriptive (non-correlative) in nature. Most of the quantitative data come in the form of graphs that articulate increases in student test scores (from pre to post, and usually without a clear research design or consideration of appropriate statistical controls). Most of the qualitative or anecdotal results may include positive responses on teacher or student surveys, or narratives that the product was “helpful for teacher understanding of problem-based learning”, “improved student discourse,” or “ improved student productive struggle”.
But not all success stories are created equal. District and school level decision makers who seek to interpret these success stories may lack guidance on what comprises a meaningful story of impact. And education service companies may not know exactly what is worth including in these stories that help communicate how their product or service translates into teaching and learning.
Below we offer a few “Look Fors” with Guiding Questions to help educational decision makers to better evaluate and interpret “success stories” published by curriculum and edtech organizations. These criteria may also equip those companies to produce stories that communicate their successes.
Have any thoughts about these criteria? Or additional suggestions? Please reach out to Dr. Danks!
What to Look for in a “Success Story”
Population and Context
Evidence of Impact
Magnitude of Impact
Scope of Impact
