School and District leaders are often cautious in applying "new" information because of their confusion between maintaining what might appear to be clinically correct "data driven" choices, and making thoughtful "evidence informed" decisions.
Data Driven decision making has gained wide usage in business, politics, and some educational jurisdictions through dependence on: market surveys (what the customer wants); targeted short-term goals; productivity markers; profitability indicators; strategic trade-offs; and computer modelling. In schools, it often rears its head in attempts accede to the demands of assertive parent and special interest groups; union demands; or, system or province-wide benchmark targets. By contrast, evidence-informed decisions, which are hopefully more common in schools, focus on: long-term trends and outcomes; maintaining the confidence of stakeholders (parents, staff, students, central administrators, community, etc.) over time; and personal communication and community values and sensitivities. Quick fixes are often data driven, while more strategic long-term change needs to be informed, not driven, by the evidence and grounded in solid educational philosophy and societal values.
One of the great, apocryphal case studies about the dangers of narrowly relying on the "data" is the story of the Ford Pinto. The Pinto hit the car market in the early 1970s (a colleague of mine had one). It was billed as a sporty compact and targeted the first-time car buyer. Unfortunately, Ford's own safety research revealed that, due to the placement of the fuel tank at the back of the car, a rear end collision at speeds over 20 mph could potentially cause the car to explode. Ford was faced with the prospect of recalling all of its Pintos for refitting (resulting in additional costs and negative publicity) or gambling that it wouldn't be that much of a problem.
In order to make this major decision, they called in the data-crunchers. The results of their analysis was as follows:
Option #1: Recall the already manufactured vehicles
Vehicles affected: 11 million cars, 1.5 million light trucks
Refit cost:$11 per car, $11 per truck
Total cost:12.5 million X $11 = $137.5 million
Option #2: Leave vehicles on the road and deal with legal liabilities
Estimated impact:180 burn deaths, 180 serious burn injuries, 2,100 burned vehicles
Incident cost:$200,000 per death, $67,000 per injury, $700 per vehicle
Total Cost:(180 X $200,000) + (180 X $67,000) + (2,100 X $700) = $49.5 million
It's unknown what discussions were held behind closed doors at the time, but ultimately Ford made the "data driven" decision to go with the less expensive Option 2 and settle lawsuits as they arose. It is estimated that between 23 (Ford's assertion) and 500 (critics' claims) deaths were caused by this design flaw. Settlements far outstripped projections and the unnecessary loss of life was tragic.
A half-century later, it is easy for us to raise our eyebrows at the seemingly callous decision-making of the time. Similar contemporary moves by Big Tobacco and currently by the U.S. politicians with respect to gun control continue to mirror the thinking behind Ford's action.
However, before we look down our noses at these other obvious examples, we should consider how many school systems and individual schools have taken the same path with respect to literacy and numeracy. To what extent are decisions made in our own industry, not because of the evidence, but by administrators using the "Pinto" model? After all, it is easy to tabulate how much money they have invested in reading and math packaged programmes; texts and workbooks; professional development; consultants' fees; etc. supporting initiatives which don't really work. It is also easy to reflect on the social and reputational cost to a school when it admits that they have been doing it all wrong for years.
What's harder to tabulate is the learning loss and educational growth squandered for successive cohorts of kids going through a flawed system year after year.
This is our Pinto moment. We need to make better choices.