Written by Gen
With the exception of enjoying a beautiful summer’s day, or avoiding the piles of laundry that accumulate throughout the week, I don’t care to waste time. I have prioritized the important things in my life and delegate my time accordingly. My family life has routines that help us to spend the majority of our time bonding and not so much time completing menial tasks. Spending time on things that don’t work, or don’t accomplish meaningful goals is a true pet peeve of mine, and yet I see it happen every day… in the field of literacy education.
I’m afraid I may have become known as “the girl who cried ‘research!'” to those who know me well professionally. When the next latest, greatest program, screening tool, or resource causes a hype, the last thing I do is read about the product. The first thing I do, however, is to go look for the research. Have you ever noticed that so many products designed to “cure” literacy ailments often have the words “research-based” in a brightly colored star-shape on the front cover, outside of the box, or scrolling across the top of the website?
In my quests to find the truth in literacy research, I came across the article, 10 Things Every Literacy Educator Should Know About Research, written by Nell K. Duke and Nicole M. Martin. This article outlines the difference between research-tested and research-based. Here is a small excerpt among many other important points:
“Let’s say that someone develops an approach to teaching reading comprehension that combines elements of three previous, research-tested approaches. We would say that the new approach that has been developed is research-based, but we would not say it is research-tested because no one has tested the impact of that particular combination of
approaches in research. The combination could be more effective, as effective,
or less effective than the individual approaches on which it was based (Duke & Martin, 2011).”
I once saw a preview for a biographical movie, one of those tearjerkers about the hard-knock life of a kid who makes it big later in life after overcoming adversities. Then I noticed that the preview said, “based on a true story.” Ugh. There’s that phrase again, “based on.” Therefore, I diligently looked up the subject of the movie to see what he had to say about how he had been portrayed. His comments weren’t flattering to the film, and he more than implied that the events in it were practically fictional. I still saw the movie, but I had a realistic expectation of what I was watching.
So, when choosing literacy curriculum, educators need to know that when the box says “research-based,” the program has the potential to be less effective than what the research has actually tested. This suggests to me that, before making a costly curriculum purchase, stakeholders ought to seek out the original research in order to make informed decisions about whether or not the curriculum seems to have the original research’s actual findings at the heart of the materials.
How do you find the original research? Well, it’s harder than it sounds, sometimes. Over the last few years as the pendulum has swung, rather quickly, towards a less complex theory of reading, I have found myself searching for the original research for many “cure-all” literacy programs and here’s what I’ve found:
- Real is usually obvious. If it truly is based on research, it will be obvious. Websites or materials will have links to the research or they will list it in their sources.
- Do your due diligence. If it’s not obvious where the original research is, comb through the materials to find citations and references. Then you’ll have to plug that information into a search engine and pick through the results for reliable research. What are the characteristics of reliable research? Well, Duke and Martin (2011) explain that too:
- A statement of the research
question(s) and/or purpose(s)
- A rationale for the study (i.e., information
about why the question was
asked, the problem the research
was meant to investigate or address,
and the research and theory that
- A description of the methods
used to collect data to address the
- A description of the methods used
to analyze the data collected to
address the question(s)
- Results of these analyses
- Conclusions that the researchers
have drawn based on the results
- Implications of those conclusions
- Limitations of the study
- Directions for future research
- A statement of the research
- Face reality. If the cited “research” doesn’t have the above characteristics, continue your search for quality literacy curriculum. Or at least come to terms with the fact that the curriculum or program is not actually based on research.
- Check the date. If you do find real research, check the date. I’ve found a lot of programs that are technically based on research, but that research was conducted in the 1970s and hasn’t been updated or confirmed by more recent studies or developments.
- Use multiple data points. Do not rely solely on informational materials provided by the publishing company, sales representatives, or people employed by the program. Even if the company is not-for-profit, their work relies on making money, otherwise the products would be free. Therefore, they will show you only what makes the program or curriculum look good. Here are a few other things to look for:
- The original research. If you can’t find any, that should tell you something.
- Independent testimonials. Not the ones posted on the company’s website. If you can’t find any, that should tell you something.
- Student performance results on standardized testing. If the program hasn’t been around long enough to have this information, that should tell you something. If it has been around long enough, but this type of data doesn’t exist, that should tell you something.
Our students deserve the best literacy teaching we can possibly provide. They deserve the time it takes for educators to make truly informed decisions based on real research. Research can be difficult to read, difficult to understand, but our students and their families are relying on us to do what is right. Search your staff for those who have a penchant for reading research, ask publishers hard questions, ask for the research. Our students’ literate lives are only as good as their educators’ diligence.
If you haven’t already read Nell K. Duke and Nicole M. Martin’s article, 10 Things Every Literacy Educator Should Know About Research, I urge you to find it, read it, and read it again!
Duke, N.K. & Martin, N.M. (2011). 10 Things Every Literacy Educator Should Know About Research. The Reading Teacher, Volume 65 (Issue 1), pp. 9-22.
**Congratulations! If you’ve scrolled this far, you’ve done your due diligence and have checked my sources. Leave a comment if you think checking sources is important!**