I want to explore evidence-based practice. Just what does this mean and why is it important? At one time, I probably would have assumed this topic would interest only research geeks like me, but all the hoopla over teaching elementary school kids to read with an evidence-based approach changed my mind. Governors and other politicians were talking about mandating that schools take an evidence-based approach. Forty states require early elementary kids to be taught reading in a certain way if the school they are enrolled in expected state money. Most of us became competent readers without such funding threats, but that is beside the point. Throwing the phrase “evidence based” around is not limited to researchers.
I use this example, even though I don’t intend to focus on reading, to show that the phrase has meaning and consequences. To reach my broader goal, which is to question a policy that would require students to take notes on paper, I want to consider the different types of research conducted and which are essential before we can assume a given approach has real value.
One more quick diversion based on an example I assume most would recognize. How do researchers get from a hunch that a given substance might have medicinal value to the meds we take when ill or to maintain our health? In medical research, scientists screen thousands of chemical compounds or biological molecules to find “hits”—molecules that interact with the target in a desirable way. Perhaps some bacteria in a petri dish die when the compound is introduced. Promising candidates undergo extensive laboratory testing in vitro (cell and tissue cultures) and in vivo (animal models). Finally, tests are conducted with humans to evaluate tolerance and side effects, and then the impact in achieving the desired goal. Even then, data continue to be gathered after release for general use to identify overlooked issues.
Switching back to reflections on how best to learn, an important practical question might be – at what point has the requirement of “evidence-based” been achieved? It does matter. Educational practice is frequently described as prone to drastic swings. With the issue of learning to read, at one point in time reading is taught with a strong focus on the use of context and word recognition and then the pendulum swings in the opposite direction and readers are expected to sound out words and even nonsense words to develop phonetic skills. Computers are exciting and valuable at one point and then distracting and inconsistent with learning basic academic skills at another. How does an evidence-based approach fit with such inconsistency? Perhaps more relevant, what goes wrong when a supposed evidence-based approach is abandoned for another supposed evidence-based approach? It turns out it is quite difficult to rigorously investigate applied educational practices.
Studying is difficult to study
I happen to be on the wrong side of the popular recommendation that notes should be taken by hand. I wouldn’t really care, except educators may require class notes be taken by hand, which I think should require a higher standard of certainty. All I can do is try to surface reasons to reexamine assumptions one might make.
Like the medical research model I identified, there is a fairly standard sequence. It is a different type of sequence based on the advantages and disadvantages of different types of research. There are two types of sequence. First, there is the ease and advantages of doing “lab” studies versus more applied studies. The second sequence involves what might be described as differences in internal versus external validity. Lab studies are best at establishing internal validity so one can be more certain of what caused what. Field studies are better at establishing external validity. Will a given treatment work as predicted when applied? With note-taking, college students are exposed to a short video to simulate a classroom lecture and are required to take notes using a keyboard or by hand, there is typically what is called a “filled delay” or distraction task to better focus on what is retained in memory, and then some type of performance task. Other research methods attempt to model actual classroom practice, by adding elements such as a longer delay (e.g, the performance task follows a week delay) and components of study such as the opportunity to review notes before the performance task. Eventually, researchers might study what students do in actual classroom settings. By considering these different methods, you can see the trade-offs between internal and external validity.
One difference between the sequence I identified in medical treatment and educational practice is that in medical applications all levels must exist before applications are allowed. When I describe my interest in taking notes on a computer, professionals I know may even cite well-known studies (e.g., Mueller and Oppenheimer – The Pen is mightier than the keyboard) to inform me that handwriting is superior and they are taking an evidence-based position. Yes, I have read that study. It does have a catchy title. I then ask if they know that in that study students listened to a 15 minute TED talk, the test was given after a short delay, and students did not study their notes. What do you think that study has to say about college classes in which students listen to maybe 4 weeks of 50 minute lectures and study their notes repeatedly before taking an examination? It is not that the study is without value, it is that it poorly matches the factors that would provide external validity. Digital notes have advantages – searchability, easy modification to fill in gaps and personalize – much more difficult with handwritten notes. A plausible case can be made for taking notes with a laptop and until a greater variety of research exists, I trust individuals to take notes in whatever way they find to be productive. Jumping on the bandwagon based on one component of the full scope of necessary research is likely attractive, but not justified. This is my point, we don’t really know that taking notes by hand is superior. If someone challenges your opposition, you don’t have to accuse them of being lazy, reading only the Abstracts of research studies, or worse yet relying on popular online posts opposing technology in classrooms. Just ask if they have read the Methods section of the note-taking studies they think are informative.
