Discovering the Illusion of Understanding

Many of you have taken physical science in middle school, and some of you likely took science again in high school. How often do you look back on your experience and find opportunities where you can draw some knowledge from those courses to answer a question. For example: is it possible to light a flashlight bulb with one battery and only one wire? If all the ice at the North Pole melted would the sea level rise? Would a cruise liner float in a bathtub just big enough to hold it?

As a former middle and high school science teacher I look back on my experience with questions of my own. What exactly did my students learn? How useful was the experience to them then and now? I didn’t get any sense of an answer until four after I had started teaching when one day I happen to looked into a drawer full of standardized tests that I had collected during my first four years.

During this period I had earned a masters degree in science education, attended all the workshops and professional development opportunities the district offered me. I had taken courses during the summer, participated in NSF projects on teaching and learning, and integrated the latest technology in my classes to conduct experiments, collect and analyze data with students. The superintendent had even asked a visiting professor of education to visit my classroom.

Later I found out that the professor told the superintendent that my lesson was a model lesson. So as I looked at the collection of tests I recognized my first educational research question and hypothesis: As I felt confident that my teaching had improved, shouldn’t I see an improvement in student scores over those four years?

You might be thinking what I was thinking, of course. As my school used Scantron forms for standardized tests, it was easy to calculate average scores for every test over every year. The numbers were stunning, so much so that I had to run the calculations again. The overall average scores had not changed by more than one or two percent in four years on the same tests, which included increases and decreases. I didn’t need a background in statistics to realize that in my very best classes students were not benefiting from what I had learned. In fact, it appeared that I was the only one learning science in my classroom.

More dramatically, we all appeared to have been participating in a complicated play called, School Life. While I was pretending to teach, my students were pretending to learn. Of course, none of us realized we were part of a play. We were simply performing in the roles that we had learned to play as students first and teachers later. We all wanted to do our very best, and eventually our acting had become real to us.

Although I was very disappointed with the findings, I also became very interested in understanding how this natural experiment went awry.