Assessment

Using Student-Generated Reading Questions to Uncover Knowledge Gaps

By Erika G. Offerdahl, PhD and Lisa Montplaisir, PhD

* The following is an excerpt from Student-Generated Reading Questions: Diagnosing Student Thinking with Diverse Formative Assessments, Biochemistry and Molecular Biology Education, 42 (1), 29-38. The Teaching Professor Blog recently named it to its list of top pedagogical articles.

As instructors, we make a myriad of assumptions about the knowledge students bring to our courses. These assumptions influence how we plan for courses, what information we decide to cover, and how we engage our students. Often there is a mismatch between our expectations about what students know and how students actually think about a topic that is not uncovered until too late, after we examine student performance on quizzes and exams. Narrowing this gap requires the use of well-crafted formative assessments that facilitate diagnosing student learning throughout the teaching process.

Within large-lecture courses in particular, instructors have traditionally relied on the use of verbal questions to gauge student learning. Verbal questioning is limited, as it reveals the thinking of only those students most willing to respond. Often these are the high-performing students in a class. In contrast, student-generated reading questions (SGRQs) provide the opportunity and incentive for all students to submit questions, providing the evidence necessary to make inferences about the range and extent of all students' conceptions. As evidenced through content analysis, SGRQs have the potential for characterizing the "conceptual ecology" of the class as a whole. While formative assessment is not a new idea, most research on its effective use in undergraduate science courses has focused on implementation in introductory courses and been limited to pedagogies that make use of clicker questions. This exploratory study provides preliminary data to spark a conversation about the diverse ways in which we can effectively assess student understanding in ways that support conceptual development.

Methods 

The context of the study was an upper-level, large-lecture biochemistry course offered at a research-intensive university in the southwestern United States. This was the first course of a two-semester sequence for biochemistry and cellular biology majors and was team-taught by three instructors. Pre-requisites for enrollment in this course were introductory biology, general chemistry, and organic chemistry.

A regular reading question assignment was integrated as a formative assessment prompt to collect evidence of student thinking in the form of SGRQs. Students were instructed to approach each reading assignment with the goal of achieving deep conceptual understanding. We expressed our expectation that through this process, students would certainly think of at least one question relevant to the material at hand. Students were instructed that their questions should not focus solely on factual material; rather, a reading question should also describe what conceptual problems the individual has with the material and how the individual arrived at that question. They then submitted the SGRQs electronically to the instructors prior to a lecture on the topic.

Eleven reading questions submissions were collected from each student throughout the semester. Submissions were often one or more paragraphs long and sometimes included more than a single question. Each submission was worth a maximum of three points. Students were allowed to drop one reading question score. There were a total of 700 points possible in the course; reading questions counted for 4% of the total grade.

To characterize the utility of the reading question assignment in producing high-quality evidence of student thinking, we analyzed the resulting SGRQs as related to three learning outcomes (Fig. 2). These learning outcomes reflect basic skills that are likely to be articulated in syllabi in the molecular life sciences. The unit of analysis was the student submission in its entirety. Often, students posed more than one question per submission. Therefore multiple codes could be associated with any single submission in each of the analyses described below.

Fig. 2

A common goal of undergraduate science courses is to develop students' understanding of and abilities to engage in scientific inquiry. Just as science is a process that sprouts from questions about the natural world, our own students must learn to approach inquiry by posing insightful questions. Therefore, a reasonable learning outcome in the life sciences might be to increase students' abilities to ask "good questions" — those that hold a kernel of a research hypothesis. We hypothesized that SGRQs might be useful for diagnosing students' questioning abilities. To this end, we revised an existing taxonomy for characterizing students' written questions in introductory biology and applied it to our SGRQs. The levels of the taxonomy represent a progression in students' questioning from superficial or definition-seeking questions to more sophisticated questions that synthesize information and more closely resemble those of practicing scientists.

Much of the assigned reading associated with this reading question assignment was a review of topics covered in introductory courses (such as natural selection and evolution in introductory biology) or explicitly reviewed earlier in the semester (as was the case of principles of chemistry). Yet the content analysis indicates that the concepts that many students are still actively trying to make sense of and build upon relate to fundamental ideas and specific topic areas with which these students were likely to have had significant prior experience. This was somewhat surprising given that this course is specifically designed for biochemistry and molecular biology majors, students who are often motivated to perform well in introductory courses. Although the conceptual snapshots revealed by this analysis certainly represent possible entry points for instruction, they also highlight how these entry points might be different than those an instructor may anticipate based on students' prior coursework.

Implications for Teaching

Instructors routinely assign textbook readings in undergraduate courses. Incorporating a reading question assignment is a simple, easy-to-implement task that reinforces the importance of reading course materials. The product of such an assignment, student-generated reading questions, has real applicability in the classroom. For research purposes we worked to systematically create and apply a coding rubric; but in practice we know that an instructor can quickly "bin" student data into rough categories, even with very large classes. At a minimum, this type of quick analysis provides a starting point for lectures to meet learners at their current level of understanding.

Taking it one step further, instructors can maximize the benefit by identifying and sharing themes in students' responses as conceptual snapshots with the entire class. For example, an instructor could communicate the prevalence of a representative sample SGRQ and then devise lecture activities to clearly address gaps evident in the question or connect ideas between questions.

Erika G. Offerdahl is an associate professor at North Dakota State University. Lisa Montplaisir is an associate professor at North Dakota State University.

Excerpted with permission from Biochemistry and Molecular Biology Education, 42 (1), 29-38.