| Using Student-Generated Reading Questions to Uncover Knowledge Gaps |
By Erika G. Offerdahl, PhD and Lisa Montplaisir, PhD
* The following is an excerpt from Student-Generated
Reading Questions: Diagnosing Student Thinking with Diverse Formative
Assessments, Biochemistry and Molecular Biology Education, 42
(1), 29-38. The Teaching Professor Blog recently named it to its list
of top pedagogical articles.
As instructors, we make a myriad of assumptions about the knowledge
students bring to our courses. These assumptions influence how we plan
for courses, what information we decide to cover, and how we engage our
students. Often there is a mismatch between our expectations about what
students know and how students actually think about a topic that is not
uncovered until too late, after we examine student performance on
quizzes and exams. Narrowing this gap requires the use of well-crafted
formative assessments that facilitate diagnosing student learning
throughout the teaching process.
Within large-lecture courses in particular, instructors have
traditionally relied on the use of verbal questions to gauge student
learning. Verbal questioning is limited, as it reveals the thinking of
only those students most willing to respond. Often these are the
high-performing students in a class. In contrast, student-generated
reading questions (SGRQs) provide the opportunity and incentive for all
students to submit questions, providing the evidence necessary to make
inferences about the range and extent of all students' conceptions. As
evidenced through content analysis, SGRQs have the potential for
characterizing the "conceptual ecology" of the class as a whole. While
formative assessment is not a new idea, most research on its effective
use in undergraduate science courses has focused on implementation in
introductory courses and been limited to pedagogies that make use of
clicker questions. This exploratory study provides preliminary data to
spark a conversation about the diverse ways in which we can effectively
assess student understanding in ways that support conceptual
development.
Methods
The context of the study was an upper-level, large-lecture biochemistry
course offered at a research-intensive university in the southwestern
United States. This was the first course of a two-semester sequence for
biochemistry and cellular biology majors and was team-taught by three
instructors. Pre-requisites for enrollment in this course were
introductory biology, general chemistry, and organic chemistry.
A regular reading question assignment was integrated as a formative
assessment prompt to collect evidence of student thinking in the form of
SGRQs. Students were instructed to approach each reading assignment with
the goal of achieving deep conceptual understanding. We expressed our
expectation that through this process, students would certainly think of
at least one question relevant to the material at hand. Students were
instructed that their questions should not focus solely on factual
material; rather, a reading question should also describe what
conceptual problems the individual has with the material and how the
individual arrived at that question. They then submitted the SGRQs
electronically to the instructors prior to a lecture on the topic.
Eleven reading questions submissions were collected from each student
throughout the semester. Submissions were often one or more paragraphs
long and sometimes included more than a single question. Each submission
was worth a maximum of three points. Students were allowed to drop one
reading question score. There were a total of 700 points possible in the
course; reading questions counted for 4% of the total grade.
To characterize the utility of the reading question assignment in
producing high-quality evidence of student thinking, we analyzed the
resulting SGRQs as related to three learning outcomes (Fig. 2). These
learning outcomes reflect basic skills that are likely to be articulated
in syllabi in the molecular life sciences. The unit of analysis was the
student submission in its entirety. Often, students posed more than one
question per submission. Therefore multiple codes could be associated
with any single submission in each of the analyses described below.
 |
A common goal of undergraduate science courses is to develop
students' understanding of and abilities to engage in scientific
inquiry. Just as science is a process that sprouts from questions about
the natural world, our own students must learn to approach inquiry by
posing insightful questions. Therefore, a reasonable learning outcome in
the life sciences might be to increase students' abilities to ask "good
questions" — those that hold a kernel of a research hypothesis. We
hypothesized that SGRQs might be useful for diagnosing students'
questioning abilities. To this end, we revised an existing taxonomy for
characterizing students' written questions in introductory biology and
applied it to our SGRQs. The levels of the taxonomy represent a
progression in students' questioning from superficial or
definition-seeking questions to more sophisticated questions that
synthesize information and more closely resemble those of practicing
scientists.
Much of the assigned reading associated with this reading question
assignment was a review of topics covered in introductory courses (such
as natural selection and evolution in introductory biology) or
explicitly reviewed earlier in the semester (as was the case of
principles of chemistry). Yet the content analysis indicates that the
concepts that many students are still actively trying to make sense of
and build upon relate to fundamental ideas and specific topic areas with
which these students were likely to have had significant prior
experience. This was somewhat surprising given that this course is
specifically designed for biochemistry and molecular biology majors,
students who are often motivated to perform well in introductory
courses. Although the conceptual snapshots revealed by this analysis
certainly represent possible entry points for instruction, they also
highlight how these entry points might be different than those an
instructor may anticipate based on students' prior coursework.
Implications for Teaching
Instructors routinely assign textbook readings in undergraduate courses.
Incorporating a reading question assignment is a simple,
easy-to-implement task that reinforces the importance of reading course
materials. The product of such an assignment, student-generated reading
questions, has real applicability in the classroom. For research
purposes we worked to systematically create and apply a coding rubric;
but in practice we know that an instructor can quickly "bin" student
data into rough categories, even with very large classes. At a minimum,
this type of quick analysis provides a starting point for lectures to
meet learners at their current level of understanding.
Taking it one step further, instructors can maximize the benefit by
identifying and sharing themes in students' responses as conceptual
snapshots with the entire class. For example, an instructor could
communicate the prevalence of a representative sample SGRQ and then
devise lecture activities to clearly address gaps evident in the
question or connect ideas between questions.
Erika G. Offerdahl is an associate professor at North Dakota State
University. Lisa Montplaisir is an associate professor at North Dakota
State University.
Excerpted with permission from Biochemistry and Molecular Biology
Education, 42 (1), 29-38.
