segunda-feira, 30 de março de 2015

Assessment



Using Student-Generated Reading Questions to Uncover Knowledge Gaps
By Erika G. Offerdahl, PhD and Lisa Montplaisir, PhD

* The following is an excerpt from Student-Generated Reading Questions: Diagnosing Student Thinking with Diverse Formative Assessments, Biochemistry and Molecular Biology Education, 42 (1), 29-38. The Teaching Professor Blog recently named it to its list of top pedagogical articles.

As instructors, we make a myriad of assumptions about the knowledge students bring to our courses. These assumptions influence how we plan for courses, what information we decide to cover, and how we engage our students. Often there is a mismatch between our expectations about what students know and how students actually think about a topic that is not uncovered until too late, after we examine student performance on quizzes and exams. Narrowing this gap requires the use of well-crafted formative assessments that facilitate diagnosing student learning throughout the teaching process.



Within large-lecture courses in particular, instructors have traditionally relied on the use of verbal questions to gauge student learning. Verbal questioning is limited, as it reveals the thinking of only those students most willing to respond. Often these are the high-performing students in a class. In contrast, student-generated reading questions (SGRQs) provide the opportunity and incentive for all students to submit questions, providing the evidence necessary to make inferences about the range and extent of all students' conceptions. As evidenced through content analysis, SGRQs have the potential for characterizing the "conceptual ecology" of the class as a whole. While formative assessment is not a new idea, most research on its effective use in undergraduate science courses has focused on implementation in introductory courses and been limited to pedagogies that make use of clicker questions. This exploratory study provides preliminary data to spark a conversation about the diverse ways in which we can effectively assess student understanding in ways that support conceptual development.


Methods 

The context of the study was an upper-level, large-lecture biochemistry course offered at a research-intensive university in the southwestern United States. This was the first course of a two-semester sequence for biochemistry and cellular biology majors and was team-taught by three instructors. Pre-requisites for enrollment in this course were introductory biology, general chemistry, and organic chemistry.


A regular reading question assignment was integrated as a formative assessment prompt to collect evidence of student thinking in the form of SGRQs. Students were instructed to approach each reading assignment with the goal of achieving deep conceptual understanding. We expressed our expectation that through this process, students would certainly think of at least one question relevant to the material at hand. Students were instructed that their questions should not focus solely on factual material; rather, a reading question should also describe what conceptual problems the individual has with the material and how the individual arrived at that question. They then submitted the SGRQs electronically to the instructors prior to a lecture on the topic.

Eleven reading questions submissions were collected from each student throughout the semester. Submissions were often one or more paragraphs long and sometimes included more than a single question. Each submission was worth a maximum of three points. Students were allowed to drop one reading question score. There were a total of 700 points possible in the course; reading questions counted for 4% of the total grade.

To characterize the utility of the reading question assignment in producing high-quality evidence of student thinking, we analyzed the resulting SGRQs as related to three learning outcomes (Fig. 2). These learning outcomes reflect basic skills that are likely to be articulated in syllabi in the molecular life sciences. The unit of analysis was the student submission in its entirety. Often, students posed more than one question per submission. Therefore multiple codes could be associated with any single submission in each of the analyses described below.
Figure 2: learning outcomes
  Fig. 2

A common goal of undergraduate science courses is to develop students' understanding of and abilities to engage in scientific inquiry. Just as science is a process that sprouts from questions about the natural world, our own students must learn to approach inquiry by posing insightful questions. Therefore, a reasonable learning outcome in the life sciences might be to increase students' abilities to ask "good questions" — those that hold a kernel of a research hypothesis. We hypothesized that SGRQs might be useful for diagnosing students' questioning abilities. To this end, we revised an existing taxonomy for characterizing students' written questions in introductory biology and applied it to our SGRQs. The levels of the taxonomy represent a progression in students' questioning from superficial or definition-seeking questions to more sophisticated questions that synthesize information and more closely resemble those of practicing scientists.

Much of the assigned reading associated with this reading question assignment was a review of topics covered in introductory courses (such as natural selection and evolution in introductory biology) or explicitly reviewed earlier in the semester (as was the case of principles of chemistry). Yet the content analysis indicates that the concepts that many students are still actively trying to make sense of and build upon relate to fundamental ideas and specific topic areas with which these students were likely to have had significant prior experience. This was somewhat surprising given that this course is specifically designed for biochemistry and molecular biology majors, students who are often motivated to perform well in introductory courses. Although the conceptual snapshots revealed by this analysis certainly represent possible entry points for instruction, they also highlight how these entry points might be different than those an instructor may anticipate based on students' prior coursework.


Implications for Teaching

Instructors routinely assign textbook readings in undergraduate courses. Incorporating a reading question assignment is a simple, easy-to-implement task that reinforces the importance of reading course materials. The product of such an assignment, student-generated reading questions, has real applicability in the classroom. For research purposes we worked to systematically create and apply a coding rubric; but in practice we know that an instructor can quickly "bin" student data into rough categories, even with very large classes. At a minimum, this type of quick analysis provides a starting point for lectures to meet learners at their current level of understanding.


Taking it one step further, instructors can maximize the benefit by identifying and sharing themes in students' responses as conceptual snapshots with the entire class. For example, an instructor could communicate the prevalence of a representative sample SGRQ and then devise lecture activities to clearly address gaps evident in the question or connect ideas between questions.



Erika G. Offerdahl is an associate professor at North Dakota State University. Lisa Montplaisir is an associate professor at North Dakota State University.

Excerpted with permission from Biochemistry and Molecular Biology Education, 42 (1), 29-38.

Mais Médicos para o Brasil




Plataforma web para a divulgação de pesquisas relacionadas ao Programa Mais Médicos


A Rede de Pesquisa em Atenção Primária à Saúde  em parceria com a Organização Pan-Americana da Saúde (OPAS) propõe-se a construir uma plataforma web para a divulgação de pesquisas relacionadas ao Programa Mais Médicos para o Brasil. Esta iniciativa responde à necessidade de se dispor de um espaço comum para o intercâmbio de informações e conhecimentos relativos às principais pesquisas que estão sendo realizadas sobre o Programa Mais Médicos. A criação de um acervo de projetos e pesquisas no site da Rede de Pesquisas em APS busca possibilitar à comunidade acadêmica e a todos os atores interessados conhecer a implementação e o impacto desta iniciativa e divulgar os conhecimentos e evidências  gerados pelas pesquisas em curso. Os temas objeto das pesquisas podem referir-se a varias áreas, desde a análise dos processos de implementação e gestão do programa até os seus impactos na saúde da população e na organização do sistema de serviços de saúde com destaque para:
- mudanças nas condições de saúde da população beneficiária atribuíveis à provisão de médicos pelo PMM;
- satisfação da população usuária com os serviços proporcionados pelos médicos do programa;
- fortalecimento de sistemas municipais e estaduais de saúde baseados na APS.
O acervo da plataforma incluirá pesquisas de diversos tipos e metodologias, inclusive teses de mestrado e doutorado, abrangendo temas específicos ou gerais e âmbitos locais, regionais ou nacional.
No caso de você ter participado ou coordenado ou estar participando ou coordenando uma ou mais pesquisas sobre o Programa Mais Médicos, por gentileza, preencha este breve formulário online que está disponível no site da Rede APS  
Se você quiser divulgar algum tipo de documento relativo à sua pesquisa (projetos de pesquisa, relatórios, instrumentos de pesquisa, publicações, outros) por favor envie a informação por e-mail  para inaiara@rededepesquisaaps.org.br, para sua publicação no site da Rede.

domingo, 15 de março de 2015

Art of Being Human



To Help Students Succeed Professionally and Personally, Teach the Art of Being Human


By Lisa M. Dolling

       Michael Morgenstern for The Chronicle



Among the many false dichotomies fostered by the continuing debates surrounding higher education, one that I find especially disconcerting is that which pits the professional against the personal. While it is expressed in a variety of ways, it boils down to this: Either you believe the purpose of going to college is to be able to secure a (preferably high-paying) job, or you think there is something more intrinsically valuable to be gained from the years spent earning a degree. My question is: When did these become mutually exclusive?

Yet believing that they are is one of the unfortunate conclusions many people draw from the endless bickering about the value of a college education, a debate that many believe was ignited by Ronald Reagan’s disparaging of "intellectual curiosity," and intensified with Scott Walker’s recent proposal that the University of Wisconsin revise its mission statement to replace references to the "search for truth" or desire to "improve the human condition" with clear (read "practical") goals of meeting "the state’s work-force needs." Politics aside, I doubt that either of these officials wanted to assert that professionals need not be thoughtful or reflective. However, that is precisely what this sort of sloppy rhetoric implies and what continues to drive the public’s misconceptions about higher education and the "value" it holds for our society.

Education is first and foremost about learning; about developing the intellectual capacities needed to succeed as professionals and human beings. This was the belief that inspired W.E.B. Du Bois to declare in a 1949 essay, "Of all the civil rights for which the world has struggled and fought for 5,000 years, the right to learn is undoubtedly the most fundamental," for which "we should fight to the last ditch."

Few people would disagree. After all, how can we expect our society to flourish without a citizenry that values learning as among the highest of virtues or expect our nation to excel without providing for everyone the resources necessary to cultivate intellectual curiosities in new and innovative ways? Most important, how will we ever establish our moral credibility if we do not encourage these pursuits with an eye to improving the human condition? The problem is that many people are able to maintain these beliefs while at the same time dismissing as "intellectual luxuries" such ideals as the "search for meaning" or "concern for the common good."

Of course, given the exorbitant cost of higher education, it is understandable that professional success would become the priority. Mounting student debt and the financial strain of paying for college are perils that deserve serious attention. As the mother of a high-school senior, I shudder at the prospect of having to pay more over the next four years than we originally paid for our house. And I too want to make sure my daughter learns everything necessary to have a successful career. But as an educator, I also know that this will involve much more than just training her for a profession. As I often remind my students, in the end it is not only a matter of what you do, it’s how and why you do it; and the knowledge that this requires above all is knowledge of oneself.

Ironically, students are the ones who seem to understand this more readily than many of the "experts" weighing in on the debates. Students tend to recognize self-reflection and quest for meaning as instrumental to their future success and not just "frills" to be consigned to the dustbin of impracticality, as many would have it. Therefore, I would entreat everyone with a vested interest in higher education to take a moment to ask students directly what they find most valuable about the educational process. I am confident that as a result, the tide of the debates would soon start to turn.

I teach philosophy at a university known for engineering and science, where students are driven and focused, with just one goal in mind: getting that high-paying job. That is, until they set foot in their humanities classes; there something magical happens. Suddenly they are no longer engineering students, but just students, with a willingness to learn as much about themselves as about the material.

A favorite example involves a math major who enrolled in my aesthetics course a few years ago hoping to get his humanities requirements out of the way. One of the works we read is Plato’s Ion, whose main character is a rhapsode devoted to the works of Homer. Ion describes how ecstatic he becomes at the mere mention of Homer’s name, and how while reciting the poet’s works he is literally taken out of his senses.

After discussing the dialogue, I ask students to identify a work of art that does to them what Homer’s poetry does to Ion. In all the years I have been teaching aesthetics, I have never had a student unable to do this assignment—until that math major. He told me he didn’t like poetry, had no interest in painting, and never listened to music, and therefore wanted to be excused from the assignment. I explained that it wasn’t about the object per se, but rather the transformative experience. His response was that he never really got that excited about anything in life and therefore was at a loss.

I advised this student to take some time to think about it and get back to me. Much to my delight, a few days later he emailed me with an idea. He asked me if he could use a mathematical proof as an example of something he found to be inspirational.

I shall never forget the experience of sitting in the classroom with the other students as this young man explained the majesty of Euler’s Theorem, swept away by his passion and enthusiasm. At one point it was as if he had forgotten where he was. Most remarkable was when he told us he had never explained the proof to anyone before; up until then he had merely used it.

After class the student thanked me for the opportunity to discover something about himself that he never knew; namely, that he was capable of becoming so absorbed in his work that he could appreciate it in entirely new ways. I’m pretty confident that is a realization that will benefit him professionally no less than personally.

Just the other day, a different student asked me why I included a selection from Aristotle’s Nicomachean Ethics on the syllabus for my aesthetics course, a course intended to examine philosophies of art. I explained that in many respects, Aristotle’s work is intended as a treatise on the "art of being human," a craft we need to hone no less than any other. The student’s response was, "Wow, how interesting. That’s an art we seem to have lost sight of these days." I had to hold myself back from replying, "Gee. I wonder why."


* Lisa M. Dolling is dean of the College of Arts and Letters at Stevens Institute of Technology.


sábado, 14 de março de 2015

Social media





Using Facebook to Enrich the Online Classroom




 



“Am I writing to myself?” That’s what I used to wonder when I first started teaching Spanish online a year ago. My learning management system, message boards, and group emails were impersonal and unresponsive—more like writing in my diary than sharing information with my students. I never knew for certain who read and understood my announcements or received an (electronic) handout or assignment directions. In the traditional, on-campus classroom, I’m a very interactive, hands-on kind of instructor, so I also went from knowing each and every one of my students by name and even a little bit about them to having nothing more than a roster with 115 names and majors. I just wasn’t satisfied, so I did something that others in the field had encouraged me not to do; I created a Facebook group for the class, and I’m not going back.
Some educators find Facebook daunting and potentially perilous, but the advantages are well worth it, especially once you master all the settings Facebook has to offer. Now, is it like being in the classroom? No, it’s not like being in the classroom, but it’s purposeful, interactive, and enjoyable, and exactly what my dry, strictly discussion-board online courses were missing. Facebook makes for an optimal virtual classroom, and this is why:
1. I get to see my students and they get to see me.
Pictures really are worth a thousand words, and they have made an enormous difference in my online courses. My Facebook group provides a personal component that an LMS cannot imitate.
2. Facebook allows me to have all class communication and interaction in one place.
Apart from the LMS, where students log on to read their eBook, watch grammar videos, and do exercises, Facebook is our home base. It’s where I post more than just announcements, assignment directions, and reminders. Facebook also allows me to post pictures and videos, add “handouts” (under files), and survey students to get their feedback. Being able to do all of these things in one place saves a lot of time and confusion. The Facebook group is our go-to for all things communication-related; the course announcements, handouts, reminders, questions, and answers are all there on our group page.
3. The Facebook “chat” feature helps with the unique challenges that are inherent to learning a foreign language online.
Online students learning a foreign language need extra guidance, reinforcement, and ideas on how to remember, compare, and contrast grammar points that build on themselves like a staircase. As instructors, we must be available to students to help them get from point A to point B in a timely manner as they work though their exercises. I hold regular Facebook office hours and “chat” with students in real time, answering questions about conjugations, pronouns, sentence construction, etc.
4. The popularity of Facebook makes it a good resource to use in an online environment.
Most students already have a Facebook account, so they don’t have to do anything “extra” to keep up with information and announcements. This means less work and stress for them as they begin what may be their first online class.
5. Facebook notifies students whenever I add a post and lets me know when they have seen it.
This is tremendous! I can see who is following along, so I know if students are “going to class” as they should. I no longer wonder if I’m talking to myself!
6. Facebook creates a forum similar to that of a traditional classroom.
Via the Facebook group, students get to “see” each other, contact each other, find a study buddy, or form exam study groups if they wish. They can also ask questions for me to answer; and everyone in the class benefits from the explanation. (Keep in mind that I use privacy settings to approve member posts and manage the page.)
7. No “friending” is required in order to form and participate in a Facebook group.
There are some misconceptions about how to use Facebook for academic purposes. Professors are understandably concerned about privacy issues and “friending,” and perhaps the best part about Facebook groups is that I don’t have to friend my students and they don’t have to friend me. All I do is create the group on the private setting and send the link to my students via email a few days before classes begin. Over those few days, they click on the link in Facebook and send me a request to join the group. Once all of my students are in the group, I change the group setting to “secret” to make it invisible to anyone not involved in our class.
8. Facebook provides a forum for some “fun stuff.”
I also use Facebook for some lighthearted comic relief by posting funny Spanish sayings, jokes, and pictures that are circulating on Facebook at the time. I also include words of encouragement and shout-outs to students for things like a great grade or a savvy question or post. I add pictures of my life, like my dogs playing in the snow and include captions in Spanish. The “fun stuff” helps me use my personality to connect to distance students in a meaningful way.
9. Facebook provides a way to incorporate real-life language examples into an online class
To encourage students to keep up with course information and announcements, I include a participation grade for real-life language “mini-activities” that take about 10-20 minutes to complete. These activities include things like Spanish songs (with lyrics) from which they have to define and translate a couple of new words or phrases of their choice, video clips from a Spanish reality show where they listen and write down some sentences they understood, and animated clips that they describe in Spanish using a certain verb tense. They post their answers as a response to the original post, knowing that their peers will be seeing their work, which makes them a little more conscientious.
10. Students like Facebook.
While I can’t please everybody all the time, my students have been overwhelmingly positive about the Facebook classroom experience. They describe it as “fun” and “easy to keep up with.” They also like the fact that they’re notified of all page updates and that files and announcements are all in one place. Several of them even liked the mini-activities and chose to do all of them! I’m sold.
As you can imagine, a Facebook group virtual classroom is a lot of work, but it can turn an uninspiring online course into a relevant learning experience. It’s not perfect, but it’s the best mode of communication I’ve found for teaching a foreign language online. During the past year, I’ve worked harder than I ever have in my career to date. I’ve experimented, I’ve changed my mind, I’ve added and discarded, I’ve asked for advice and feedback from students, and I’ve updated my syllabus many times. And, at least until something better comes along, Facebook is staying on it.
Dr. Danielle Geary is a lecturer of Spanish and coordinator of online learning at the Georgia Institute of Technology, where she recently created the university’s first Spanish online program

See more at: http://www.facultyfocus.com/articles/online-education/using-facebook-enrich-online-classroom/#sthash.GbfRMu5J.dpuf


quinta-feira, 12 de março de 2015

Science factory



How does scientific research suffer if data comes from online survey forums? 



What used to be the job of university students, being surveyed for scientific and academic research, is now often being done by people online, which could arguably lead to flawed and unreliable data. 



Mechanical Turk is an online job forum – an informal work force of people who participate in studies – basically a pool of professional survey-takers. Researchers are now taking data from such sources because it’s inexpensive and quick. But there could be discrepancies on the reliability of answers given. What kinds of surveys are we talking about?

“You have a lot of psychology, social science research, but also political scientists are using it. You even see it in medical research,” said Jenny Marder, who reported the story for PBS Newshour. “And we looked at a lot of the studies that are using Mechanical Turk, and a lot of them are asking really big questions. There’s a lot of research on human behavior, but also on teen alcohol abuse, a lot of research on decision-making, how people perceive scientists and climate scientists. So these aren’t obscure studies. And they’re asking some pretty big questions.” 



This issue with surveys like those from Mechanical Turk is that many people do them from home, even thousands of them, and there’s no way to know if the person was distracted, actually paying attention, or if they are on automatic because they have seen some of the same questions repeated over and over. The people who take these surveys get paid (albeit, very little) even if they answer questions in an automatic, not entirely thoughtful way. So what is the incentive to really sit down, focus, and answer truthfully? 

Data isn’t valuable if it isn’t an accurate reflection of the subject’s perspective and the topic, but it continues to be used.


Watch How pro survey-takers are shaping scientific research on PBS. See more from PBS NEWSHOUR