Summary of Findings

explored—and continue to explore—metacognitive interventions; informed by work with research methodologists, their various reports from the classroom generated through the project point to several suggestive conclusions:

About reading reflections, Karl Wirth states, "There is no question that reading reflections have a significant positive impact on student reading. In courses with reading reflections, students report more regular and deeper reading before coming to class. Students in reading reflection courses also report using a greater number of strategies during the pre-reading, reading, and post-reading phases. On average, students in Macalester courses with reading reflections earn higher overall grades than students in courses without reading reflections, and this trend holds across SAT scores." Tim Tibbetts comments on how metacognition in general gave him "a new language to use when talking to students about their learning. Reading Reflections give me a tool to hear where students are struggling and respond."

Clara Hardy used exam wrappers in her Latin classes, and, through her study, "confirmed my initial hypothesis that making learning strategies explicit for students and repeatedly requiring them to practice metacognitive activities like self-monitoring seemed to pay off in better success with the material." David Thompson, on the other hand, found "very little difference in both metacognitive self-monitoring practices and course performance" between two groups of students—some doing explicit metacognitive work with exam wrappers and others not—and little change in metacognitive skills during the term. However, he came to see that instruction in metacognitive skills may be particularly important for first-year students as they adjust to the expectations of college-level work and learn to evaluate and monitor their own understanding relative to those expectations. Tracking metacognitive growth in first-year students and in the weakest test-performers gave Thompson a new focus in his teaching, which he will carry forward in a recursive cycle of new interventions and modifications.

Tim Tibbetts writes that knowledge surveys "help students see what topics are important and what types of questions they should anticipate on exams." There are clear improvements in learning outcomes for the students who did exam wrappers in 2009 versus those who did not. He goes on to write, "it would be premature to attribute all of the improvements to exam wrappers or metacognition in general, though I am convinced of their value. Many other elements of the course also changed," motivated by his metacognitive interventions, including more supplemental instruction during this period. Diane Angell, also a biologist, found that metacognitive assignments in the form of "exam preparation assignments" and "wrappers" produced a consistent, if small, effect on improving student learning. She speculated that even more explicit metacognitive instruction, especially for underprepared students, would be helpful.

Tricia Waters addresses the important issue of development of thinking skills more generally over the four years of college. "I am convinced that careful, integrated discussions of critical thinking skills in the context of regular course assignments and discussions improves students' metacognitive abilities. The fact that the Advanced Seminar students showed higher mean levels of Reflective Judgment is consistent with previous research (King & Kitchener, 2004; Kitchener, et al., 1993), suggesting that Reflective Judgment increases across emerging adulthood. That students' critical thinking skills increased across these courses indicates that the interventions we used may be working. The change in critical thinking was greater in the Advanced Seminar and may have been a result of asking students to evaluate their own writing."

These findings, like much work in the scholarship of teaching and learning, are by no means definitive, but they are suggestive and promising. As the ACM instructors continue to use these tools, continue asking questions about the effects of greater metacognitive awareness on their students' learning, they will continue to generate more evidence of the connections.

Lower-achieving students

emphasis on metacognition in the classroom. Hardy found methods such as exam wrappers especially helpful for the "very lowest-achieving students." She argues that "even if the lower-achieving students weren't practicing metacognition at the same level as their higher-achieving peers, the interventions we made did encourage them to do some, and the some they did was helpful in their final grades." David Thompson's results were mixed; increased metacognitive skills, as measured by the Metacognitive Self-Regulation subscale of the Motivated Strategies for Learning Questionnaire (Pintrich 1991), didn't necessarily lead to better learning of Spanish. But in comparing the gaps between student predictions and performance on exams, he came to see that instruction in metacognitive skills may be particularly important for first-year students as they adjust to the expectations of college-level work and learn to evaluate and monitor their own understanding relative to those expectations.

Joy Jordan's project in a statistics class led her to investigate—using knowledge surveys—students' confidence in their knowledge; what she found has particular implications for lower-achieving students. She writes,

I further investigated this increase in confidence, and found it most prevalent (significantly so) with the bottom-performing-half of the class, in terms of course grade. As already mentioned, there wasn't a significant increase in scores on the assessment. This held up when breaking the students in half by course grade. So the bottom-performing-half of the class was, on average, more confident (significantly so) after the activity, yet they did not perform significantly better on the post-activity assessment. What most surprised me was the repeated overconfidence in the lower-performing-half of the class. These are only initial results, but they seem meaningful. For example, when lower-performing students work in groups, do they perceive an increase in confidence even though objectively their understanding hasn't changed? This question—far afield from my initial research question—now has my interest piqued.

Wirth's San Francisco presentation featured several charts showing how knowledge surveys matched up with student performance on exams. Perhaps most strikingly, he found that students in the lower quartile of performance on exams were least able to predict their performance; students who performed best were often likely to underestimate how well they understood the material. Steve Singleton also found that his use of a studio lecture/lab environment including explicit instruction in metacognition in a chemistry class worked especially to "mitigate the performance decline of weaker students."

General awareness/ departmental institutional change

Hardy writes that her study did not provide "the silver bullet of successful Latin learning... [but] did convince me that a general awareness of the benefits of metacognitive activities is a helpful tool in the arsenal of any language instructor. This is the sort of development that I feel is most effective as it infiltrates the culture of a department or institution; as more faculty are aware of potential benefits and more likely to integrate some discussion of them into their daily practice, more students are likely to encounter them in some context and then transfer them to their general approach to learning." She also reports on the effect of requiring students to post three times over the course of a term to forum on metacognition. "This ensures that at least a few points students step back from the course material itself and address metacognitive issues: something that will minimally get them thinking about the area and practicing the skills. As faculty share and take up this sort of exercise, students will encounter them more and more frequently, and thus (one hopes) begin to get the same level of practice with metacognition that they get with other skills associated with a liberal arts education such as writing, speaking and critical thinking."

Holly Swyers found the "real breakthrough" in her Lake Forest "pod" project "the value of a shared vocabulary for talking about what is happening in student learning. The metacognition frame proved adaptable by all members of the pod, so students would hear the same ideas in the classroom, in the dorm, in study sessions and on the playing field. Most members found the principles very similar to ideas they already had in practice, so the real value was in helping students see that the overriding premise of all their college activities...was consistent."

Kent McWilliams, in writing about his music classes at St. Olaf College, states that "from the evidence gathered, it is apparent that students gain tremendously from reflecting at each stage of a thoughtful and gradual process of learning new repertoire. Students posed a wide range of questions of themselves as they began to become more familiar with the repertoire they were studying. These questions ran the gamut from straightforward, well-structured questions to higher-level ill-structured questions. At all levels of questioning, I have been interested to note that these students have transferred these same learning processes to other repertoire they study." Steve Singleton from Coe College's chemistry reports on moving from incorporating such "self-regulatory practices" such as knowledge surveys and reading into his classes toward a comprehensive learning portfolio. The portfolio not only increased students' awareness of their learning process but offered artifacts of learning, which in turn became material for assessment purposes. He reports growing interest in his work from other members of his department, leading to a comprehensive re-design of the chemistry curriculum.