Evaluating Learning

We used a backwards design to develop embedded assessments for assignments and activities to assess student learning in a progressive manner. Each of the five major learning outcome categories was assessed by at least two different types of activities: one traditional activity such as labs or exams and another more student-centered and constructivist learning activity such as the case study role-play and outreach projects (darker shading). Interestingly when the student designed outreach projects spanned all the learning outcomes when they were viewed as a collective i.e. each project hit on at least one learning outcome (lighter shading). In addition to the standard course evaluation form, we designed a customized one and also customized the SALG and plan to revise these course evaluation and student learning surveys. The assessment and evaluation strategies are outlined below and are being revised based on the pilot run of the course in spring 2011. Exams: Students were given a pre-course, midterm, and post-course exam that measured their understanding of the biological, ethical, social, and political dimensions of stem cell research and their historical understanding of events that influenced this field of research. via timeline constructions or Historical Episode Maps (Lin , C.Y, 2010). The exams were intended to assess developmental learning through the Revised Blooms Taxonomy of Cognitive Learning such that some questions were simply asking student to recall information while others asked students to apply or synthesize knowledge learned to address a specific question or problem. Exams are being analyzed for student learning gains. The richness of students answers is being measured, not simply by marking a correct or incorrect answer, but rather coded, as to determine the degree to which a student made developmental progress in the course.
Authentic and Performative Assessment: Case Studies. The rubrics adapted to guide students in the case studies were also used to evaluate analytical skills and communications skills, both oral and written. One rubric was an adaptation of the Evidence and Trade Offs rubric developed by the BEAR Assessment System (Berkeley Evaluation and Assessment Research) originally designed for middle school students within the context of an issues oriented curriculum.
Authentic and Performative Assessment: Outreach Projects. These projects were progressively assessed using rubrics for each of the following with 1) a progress report with annotated bibliography and metacogntive questions regarding research challenges and surprises and needs assessment for the product 2) Progress report addressing very structured set of twenty questions regarding medium, collaboration, community partner, etc, 3) final report that returns to the twenty questions 4) Oral presentation to the class. We are analyzing the final reports and mapping student language back to the five major categories of learning outcomes.
Formative In Class Assessments: Given the range of student's backgrounds in this course, it was necessary to take the pulse of the course frequently. In class concept maps, and in class " quizzes" that were not graded, and in-class questions/group work were a common practice in the course to assess student learning and challenges, and to inform my teaching. In addition, video news clips that captured the essence of the subject at hand were used as discussion prompts to see where students could take the conversation and the types of questions they would generate demonstrating an ability to be critical.
Peer Assessment: We had the unique opportunity of having two students who formerly took this course in a different format sit in on the class sessions and record notes of what transpired in the classroom, reflecting on the learning, or the lack of learning or learning challenge, in the classroom; this peer insight was invaluable. Since both of these students serve as Science Fellows, in that they have read articles on biology education, visualization in biology, and critical pedagogy, their perspective was rich and provided a dimension that was quite helpful in monitoring student engagement and learning.
Lin, C-Y. et al. 2010. Making science vivid : Using historical episode maps. International Journal of Science Education. 32(18): 2521-2531.
Chamany K., et al. 2008. Making biology learning relevant to students: Integrating people, history, and context into (college) biology teaching. Life Sciences Education. (7):267-278. http://www.lifescied.org/cgi/reprint/7/3/267.pdf
Wilson, M. and K. Sloane. 2000. From principles to practice: An embedded assessment system. Applied Measurement in Education. 13 (2): 181-208. (BEAR Rubric).

Policy Maker Assignment: Students were asked to adopt the role of a congressional aide of a specific policymaker. The policymaker asked the aide to review issues on stem cell research and help the policymaker craft a position statement. The aide must write two variations; one that supports the research and one that does not using the same evidence base and being careful to stay in line with the policymakers overall position on related issues. Student review the life and voting history of one NY Congressional Representative and both Congressional Senators and pay close attention to their position and voting record on stem cell research and related issues and their personal history where applicable (where did they go to school, how were they educated, what is their profession, background, district, who do they align themselves with?) Then they write two 300- word statements. Students provided summaries that were particularly useful when the class prepared to send representatives to the DC Symposium. In the spring 2011 course, this assignment set the stage for developing contextualized relativism.

Congressional Memoir and Academic Essay/ Controversial Quote: Students read chapters from the book Sex, Science, and Stem Cells: Inside the Right Wing Assault on Reason by Diane Degette, which provides an account of Colorado Representative and Deputy Whip experiences in bringing bipartisan support forward for embryonic stem cell research. Students then chose one quote from the text and formulate an academic essay that is evidence based and takes a position with respect to the quote.

Case Studies: There are four case studies in this course and they all focus on human subjects research, but each has a unique context that encourages students to move from dualism to a position of relativism but only after being exposed to all stakeholders positions. Two of the course case studies required students to conduct a deep research dive to adopt the single perspective of a real person involved in some aspect of stem cell research or cell biology research, engage in dialogue with other stakeholders via role-play, and then reflect on the multiple perspectives presented during the dialogue and readings, and then come to a position via a written essay on the subject that required them to address at least TWO different perspectives using evidence that illustrates their understanding of the benefits, risks, and trade-offs involved with any social policy making. We constructed the case scenario to promote discussion and dialogue rather than debate, as we wanted to expand students' worldview and tolerance for value systems that might differ from their own. The first case is focused on the history of HeLa, the first human cell line to be cultured, the second revisits some of the same issues but on an international scale addressing oocyte procurement for embryonic stem cell research, and explores other female tissues that might serve as better sources of stem cells such as fat cells or menstrual blood cells, the third case explores international concern regarding therapeutic misconception surrounding stem cell therapies, and the last case study is an exploration of how regulatory and advisory boards are composed and what governs their operation.

Student Outreach Projects: To empower students to make the transition from personal interest to situational interest (course on stem cell research), we asked students to self identify an outreach project topic and medium that resonated with them. Students with a feminist studies perspective tended to focus their attention on oocyte procurement and the biology of the egg can created Dipity Timelines, students with interest in literature wrote critical book reviews, students grappling with basic biology developed educational songs in the format of schoolhouse rock to educate youth about the central dogma, and students interested in policy developed timelines infographics. Students were organized into groups or worked alone on six different projects focused on raising awareness about stem cell research and its ethical, legal, and social dimensions. These independent projects were coupled with proposals, progress reports, and project reports. They ranged from and educational workshop held at an undergraduate conference on social justice, to a 7- minute video on Stem Cells in the City. '

Lab: Planaria. Though planaria are often used to teach developmental biology, and cell regeneration, we reviewed existing curricula and took the elements we felt best reflected our goals and added a dimension of inquiry that we felt highlighted the genes and environment relationship that is often very challenging for students to grasp. Though the activity included a worksheet and collaboration among all groups of pairs of students to make meaning of the results, students felt lost as the placement of the activity came too early in the course, did not requires students to submit a report, and in general remain disconnected for students. We are in the process of revising the implementation of this activity and designing appropriate evaluation instruments for students learning. Note: the appearance of questions that extended the knowledge learned here on the midterm exam caught students off guard.



Media Viewings: Students watch many forms of media, some created by students, others included videos of the National Academy of Sciences Hearings on C-Span or video archives from The New York State Stem Cell Science Ethics Committee Hearings, The New York Stem Cell Foundation Media Event on the State of Stem Cell Science, and more. Each of these viewings or events demonstrated how people in different venues were preparing a message to a wider audience and how the message was framed or captured by the media and for what purpose. These viewings then informed their reflective essays and progress reports on their self-designed community outreach project.

D4D- Debating for Democracy Workshop: Students attended a one-day workshop on advocacy hosted by Soapbox Consulting that helped them prepare for policy maker meetings, letter writing, and campaigning. Kush's book the One Hour Activist was provided via Project Pericles in the spring 2009 version of the course.

DC Symposium Student Representative Proposals and Reports: Students were asked to submit a one-page proposal stating why they should represent the class at the SENCER DC symposium. The class then voted after reading the proposals and two students secured funding from the College Student Union to attend the Symposium. Students who attended the DC symposium were asked to develop and rehearse their "take aways" for the local policy maker meeting (Rangle's education aide) and to also report back to the class about the experience and what they learned. during the spring 2009 version of the course.

Outside Events Reflection: Students attend an outside event about stem cell research and comment on Who (who attended who hosted, who asked questions, who presented), What (purpose, outcome, intended effect), How (what media or message was used, how did the event connect to course readings, topics, experiences, projects).