On the Cutting Edge Evaluation

The evaluation plan addressed two major effort domains of the project: the workshops and the website. All evaluations provide information to enable project staff to answer the following questions: "What are the results we are trying to achieve?" "Given these, who do we have to reach?" "What programs do we have with which to reach them?" "What evidence do we have that we did what we planned?"

The intent of the evaluation is both formative and summative. From a formative perspective, we look to improve our overall program including the workshops, website collections, and leadership program. Our summative evaluation focuses on understanding how the program impacts geoscience faculty, community, and culture. A second aim is to evaluate how the program contributes to the research base on effective faculty development.

We use a tiered evaluation framework that combines mixed-method approaches. We use a small samples of indepth interviews to inform larger surveys of all participants as they exit workshops and periodically through the web. The results of these interviews and surveys also inform the design of a larger survey which we administered in 2004 and will administer again 2009 to the larger geoscience community.

Overall Summary Evaluation Report (Microsoft Word 259kB Apr17 06)

Publications and Presentations

• Iverson, E.R., McLaughlin, J.A., Manduca, C.A. (2006) The Logic Model as Compass: Guiding Stakeholders Through the Consequences of Evaluation. Presented at American Evaluation Association 2006 Annual Conference.presentation (PowerPoint 1.5MB Nov29 06)
• Macdonald, R. H., C. A. Manduca, D. W. Mogk and B. J. Tewksbury (2004). "On the Cutting Edge: Improving Learning by Enhancing Teaching ." In, Invention and Impact: Building Excellence in Undergraduate Science, Technology, Engineering, and Mathematics (STEM) Education. Washington, D.C., AAAS: 381.
• Macdonald, R.H., C.A. Manduca, D.W. Mogk, and B.J. Tewksbury (2005). "Teaching Methods in Undergraduate Geoscience Courses: Results of the 2004 On the Cutting Edge Survey of US Faculty." Journal of Geoscience Education, 53(3): 237.
• Iverson, E.R., C.A. Manduca, J.A. McLaughlin, R.H. Macdonald, (2005), Enhancing your Teaching and Developing New Leadership: Impact of the On the Cutting Edge Professional Development Program (PowerPoint 2.6MB Jan18 06), Eos Trans. AGU, 86(52), Fall Meet. Suppl., Abstract ED42A-02
• Manduca, C.A. and E.R. Iverson (2005). Understanding Faculty Work Habits as a Foundation for Professional Development (PowerPoint 1.8MB Jan18 06). Presented at POD Network 2005 Annual Conference.
• Fox, S., Manduca, C. A., Iverson, E., (2005). Building Educational Portals atop Digital Libraries D-Lib, 11(1).

Evaluation Reports

• 2008 SERC Results of an Email Poll (Microsoft Word 42kB Dec12 08)
• 2006 SERC Pop Up Survey Results (Microsoft Word 197kB Apr28 06)
• Summary report of awareness survey conducted November 2005-January 2006 (Microsoft Word 36kB May1 06)
• Summary report of phone interviews conducted with 54 workshop participants in 2005 (Microsoft Word 79kB Jan18 06)
• Summary report of phone interviews conducted with 12 non-workshop website users in 2005 (Microsoft Word 33kB Jan18 06)
• Summary report of Cutting Edge users from SERC pop-up survey 2005 (Microsoft Word 44kB Jan18 06)
• Summary report of 235 responses to impact survey 2005 (Microsoft Word 89kB Jan18 06)
• External Heuristic Review (Acrobat (PDF) 979kB Jan18 06)

• 2004 Summary report of 95 responses to follow up survey (Microsoft Word 127kB Jan18 06)
• 2004 AGU Focus Groups (Microsoft Word 38kB Jan18 06)
• 2004 summary of website evaluation activities (Microsoft Word 28kB Jan18 06)