Skip to Main Content Skip to Navigation

SERC > Program Evaluation for SAFRO

Evaluation and Assessment in Grant Development

A Discussion at the SCAFRO Meeting, July 14, 2016 10:30-12:00

Program Evaluation Resource Website

More about SERC Evaluation and Research

Goals

Intro to SERC
The role of evaluation, assessment and research in projects and proposals
Show

Hide
- What is the difference between evaluation, assessment and research?
- When do you want what? [hidden]
  - Role in project
  - Requirements/goals of grant program

What makes a strong evaluation plan?

alignment with goals and needs of projects - role of logic model or theory of change
integrated with the rest of the proposal (text, diagrams, timelines, meetings)
formative feedback loop explicit - how will data/findings be used in program management[hidden]

summative evaluation judgements are credible

authoritative voice (objective, credentialed)
based on appropriate data

audit/report (your record or theirs)
quantitative vs qualitative vs mixed methods
challenges of self-report

research products are defined and in the project plan

doable within time, budget and personnel

How does evaluation from one project support the next project?

Keck Consortium
HHMI broadening participation -> STEP
HHMI transfer framework

How to questions/common challenges discussion.

budgeting
finding external evaluators
managing external evaluators
using internal evaluators
engaging the project participants in research

Collective Impact, Systems Approaches, and Emergent Outcomes

Here are some of the refs I talked about and a few more
Kania, J., Kramer, M., & Russell, P. (2014). Strategic philanthropy for a complex world. Stanford Social Innovations Review, 12(3). - this is the paper that advocates collective impact for philanthropy
Kania and Kramer (2011) Collective Impact Stanford Social Innovations Review,http://ssir.org/articles/entry/collective_impact - this is the original paper defining what they are talking about
Meadows, D. (1999). Leverage Points: Places to Intervene in a Systems: The Sustainability Institute. - this is a really nice paper for talking about how your proposed work will lead to larger changes
Preskill, H., Gopal, S., Mack, K., & Cook, J. (2014). Evaluating Complexity: Propositions for Improving Practice. Practice Brief. - this is a great intro to complex systems thinking
Preskill, H., M. Parkhurst, M., and J.S. Juster, 2015, Learning and Evaluation in the Collective Impact Context Guide to Evaluating Collective Impact: Collective Impact Forum: FSG. - this is very good for thinking about how to frame the evaluation for different stages of the project and thinking about reasonable timelines
Hargreaves, M. B. (2010). Evaluating System Change: A Planning Guide. - this is a great very short book with 10 things your should do.
Quinn Patton (2014) Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use ISBN-13: 978-1606238721 - this is a big book that I think of as a definitive work
Kastens, K. A., and Manduca, C. A., 2016, Using systems thinking to design, implement and evaluate the InTeGrate project: InTeGrate White Paper. downloadable here - this is the paper I helped write about InTeGrate and how we applied all of the above to this particular project and its design, management and evalauation