How an external evaluator helps clients develop theories of change

Stephanie Chasteen, ASCN member and co-leader of the Change Leaders Working Group, was recently featured in two blog posts on the blog of the American Evaluation Association, AEA365 . In these posts she shared how she helps clients develop Theories of Change as an external evaluator for NSF-funded projects aiming to improve STEM education at the university level. Read more and follow the links to these posts below!

From Helping clients develop Theories of Change as an external evaluator (Part 1):

I'm Stephanie Chasteen, an external evaluator on NSF-funded projects aiming to improve STEM education at the university level. This is Part 1 of a 2-part series on Theory of Change. Today, I'll share the process I use to help clients develop Theories of Change (ToC). Tomorrow, I'll share how I've been able to work with an evaluation champion within the organization to develop organizational capacity in Theory of Change development.

The National Science Foundation (NSF) has required Theories of Change as part of many educational proposals for some time, but there has been quite a bit of confusion among my clients as to what exactly is meant by a ToC or why it would be useful.

Lesson Learned:

I've learned not to back down when a client doesn't want to develop a ToC (or take the time to do it well). When I have not done so, target audiences and outcomes were usually muddy. One client fought me the whole way, feeling it was confusing exercise without practical value. Later, however, she related to me, "Now I know why you had me do that, and it was totally useful. Now I'm developing one on my own for this new project and it was so much easier." Most clients who develop Theories of Change go on to create (very good!) ToCs to guide other projects, and this gives me pride.

Hot Tip:

×

Click here to view the image at a larger size

Developing a Theory of Change is confusing the first time around. Emphasize with clients that boxes represent outcomes, not strategies. Get your hands dirty with post-it notes; write down outcomes, move them around, and ask questions about how they connect. (The image on the right is our in-progress diagram; see [Post 2] for the final version.) Give clients permission to do it their way: The test of a good TOC is that it is useful, not that it is perfect or follows a prescribed model. My clients have been very creative in using color coding or organizational elements to help their diagrams.

Questions I like to ask are:

1. How do you want the world to be different as a result of your project? (Project vision)
2. Name a few changes that would need to happen to achieve your vision? (Long-term outcomes)
3. What changes would need to happen to achieve those outcomes? (Short-term outcomes)

Thus, we essentially "work backwards" from the project vision. Don't skimp on the project vision! It is such a valuable clarification and inspiration moment for most leaders.

During this process I keep my eye open for "miracles" – steps which seem to "magically" lead to the next step. A common example is "High quality materials are developed" and "People use those materials effectively." Wow, magic! The missing steps might be "People find the materials and see that they are valuable" and "People learn how to use the materials." Such insights lead to valuable additions to the project strategy.

Stephanie goes on to share more lessons learned, hot tips, and some helpful resources:

Rad Resources:

• TheoryofChange.organd their examples
• EstherJames.com's Tutorial: How to write a Theory of Change
• SMART goals
• Padlet ( This site may be offline. ) for virtual sticky notes
• Theory of Change 3-minute video on YouTube is great for clients

You can read more on the AEA365 blog about Stephanie's process (in Part 1) and an example of her work with a client (in Part 2).

Suggested Citation:

White, K & Chasteen, S. (2019, April 30). How an external evaluator helps clients develop theories of change. Retrieved from https://ascnhighered.org/ASCN/posts/ee_change.html.