Workshop on Engaging Faculty: Post-Workshop Survey

Executive Summary of Post-Workshop Survey Responses

An in-depth analysis of the feedback received from the Workshop participants is included below. In summary, we had a response rate of 63%. The participants of the survey were university administrator (37%), department chair (26%), physics faculty (22%), chemistry faculty (19%), STEM education center administrator (15%), and facilitator/guest (7%). Respondents were able to choose more than one role at the university.

Almost all participants (96%) would recommend this Workshop to their colleagues. They rated this workshop compared to others as excellent (43%), very good (43%), and average (13%). They were most satisfied with the team working time (3.93 on a 1-4 scale; 4 being very satisfied). They felt it was really valuable to have dedicated time away from campus to meet with their team to work on a focused problem, have facilitation with change experts, have a range of approaches by different institutions, a range of roles at the Workshop, and an expectation for an action plan by the end of the day that would have some follow-up.

The trend in participants' responses was that they would have liked more team working time and networking time and less time spent on tools and resources and doing a pair share with another team.

We asked "What could this organization (APLU / Sloan/ Grouping of people at the workshop) do next that will help you and your campus initiative?" The responses centered around the following suggestions:

1. Communicate the importance of this work with provosts ("Contact Provost about participation, encourage support of initiative.")
2. Reconvene or have follow-up ("Based on how the initiatives get off the ground, it may be valuable to bring together the same group of people. Or the leadership among them. To learn from each other's experiences and stimulate further work. Getting something off the ground usually takes repeated efforts.")
3. Provide resources and knowledge of what works (e.g. webinars, successful case studies, list of experts who can do site visits)

If we were to do this workshop again, we would follow essentially the same format with some modest tweaks to the length of certain sessions. All in all, we feel the format was very successful.

In-depth analysis of post-workshop survey

We distributed the survey on June 5 and closed the survey on June 23, 2015. We had a response rate of 63%. The participants of the survey were university administrator (37%), department chair (26%), physics faculty (22%), chemistry faculty (19%), STEM education center administrator (15%), and facilitator/guest (7%). Respondents were able to choose more than one role at the university. One respondent identified as a director of a center for teaching and learning and another identified as a biology faculty who works with physics faculty.

On a 1-4 scale (very dissatisfied, dissatisfied, satisfied, very satisfied), overall the attendees were quite satisfied with the SMTI conference (3.57) and Workshop (3.62). They were most satisfied with the team working time (3.93), then wrap up session (3.36), tools (3.31), and pair share with another team (3.31). Almost all participants (96%) would recommend this Workshop to their colleagues. They rated this workshop compared to others as excellent (43%), very good (43%), and average (13%). No one said poor or very poor.

The trend in participants' responses was that they would have liked more team working time and networking time and less time spent on tools and resources and doing a pair share with another team. But for the most part, most participants felt each session was just about right in terms of length: wrap up session (82%), networking (81%), institutional team time (74%), pair share with another team (73%), and tools and resources (56%).

We asked, "Did you find new resources and/or approaches that will facilitate your transformation of upper division courses and engagement of faculty in this process?" The majority of respondents (81%) said yes. Comments included:

• "I learned about a range of people, projects, and online/print resources that I expect will be very helpful."
• "Not new, but within the context of research based discussions about change processes, we were better able as a team to bring team members along who may otherwise have been less willing."

We asked "Did you have the opportunity to intersect with people or programs outside of your traditional communities? Was this valuable?" All of the comments indicated that this aspect of the Workshop was valuable. Comments included:

• Yes, I always learn from interacting with representatives from the DBER community. Interactions with a range of institutional representatives were insightful as well.
• It was very useful in getting our own team integrated. I also learned interesting things from other institutions, although the Midwest Chair's meeting actually has more useful interchange on what people are doing successfully.
• Yes, and it was valuable. At both conference and workshop.
• Yes, interacting with a variety of faculty, administrators, etc. was extremely valuable. The entire workshop was outside of my community and so was particularly valuable to me!
• Yes and this helped give a broader perspective of what could be done.
• Yes and it was very valuable to building internal and external champions.
• Yes, I and other members of my university made some very good connections and will engage in some exchange visits.
• Absolutely and it was very helpful in increasing my awareness of other approaches and resources. I also believe I had enough time to genuinely network so that I feel relatively comfortable following up with a few people after the conference and workshop.
• Incredibly so. It gave me a much broader view of what the administration has to work with and deal with. Something I had little to no experience with before this.
• Yes, it was very valuable to get together with other discipline-specific innovators and learn from them. Even those in my same institution, because we don't often get to get together for long periods of time.

We asked, "What did you like most about the Workshop?"

• Organizational ideas put on the table by the various groups on how to initiate and drive change.
• It was great to have teams and give them time to map out plans. The pair-share helped people articulate their plans and gain insights/inspiration from others.
• The requirement that the teams produce deliverables that could then be commented on by outside experts.
• The chance for the team to talk, to hear from others, and the meals together.
• My team attended the conference and the workshop, which gave us a common background for beginning our conversation. Meeting far away from campus helped us to focus on our ideas and to develop ideas in the absence of other competing demands. I am very excited about our proposed project! We are planning to write several grant proposals as a result of participating in the workshop.
• Team approach -- having us bring a team, then setting aside time for us to interact and devise a plan -- with resources and chance to hear from other teams -- was brilliant.
• It is prompting me and others at my University to start taking some more concrete steps towards things that we know are valuable. I intentionally brought one faculty member who is very involved in active learning and one who is not yet. The conference provided us with good ideas from others. I'm very excited about beginning discussions in my department.
• The variety of disciplines represented and the stages of development represented.
• Focused time w/my institutional team, since they are administrators that I rarely get extended planning time with.
• The retreat character that focused our team on solving a specific issue/problem.
• The time for our team to "iron out" significant planning.
• The working time with our team and the usefulness of the template that the prepared PPT outline provided for us.
• The freedom to discuss with experts and work on things with appropriate check points to keep everyone on track and moving forward.
• The ability to concentrate on an issue over a few days' time, and draft concrete plans to attach some problems and plan new innovations.
• Drawing on work of others with more expertise in thinking about our own work and action plan. Willingness of others to share resources and instruments for us to consider in our work.
• My team time with the golden nugget; we all had to travel to NOLA to work together.

We asked, "In what ways could this Workshop be improved?"

• Different formats, such as a panel discussion or small group discussions or resource fair, would have been more effective for presenting tools and resources.
• As a facilitator, I did expect to help guide or push discussions a bit more. The report out sessions for the teams could have been enhanced by having the rest of the participants submit questions and feedback (via note cards or online).
• Scheduling - it was during the quarter for some of us and participating in the full 3 days was impossible. Chairs are expected to be present during student recognition events and cannot take 3 days off to go to a workshop, no matter how useful it is.
• Maybe presentations of successful case studies so we can learn from others successes.
• Define the goals clearly from before the meeting through the meeting; it was sometimes vague and I felt there was some confusion.
• Reconvene the groups to understand progress and roadblocks or challenges and their solutions.
• The one suggestion I can make is a very picayune tweak to the group reports. I would have welcomed any suggestions the other teams might have had for us (beyond the institution that we were paired with- which was also a very helpful exercise).
• Direct feedback from experts on our team's plans would have been very valuable.
• The resources presentations were all over the map. Have an actual set of resources and review them briefly. Replace the "talks" with some workshop-like materials on approaches, techniques, fundamentals on EBIPs and cultural change for institutions. Also, since teams had non-education people with them, it would be VERY useful to use less jargon at this meeting and to not act as if everyone in the room knows what you're talking about (or has buy-in yet.)
• I would have liked to see other institutions present some poster papers about innovative things they've been doing. It helps stimulate discussion, leads to more "take home" messages, and lets you know who to contact afterward to hear about specific examples of things tried and successes/failures.

We asked, "What could this organization (APLU / Sloan/ Grouping of people at the workshop) do next that will help you and your campus initiative?" The responses centered around the following suggestions:

Communicate the importance of this work with provosts:

• Ensure good visibility of projects like these.
• Contact Provost about participation, encourage support of initiative.
• Funding is always the key issue in getting any changes implemented! Keep reminding those provosts to pay attention to funding the innovators on their campus.
• At the APLU meeting of Provosts, it would be extremely helpful if you help to convince them that course releases are important for course reform. People are clamping down on those as money gets tighter.

Reconvene or have follow-up:

• The workshop itself was extremely useful and follow-up in December on expected deliverables will help light a fire under us if we backslide.
• Based on how the initiatives get off the ground, it may be valuable to bring together the same group of people. Or the leadership among them. To learn from each other's experiences and stimulate further work. Getting something off the ground usually takes repeated efforts.
• Check in with us around progress and next steps.
• Reconvene in a year to understand progress. What worked?
• I really look forward to the report outs and an update on what worked and didn't work.
• Bring us back together in about a year or so to check in about progress: what has been achieved, what has not been achieved, why, and what can we learn from this?
• Help hold us accountable to our plans. Provide seed money to propagate our plans initially.
• Giving feedback on a draft proposal might be helpful. I am grateful that our team was selected to participate in the workshop. Thank you for this wonderful opportunity!

Provide resources and knowledge of what works:

• Develop initiatives explicitly focused on efforts to convert the unconverted, as true institutional change will require this. I was impressed that the Physics Dept at CU Boulder, a clear leader in active learning, has 50% of its faculty who are "not agnostic, but antagonist" towards active learning efforts. The SMTI field must think and talk outside of its confines. In evangelizing, I caution the community to avoid even terms like "evidenced based." Labels that indirectly criticize your potential convert will only work against your efforts. A new marketing campaign is essential in achieving the new normal.
• Have a list of "experts" that could visit and present at our institution on the specific reform areas that we picked.
• Present successful case studies.
• Hold a webinar to share our progress and hear others' progress and provide opportunities to get feedback and draw on expertise of others.
• Build the community; keep it informed on best practices and models for change.

What we would change:

• Increase the break between meetings.
• Shorten the tools and resources from 60 minutes to 30 minutes. Encourage presenters to just provide pointers to a pre-distributed resources/tools document. Ensure that there is not overlap between the SMTI session and Workshop session.
• Shorten the pair-share session with two teams from 45 minutes to 20 minutes.
• Add a second pair-share session before the group report outs for 20 minutes to address the desire for the teams to get feedback on their final plans.
• Give each team 15 minutes for last minute revisions before the report outs.
• If possible, increase the time for report outs to provide some time for group discussion on each plan.
• Provide a poster session.