Pre-Workshop Readings - Question 2

I do have a weird idea for this. I know this is kind of out-there, but I might as well propose it. TBL offers a really great environment for doing some “scholarship of teaching and learning” experiments on inclusion. I am envisioning even a potentially “blind” trial. Here is how it might work:

Professors devote 30 min of a class period for students newly assigned to a team to (a) watch a short video, (b) discuss the video and put together team norms based on the video and (c) fill out a pre-class feedback form with some data. Ideally the professor would not be present so that the professor did not know which of the three groups teams were assigned to.

Control group: No introductory session or an introductory video unrelated to inclusion

Experimental group 1: Introduces theories on privilege, oppression and microaggressions (possibly in addition to the skills from group 2).

Experimental group 2: Introduces skills in boundary-setting, listening, assertiveness, and shame-resilience without reference to privilege, oppression & microagressions

The “group norms” would be free form, and each team could establish whatever norms they felt appropriate after a discussion of the video. They might submit this to a centralized experimenter who would be collecting the data.

In the middle and end of the course, the team would fill out similar evaluations of their team dynamic and how it has evolved over the semester. The professors could also submit an evaluation of team dynamics.

*

I realize that this seems unrelated to the readings for the TBL workshop. But I think an experiment like this could bring in people who are interested in inclusion, and interested in building a base of evidence on inclusion. It could drum up interest and coordination across many different universities. I think this fits strongly with engaging policy and a shared vision.

A couple possible strategies to support the use of TBL and increase its impact:

(1) Encourage communities of practice across disciplines. One institution may not have enough critical mass within the Economics department to have like-minded faculty talk about TBL and share their experiences. Particularly in these cases it may be useful to facilitate the creation of a group of like-minded faculty that trascends economics, bringing together folks from physics, chemistry, etc. Beyond the shared knowledge and the sense of community (not operating in isolation), this group may be able to tackle common challenges such as classrooms that are more conducive to TBL/flipped classroom techniques, internal grants or time-off or student support to create TBL applications within their disciplines, etc. If an institution has a center for teaching and learning or equivalent office, perhaps they could be the ones spearheading the creation of this community of practice. We could reach out to them directly, or (even better) through one of their economics faculty who has taken one of our TBL workshops.

(2) I think having a peer mentor within economics that could observe classes, give feedback, act as a sounding board, is important for the quality and permanence of TBL implementations. Since some institutions may start with only professor who is interested in implementing TBL in economics, perhaps we could offer the services of an experienced professor from another institution, who could come by one day to serve the peer mentor functions, at least initially.

As a new faculty member, I always appreciated the opportunity to meet and discuss teaching with more experienced faculty members in the discipline. At the community college level this can be a challenge since we are often the only economist in our departments and those departments are a mix of other disciplines (some business and some social science). Therefore, travel to conferences was/is my preferred way to have this interaction. One issue that faces CCC faculty is the lack of funds for travel. This can be circumvented with scholarships to attend conferences or local or regional faculty development activities.

However, in order to move out of the Quadrant I phase we need faculty to “develop new teaching conceptions and/or practices” (Beach, et. al, 2012). One of the greatest predictors of lasting implementation is “interest”. Since economists are researchers at heart and since research requirements have proven to be a barrier to adoption, creating an outlet for research in TBL (and possibly other active learning strategies) may be an effective compromise. Since I am not at a research institution, I will leave this to my colleagues to define how this outlet would best be structured (online or print journal, compilation of essays published annually, regional TBL conferences with papers and proceedings published).

“Support for TBL teaching practices that move us out of the Dissemination quadrant…” is ambitious. One of the primary obstacles, and noted on page 57 of Beach, Henderson and Finkelstein 2012 is that there may be far from consensus that there is a problem. There are relatively few articles that show success with non-traditional methods in economics (Beach, Henderson and Finkelstein, 2012, p. 54). I would add that TBL has an even more sparse set of studies to point to. We need more data on the effectiveness of TBL. Discussions on these points would definitely be needed before moving beyond the efforts of individual faculty members.

Designing support as needed will take more commitment than is typically found. If STEM professors are further along, faculty within the institution may be willing outline their best practices. I think Marcelo’s comment above is therefore apt. If there are too few in the economics department, efforts need to extend beyond the department.

What’s in it for STEM faculty? Building on Ashely’s comments, perhaps the data generated with empirical work would be useful to all the disciplines involved. Also, the rhetoric of many institutions seems to be in the direction of collaboration across disciplines, so this could be a way make that a reality. The TBL effort could thus plug into existing efforts. It would obviously be more difficult where none exist.

Not I agree with this, though it may depend what you mean by 'non-traditional methods'. There are now tons of articles that show the effectiveness of active learning techniques in economics!

Again, I'm going to echo Marcelo - communities of practice can be super-important. To that end, I think we should encourage people to connect with whoever the faculty developer is on their own campus, both to find out if there are others on their campus using TBL and to enlist their support (how many of you all have talked to your local CTL director?) I thought the point in the Change article about how the different research communities don't talk to each other - and need to - was sad but spot on. Within Econ, I think building THIS community of practice is going to be key. The AE library will lower barriers to entry but folks need to know that there are also people to ask for advice when they start hitting bumps in implementation.

To increase implementations of TBL (or just TBL AEs), more evidence that it benefits students (e.g., improved economics knowledge, persistence in the discipline or to the next year of college, critical thinking) could help. Empirical evidence, perhaps from an RCT of TBL AEs, can provide convincing evidence. This kind of work could increase communication among the different research communities mentioned in BHF (2012, p. 54) and help build the TBL research base.

I think we could help develop reflective teachers by modeling it. We could include with our materials some reflection on our experience implementing TBL, especially focusing on the challenges early on. That may help when people try it but become frustrated.
Evidence is also important. There is an effort going on to establish some standard learning outcomes and assessments for economics beyond the TUCE, starting I believe with econometrics. Where possible, if we can link our material to standard learning outcomes and means of assessment we provide means for instructors to assess their current practice, try TBL and then see the impact. Providing an external benchmark can also give faculty cover if trying a new method leads to lower course evaluations, which I think is a commonly perceived risk.
Lastly, we could take some time to consider broadly what our vision is, in the hopes of working towards box IV. Why was this project worthy of NSF funding? Presumably because better teaching has positive spillovers beyond just our own classrooms and students. Perhaps we could provide some sort statement along these lines...

I agree with several others above that evidence of effectiveness is key. As Alan mentions briefly, even moving beyond principles courses. For instance, I use variants of TBL in intermediate macro and econometrics. Can we extrapolate its effectiveness to these courses using assessment?

In terms of facilitating conversations within and across institutions, my experience as a QR Center director leads me to believe that there could be some outreach done to these kinds of centers that span all STEM disciplines (at least at the introductory levels). Either through and teaching and learning or QR center, many institutions offer faculty working groups related to innovative teaching practices. These could even be offered to a wider (regional) community, since not every institution has the capacity for this kind of resource.

I agree that evidence of effectiveness of TBL in promoting learning outcomes across the economics curriculum is important. If we could also show that TBL increases diversity or perhaps has a positive impact on the outcomes and persistence of females/minorities in economics that would be very helpful.

There are many others who have spoken about a community or mentor who can support new users. I agree that this would be helpful in moving to sustained adoption. Here's a weird idea ... I was told that back in the day William Greene used to serve as the customer support for LIMDEP. That is, when you sent in a question, your answer came from Bill. Maybe there could be an "ask the expert" option on the website that is answered by Scott/Mark/Phil.

Getting to Boxes 3 & 4 is challenging, I think. I know very little about institutional change and look forward to learning more at the workshop. Personally, I feel like it happened almost accidentally in my department. Some junior professors were interested in classroom experiments. We used it and started doing research into its efficacy at promoting student achievement. We talked about it some to our colleagues, but I wouldn't say we were particularly evangelical. But slowly (for nearly 2 decades now) - especially as new faculty were hired - there was a consistent move towards using a variety of active learning techniques in the classroom. How to make that happen elsewhere, I'm not sure. But I think it does have something to do with the junior faculty (who eventually become the senior faculty and leaders).

So, how do we move beyond dissemination? My take-away from the RBIS article was that those who persist with using research-based instruction strategies tend to be a self-selected bunch. I quote: "...high users are more likely to read journals about teaching, attend talks and workshops related to teaching, be female, be satisfied with meeting instructional goals, publish fewer research articles, and teach smaller classes." (p. 11). To me, this sounds like a specific type of faculty member at a specific type of institution, even though institution and position are allegedly not significant (hard to know for sure, given that we don't know how much variation there is in either the institution or the position variables, or the extent to which institution, position, and gender are collinear).

My comment on the above result from the article reflects the challenge that we are going to run into with economists. If we want to move beyond dissemination, to convince economists, we're going to need GOOD evidence from well-designed studies that anticipate the types of comments we are likely to get from statistically literate referees and, to the extent possible, addresses those in the study design. I think that we also need to clearly articulate what the goal is. Is our goal adoption of the full TBL pedagogy in intro econ courses, or is adoption of some portion of the full pedagogy the goal?

[quote=Marcelo Clerici-Arias]

I could not agree more with Marcelo and Jennifer's comment on the community of practice. A bottoms-up faculty driven approach could be instrumental in changing the culture of a community. This is not to say a top-down approach would not work, shared vision is the key. Some colleagues and I had moderate success in developing a hybrid voluntary faculty exchange program where colleagues host and observe each others classroom with no punitive stigma attached. These opportunities could be used to highlight different teaching strategies such as TBL which could be further showcased during annual teaching retreats/workshops. On an experimental basis, the college is now completely renovating two classrooms with state of the art technologies to enable active learning and will be collecting data to analyze and disseminate.

From a community college perspective where faculty are teaching 5 to 7 (or more courses) per semester in a variety of settings: online, traditional, dual credit (driving to multiple high schools and college campuses) - Time is the biggest impediment. The more that can be done to help faculty implement, improve, and measure the effectiveness the more likely faculty will adopt new teaching methods.

I completely agree with Marcelo and others on Peer Mentoring. This would provide support and ease faculty implementation reducing barriers of continuation. These programs could be created within and between institutions and potentially modeled after the TBL program for trainers and mentors.

Providing specific methods for evaluation may help faculty measure the impact of their programs, spread the word within their institution, and build on the evidence of effectiveness.

Question 2 (from the workshop pre-reading emailed to you): How can we design supports for use of TBL teaching practices in economics that move us out of the Dissemination quadrant to create even more impact? What would that support need to look like for different institutional contexts?

In terms of the taxonomy presented in the Change magazine article, to move beyond disseminating a prescribed set of teaching practices to individuals, we need to spend more time developing more teaching reflection on the part of sage-on-the-stage economics teachers who might potentially shift their classes toward including more active learning, including TBL. Also, we need to spend more time trying to develop supportive communities for teachers using TBL and other active learning pedagogy.

The group of faculty that I have worked most with so far are colleagues already interested in active learning, far down the road of reflecting deeply about their teaching. In conversation with student colleagues about teaching and learning, I have drawn extensively from reading in the cognitive science literature on how students learn best and the ramifications for teaching practice. That seems the place to start if we want to reach a wider audience for the conversation on TBL. However, that seems beyond the reasonable scope of the current endeavor.

More promising and feasible would be to develop and support virtual and regional communities of practice using an array of approaches that include workshops for entire departments of economics, the development of regional groups of economists teaching extensively with active learning pedagogy, including TBL, and supporting the wider group of teachers using TBL via social media and other means of sustaining virtual communities.