Norms (Resource & Ed Researcher)

Inclusion and Respect within our Group

• Make sure all participants have a voice and get a chance to talk
• Be respectful in how questions are asked
• Respect others' statements and opinions even if you disagree!
• Active listening and eye contact
• Speak up

Use Positive and Respectful Language

• Be respectful of language but also give each other the benefit of the doubt
• Be sure to pause to define vocabulary (jargon)
• Avoid deficit language and encourage an asset/anti-deficit apporach

Openness

• Don't be afraid to ask questions
• Be open to all ideas
• Leave your box... step out of your frame of reference
• Don't take things personally

Stay Focused and Be Concise

• In discussion, build on knowledge (e.g., stay on topic until done)
• Be concise
• Minimize rants
• Limit chit chat
• Limit any overly talkative participant should it curb full participation of others

Have Tangible Outcomes

• Create tangible actionable outcomes
• Advance the science

Advocate for Others

• Represent broader interests of the community

Save Work Often!

Follow-up Discussion from Guest Speaker: Jeff Froyd on Increasing the Impact

• Marketing? How do we do this without the marketing/business expertise?
  
• Just a marketing expert won't work either, need to deeply understand the product
• Engage adopters from the beginning, instead of marketing after development
• What strategies work to convince faculty to change/adopt? Evidence doesn't seem to work.
• Need to decrease "activation energy" for change
• Professional development is necessary but not sufficient
• How can change and teaching innovations be incentivized for faculty?
• Faculty are more interested in teaching than you may think (give them some credit), doesn't mean they will adopt innovation
• They may think they are doing a good job. Don't start by telling them they are doing it wrong!
• Who do you want to adopt? What are their reasons for adopting?
• What can we learn from medical community? Speed of research to adoption
  
• Some things still don't make it through to practice
• Medicine has lots of institutional structure to communicate between research and practice
• People more likely to uptake simple changes idea - too complicated is offputting
• ex. Instead of food pyramid: drink low fat milk
• Need to synthesize down to very focused talking points
• Interactive dissemination? Interactive development?
• Not separate ideas - dissemination and development go hand in hand
• Engage with your adopters - create advisory board of potential users
• Sometimes a piece you overlooked is more interesting to users than your key feature
• Conference booths, workshops with departments, test cases in small increment - don't wait for whole package, modular
• Time and resources? How much more does it cost to move up the rubric to have a successful program?
• Proposals- most of budget on development in a specific context
• Budget should be spent more on multiple contexts and engaging more people
• Don't spend it all on assessment either, since evidence isn't most compelling reason to adopt
• Again, don't wait until end for large dissemination effort, spread it out over the development

Gallery Walk 1 Wrap-up

What are the primary goals or vision of your organization in an educational context?

• UNAVCO: Disseminate and educate beyond key stakeholders (university faculty)
• Aligned by strategic plan; advisory committee; staff retreat review performance metrics....
• Facilitate workshops, field work, online conferences social media, conferences.
• TERC: Main focus On K-12; improve math and science for all learners; contribute to research literature on best practices in STEM; provide content knowledge for Teachers; identify need based on experience and literature; NGSS on science and education practices, need to build into college and lifelong learning; Settings: K-12 classrooms K12 in-service professional development; Climate literacy stakeholders; informal education (museum, aquariums)
• Mission driven agencies would like to, but don't have capacity, to write articles on research on education, what works
• Priorities....getting info out to public, so little time for recording results in literature.
• IODP: Work with scientists directly; inform about STEM careers; educators involved with research cruises; work directly with large network of educators about what to create next. Challenge_- how to get info out to larger community.
• USGS: charge to explore the country; broad-based responsibility for citizen understanding; effort to integrate education and research; Getting science out to mission areas, general public, K-12 in form that they can easily access and use. What we have, what does it look like, how do you use it–info for teachers. Citizen Science: huge inventory and investment; Lab and field work (internships)–match has to be right, must be good match with scientists' proposals; Interagency STEM collaborations a priority–partnerships are central NASA, NOAA, USGS. ESCI a great way to engage students, as gateway also to other STEM disciplines.
• SERC: NAGT–High quality teaching activities; Professional Development; Work with everybody, faculty, in the classroom, outreach, field work....Connections across the community.
• Need to worry about maintenance, upkeep of resources;
• Illinois State Survey; emphasis on public outreach; Similar issues as USGS, reaching out to K12 teachers; getting ready for NGSS; field trips for public, guidebooks across state to promote for teachers; scope is so large, need to prioritize bang for buck, scope, limited resources;
• Outreach for large energy grant; Part of mission is outreach and education driven, community, government education; staff availability (what they are interested in), needs–defined by topical issues like fracking; teachers are desperate for resources for NGSS; Provide unbiased science and education to public.
• IRIS: NSF Facility; How to make decisions: formal program evaluations, program / product gap analysis; How to decide what we do: fun and interesting; creativity an important component.
• Input from Advisory Groups to decide what to do.
• Interactive development, TERC engages teachers, pilot demonstrations, feedback into material development.

Gallery Walk 2 Wrap-up

Specific to diversity:

What has been done?

• Workforce Development; geoscience career opportunities; focused on ethic minorities, but expanded to socio-economic; target first and second year students in new funding, come to Boulder to develop research-ready skills; provide infrastructure.
  
• Community workshops to inform work; experts brought in to advise community aware of how to broaden participation.
• EarthLabs, iteratively developed, when piloting materials, assessments had to be done orally due to low reading competencies; Have and have not: Texas had funds, Mississippi did not have access to funding
• Small targeted internships partnerships with HBCUs and MSIs
• System wide outreach across program with personal relationships with MSIs; more successful
• Programs come to end, not sustainable.
• Problems with recruitment.
• Recruitment is more than advertising; get on the telephone...with colleagues;
• Early engagement, there is a pipeline for "best and brightest"; 2 year course in certification. Don't just think about cloning yourself... e.g. Water Technology certification
• IGSS partnered with Parks, to provide walking tours; coastal processes/erosion; way to reach more diverse audience in Chicago; blend with culture and history.
• Extension services; plugged into all counties in state; work with depressed counties to learn what their needs are.
• SERC: whole suite of resources at the program level; Whole Student Model (engagement, capacity, continuity); teams to teach sustainability; implementation teams implementing in new contexts;
• Sage 2YC, develop compatible suites of information for application to 2YCs;
• Workforce resources
• Minority Recruitment Lecture Series: IRIS and UNAVCO; Alumni from internship programs visit department to show students what careers can look like; financial benefits of careers in geophysics; what kinds of degrees are needed;
• Intensive professional development workshops focused on in service teachers.
• Partnerships with 2YCs that have sequestration programs; program feeds directly into U Illinois

What is on horizon?

• Monster.com broker short duration field work opportunities; Lower hurdle opportunities to participate without taking whole summer.
  
• NGSS need new curriculum; evaluated on how activities are aligned with standards.
• Find ways to make materials flexible enough to be adaptable and locally relevant; learning pathways that identify key issues in a region, discussion questions with students, point towards resources that can be used in regional contexts.
• HOw to increase chances for serendipitous encounters with geoscience?
• State Surveys, need to demonstrate geoscientists are real people and here's how we're serving you.
• GSA Diversity: national and section meeting; student lunches; mentoring programs;
• AMS: women in ATM lunches, board on women and minorities; .....
• SACNAS–Awards at meeting; no award in geoscience, needs representation. Whole meeting focused on students. There is a geoscience field trip every year.
• AESIS, NABG
• Presidential Awards for Teachers
• Challenges: Sustainability. Scale Up.
• Priorities Shift. Mission change. Mission Creep.

Culturally Relevant Resources

Its content focuses explicitly on the geological and other natural attributes of a place

• USGS has done a lot of work on Tribal Lands, GIS, to acquire information that they can draw from, to understand resources on their lands; also a bridge to rational inquiry; get into cultural story, history of land; related to natural science; for decision-making
• Transferable to multiple locations; place based, but place based anywhere.
• Place based isn't just natural; could be place based for topical issues like brown fields; Environmental Justice
• This framework may not be relevant to more general case; continuum between fully place based or not, there is a balance point where an activity is most useful.
• Identify topics that are necessarily place based, but generally applicable. Identify overarching frameworks, and then applying details to locally relevant situation.
• More skills than knowledge. How do you get to the answer, process important.
• General topics/info can be oriented so that they are applied to place based.
• Illiinois Kickapoo River point bar exercise
• EET exercise for stream flow and precipitation. Learn skills on how to acquire data; ask questions for a given region. Develop materials that are comparative for different regions.
• Temporal aspects of our science; fits in nicely with oral tradition; see how things have changed over generations.
• Big questions in Earth Science classes: anything you teach, minerals, erosion, can be related back to your region.
• Bring this back to societal issues: hazards, resources,.... so that basic questions have relevance to societal issues.
• Must provide scaffolding to give educators ideas about how to use the materials transferred from one place to another.
• Pedagogy similar to many other active learning activities; need training in how to use Problem Based Learning and see connections, and be able to transfer to local settings....
• IRIS will not be the agency to provide specific info on how to do pedagogy (out of scope of this group), but would like to refer to groups that have done this.
• Local partners, State Surveys that have this mission, State Teacher Organizations are important partners.
• EVENT BASED: place is important, but events happen all the time;
• IRIS a good model for rapid outreach; have "canned" activity ready to go for next event; same with streamflow , floods, Tools where teachers are familiar with the "recipe"

It integrates, or at least acknowledges, the diverse meanings that place holds for the instructor, the students, and the community

It teaches by authentic experiences in that place, or in an environment that strongly evokes that place

• Conducting experiment, Ritual, Contributing to regulations for urban planning; Building infrastructure; (Group 3); readily linked to place.

It promotes and supports ecologically- and culturally-sustainable living in that place

• This may not be broadly applicable;
• More generally place based materials give students a context ...(Group 3)
• Better fit for other NGOs Sierra Club
• Give the evidence....e.g. historic topo sheets, number of streams that aren't there anymore.
• PBL used to teach issues.
• Teach to inform? Or Teach to get action?
• PBL is a kind of pedagogy, to get students engaged, it is not an end to itself. Use as portal for students to understand world, make decisions, .....Decision-making based on understanding.
• One of many tools.
• Central to geoscience pedagogy (PBL and Event Based)

It enriches the sense of place of students and instructor

Comments on these 5 characteristics (too restrictive? too broad?)

Assessment and Evaluation of Materials

Current methods and examples for assessing effectiveness

• Institutional Goals
• Useful and used
• Gather attitudinal data
• Knowledge data, pre and post
• Sample essays
• Individual student data, are these materials working in the classroom
• Rubric for creation of materials
• 
  
• Enrollments, Retention Rates, Graduation Rates
• Survey Monkey after Public Field Trip
• Social media (Facebook, Twitter Followers)
• Effectiveness or impact: longitudinal evaluation study to survey all past teachers in the project; are they still using products, how are they using the products
• Are metrics such as web stats more or less important than high impact on a few individuals
• e.g. IODP., shallow, relatively cost effective broadcasting vs. bringing a cohort of teachers on ship (expensive and limited).
• Implementation fidelity? How closely will a teacher actually employ activity, there may be some deviation from initial design. What is power of impact if they deviate: diminishes intent, or possibly gains value?
• Adoption vs. Adaptation.
• Qualitative vs. Quantitative data.
• Inter view protocols, surveys (e.g. pre- and post), observational protocols, focus groups,
  
• Transfer skills to new situation.
• Compare general skills to application to local setting.
• How to make determination of to go with evaluation---some materials are very expensive. Need to create new revenues. Increased pressure for oversight. How far to extend limited evaluation resources. How to educate NSF about why we cut-off at certain point evaluation, informed by return on investment.
• Embedded assessments for formative purposes.

How could we evaluate effectiveness with underrepresented groups?