Materials Testing with Data Collection

Thank you for deciding to be involved in testing InTeGrate curriculum. You are helping blaze the trail to a new, collaborative process of developing teaching materials and assessing student learning that will lead to a set of rich, relevant, flexible and pedagogically refined materials.

Jump to: Module/Course Testing GLE Question Testing Questions?

Module or Course Testing with Data Collection

The first step is for each participant to fill out a registration form which will:

1. Notify SERC to give you permissions to module material and the survey instruments.
   
2. Capture demographic information we need and give us a record of your participation in the project. Please indicate you are a materials tester in the last question on the form.

Data Check A: Before your course begins:

• Obtain IRB approval
• Decide how you will provide data to InTeGrate
• Decide on a Student ID, which will be used to label all student material submitted to InTeGrate (e.g. the student ID your institution assigns each student)
  
• Decide on a mechanism to measure student engagement, such as class attendance
  
• Set up a SERC account
• Complete the Course Setup Form (to initiate the processing of your official contract and access to GLE questions)
  

Data Check B: At the begining of your course (first week of class)

• Collect student consent forms (if needed)
• Submit initial roster of student ID's on your course status page
  
• Direct students to initial attitudinal survey
• Administer the initial GLE multiple choice questions

Data Check C: While you are teaching InTeGrate curriculum

• Administer and collect summative assessments, ensuring they are labeled with student ID numbers and that student names have been removed
  

Data Check D: At the end of your course

• Administer the final GLE content questions (multiple choice and essays)
• Direct students to final attitudinal survey
• Provide a final course roster, measure of student engagement, and ungraded student assessment responses, labeled clearly to indicate which assessment they pertain to and with student ID numbers only (no names)
  
• Complete the faculty reflection survey and provide any feedback you would like to provide to the material authors
  

Geoscience Literacy Exam (GLE) Question Testing

For those that are testing GLE questions but not a full InTeGrate module or course, an abbreviated list of data collection steps are listed below. After you are registered (see above), we can give you access to view the full set of Geoscience Literacy Exam questions so that you can select which questions you would like to administer. In addition to the questions you select, all testers will administer a common set of 8 GLE questions and two essay questions, which will allow us to make comparisons across a broad spectrum of institutions and disciplines. The GLE multiple choice questions will be administered as both a pre- and post-test, while the essay questions will only be administered at the end of the term.

Before your course begins:

1. Fill out the registration form, indicating that you are testing GLE questions in the last question
   
2. Set up a SERC account if you don't have one already
3. Once you receive access, indicate the GLE questions of interest to test in your classroom. InTeGrate will compile your selected GLE questions (plus the common set of 8 questions) into a pdf file, which will be sent to you prior to the start of your course.
   
4. Before your semester begins, obtain IRB approval (if required at your institution, see link for more details)
5. Decide on a Student ID
6. Complete the first portions of the Course Setup Form- everything through the GLE Multiple Choice (1 and 2) section. This will give us information about your course, timing for the data collection, generate an online attitudinal survey for your individual course, and give you a place to upload data (see Using the Course Status Page for details).
   

At the beginning of your course (first week of class):

• Collect student consent forms (if this is required by your IRB or if you're Exempt and covered by Carleton's IRB) with student names and ID's and provide to InTeGrate
• Direct students to your pre online attitudinal survey (student's will need to enter their ID #, no names)
  
• Administer the provided GLE questions as a pre-test (these will be submitted to InTeGrate with student ID's, no names)
  

At the end of your course (last week of class):

• Administer the post GLE test - same multiple choice questions with two additional essay questions (these will be submitted to InTeGrate with student ID's, no names)
• Direct students to your post online attitudinal survey (student's will need to enter their ID #, no names)
• Provide a final course roster on your course status page
  
  Show How to provide student ID's
  Hide
  Your course status page provides a place to copy and paste in a list of student IDs. If you keep an electronic student grade book or have access to a course roster that includes the student ID numbers, this should be just a simple copy and paste. List one student ID per line without delimiters such as commas. Copying a column from Excel or Google Docs works perfectly (just be careful the column header isn't included). You can submit this list more than once if there's an error or change in enrollment, but be sure to include the full roster each time as it does not merge the changes; rather, it overwrites previous rosters.
• Provide attendance data if it is possible, which acts as a proxy for student engagement. We would appreciate the data if you're already collecting it for your class.
• Provide your GLE data to InTeGrate
  
  Show How to provide GLE data
  Hide
  
  

  • Label all data with the corresponding student ID if they aren't there already (no student names). We've found this is often easier to do by hand before scanning, but can be added electronically as well (blue painters tape works well to black out names, but can be removed without damaging the paper. This may not work with all scanners).
  • If possible, provide the data in an electronic format through the upload forms on the course status page. This is likely the simplest way for you to provide us with copies of student work: scan the stack with your copier and provide us with the resulting PDF file. Standard image formats are acceptable if scanning to PDF is unavailable. JPEG and PNG formats are the safest bet for image scans. Be sure the documents are clearly labeled with the student ID. And in cases where the data needed by the project is mixed with other material (e.g. a GLE question embedded in a larger exam), consider how you might scan only the part that is relevant.
  • If desired, you may provide the pre-test results earlier in the semester rather than waiting until the end.
    
  • If you need to provide the data as paper, do not send the only copy of the documents. Always photocopy the original documents before sending. Documents can be mailed to the SERC office and should include a cover sheet with a clear description of what they are (e.g. what course they relate to, what questions are included). Packages should be sent with a tracking number regardless of shipment method (USPS, FedEx, etc.). Please forward this tracking number to Krista Herbstrith so that we know when to expect the package (kherbstr (at) carleton (dot) edu).
  • Science Education Resource Center
    Attn: Krista Herbstrith
    200 Division Street, Suite 210
    Northfield, MN 55057
  
  

Questions?

Check out the Frequently Asked Questions page or contact your webteam liaison, Kristin O'Connell (koconnell at carleton dot edu)