The Mineralogy Concept Inventory: a statistically validated concept inventory to measure learning gains
Wednesday
4:30pm-5:45pm
Beren Auditorium
Poster Session Part of
Wednesday Poster Session
Since 2007, an initiative to improve science education at the University of British Columbia (UBC) has prompted a shift towards more learner-centred pedagogies in undergraduate courses, including introductory mineralogy. To assess the impact on student learning, we developed a statistically-validated mineralogy concept inventory and implemented it as a pre- and post-assessment in two settings: (1) at UBC, which uses learner-centred pedagogy, and (2) at another university (UX), which has similar course content but uses more traditional, instructor-centred teaching methods.
Items on the Mineralogy Concept Inventory (MCI) were created through consultation with experts from both universities and think-aloud interviews with students. Grounded theory and Rasch analysis were used to validate the items. As much as possible, the item answers and distractors were written using student language, so as not to confuse the comprehension of technical jargon with the understanding of concepts. Rasch analysis was used to generate a scaled test score that more accurately reflects the difficulty of questions. All questions were analyzed for differential item functioning to ensure that the test is not biased towards any subset of the population.
The pre- and post-assessment was administered to 88 students at UBC and 63 students at UX. Results from the pre- and post-assessment indicate that UBC had significantly higher learning gains than UX (normalized learning gain of +0.56 and +0.33 respectively; effect size of 2.64 and 1.43 respectively). Interestingly, UBC had a lower average score on the pre-assessment, likely because enrolment in the mineralogy course requires no geology pre-requisites, while students at UX are required to take at least one 100-level geology course. This suggests that instructional methods can overcome a lack of prior knowledge. The MCI is a validated concept inventory that can be implemented in any introductory mineralogy course to assess prior knowledge and learning gains.
Items on the Mineralogy Concept Inventory (MCI) were created through consultation with experts from both universities and think-aloud interviews with students. Grounded theory and Rasch analysis were used to validate the items. As much as possible, the item answers and distractors were written using student language, so as not to confuse the comprehension of technical jargon with the understanding of concepts. Rasch analysis was used to generate a scaled test score that more accurately reflects the difficulty of questions. All questions were analyzed for differential item functioning to ensure that the test is not biased towards any subset of the population.
The pre- and post-assessment was administered to 88 students at UBC and 63 students at UX. Results from the pre- and post-assessment indicate that UBC had significantly higher learning gains than UX (normalized learning gain of +0.56 and +0.33 respectively; effect size of 2.64 and 1.43 respectively). Interestingly, UBC had a lower average score on the pre-assessment, likely because enrolment in the mineralogy course requires no geology pre-requisites, while students at UX are required to take at least one 100-level geology course. This suggests that instructional methods can overcome a lack of prior knowledge. The MCI is a validated concept inventory that can be implemented in any introductory mineralogy course to assess prior knowledge and learning gains.