Using Student Data from Your Own College or University to Identify the Best Predictors of Student Success in College

Through the project itself, by exploring the stages of the research process, students are exposed to skills satisfying each of Hansen's proficiencies.

Specifically, the student will:

• Be able to explain to college administrators the relative importance of standardized test scores and grade point averages in predicting student success in college.
• Be able to clearly state a research question and a corresponding hypothesis.
• Be able to critically evaluate and synthesize literature relevant to the research question.
• Be able to obtain data from the Office of Institutional Research, manipulate the data and display them in an Excel spreadsheet, transfer the data into a statistical software package such as SPSS or SAS, and conduct a linear regression analysis.
• Be able to address the results in the context of the broader topic and present the results in written and oral form.
  (These goals were modified from those stated by Elizabeth Perry-Sizemore in her service learning/undergraduate research Starting Point examples.)

This example comes from an experience with an individual student who wrote a capstone thesis in public administration. Since capstone theses are written at the end of a student's program of study, the student had completed a policy analysis course and a research methods course. It would be a suitable final thesis topic for students majoring in economics, political science, sociology, or education. The project could be modified to be a group project in a research methods class or in an economics of education special topics course.

Before beginning any individual research project with a student, it is a good idea to create a contract with the student that sets forth the student's responsibilities and the faculty mentor's responsibilities. This should include a timetable for completing various components of the research project such as the literature review, stating the thesis statement/research hypothesis, data gathering, data analysis, outline of the paper, first draft of the paper, etc. It is also a good idea to set up a regular meeting time with the student. Even good students can founder when they are left too much on their own. We are lucky at the University of North Florida that students who wish to undertake an Honors thesis can enroll in a one hour thesis prospectus course that walks them through the beginning stages of a research project. A copy of a student research contract is here: undergraduate research student contract (Microsoft Word 2007 (.docx) 14kB Jul14 10). A copy of the syllabus for this one hour course is here: Syllabus for One Hour Research Prospectus Course (Microsoft Word 565kB Oct6 09).

The literature review is the first task to be conducted when the student begins the project. It is easy for students to become overwhelmed with the volume of information available on some topics. Therefore, it is a good idea to identify a few papers or a scholarly book for the student to read first. This accomplishes several objectives. First it identifies for the student the kind of scholarly articles that are appropriate for this type of project. Secondly, the references to those articles can be used for forward and backward searches of the literature using Google Scholar ( This site may be offline. ) or one of the many scholarly databases available at the library. Don't assume the students know how to do this. It is important to show students how to do an effective literature search. As an example, I identified the book Fair Game?: The Use of Standardized Admissions Tests in Higher Education by Rebecca Zwick of the Educational Testing Service as a starting point for the student. This was a book with many references, and it also served as a good overview of the numerous studies that had been done on this topic.

Once the student is familiar with the literature, it is important for you to meet with him or her to discuss the research thesis. The student will probably come to you with a vague concept of his or her research project, but vague is the operative word, here. The student will need your help in refining a very specific research question that contains a testable hypothesis within it. My student chose this topic because she had been a student with a high GPA, but a relatively low SAT score, when she entered college. Therefore, she felt strongly that GPA would predict success in college better than the SAT score. Together, we turned her personal belief into the following research questions:

1. What are the significant factors that influence a student's final cumulative grade point average at the time of gradation from college?

2. Do any of these factors interact with each other?

The student may also need some guidance in collecting data. Often a call from a faculty member is better received by people in institutional research than a call from a student. Even if the data are available on-line or in the library, don't assume the students know how to access it. Don't assume you know how to access it, either! Solicit help from your research librarian.

Once the data are obtained, students will also need help in transforming the raw data into functional variables in a regression analysis. This is often harder than it looks. They will probably have to recode several variables and/or create dummy variables from categorical variables. These things are second nature to us, but once again, don't assume that students know how to do this.

The student will need your help with the data analysis part of the project as well. They will have no problem running the regression analysis in one of the readily available statistical software packages, but interpreting the results is a whole different story! In my experience, it is a good idea to go over the computer printouts with the student in your office or somewhere where you have access to the data and the statistical software package. That way, when you look at the results with the student, you can ask if she tried entering one variable or another, and if she hasn't, you can try it right then. Using this trial and error method of model specification, allows the student to see that empirical research is sometimes as much of an art as a science.

Using this technique, she discovered that the results varied substantially when the data were separated by race. When only Caucasian students were included in the regression model, both the student's high school GPA and standardized test score had a positive and significant effect on the dependent variable (GPA at the time of graduation). However, when only non-Caucasian students were included in the regression equation,only the student's standardized test score positively affected the student's graduating GPA.

What did the student conclude from her research? That both high school GPA and standardized test scores are positive predictors of a student's graduating GPA for Caucasian students, but only the standardized test score is a positive predictor for non-Caucasian students. Also, judging from the size of the standardized coefficients, the GPA was a stronger predictor for the graduating GPAs of Caucasian students than the standardized test score. This last result pleased the student since her initial belief (GPA was a better indicator of college success than test scores) was partially substantiated, at least for her own subset of the population. She also discovered that Caucasian females had higher graduating GPAs than Caucasian males, and that there were significant differences in the graduating GPAs of students who majored in different colleges.

The student presented her results in a final paper as well as a poster that was presented at UNF's annual Undergraduate Research Symposium.

undergraduate research student contract (Microsoft Word 2007 (.docx) 14kB Jul14 10)

Syllabus for One Hour Research Prospectus Course (Microsoft Word 565kB Oct6 09)

1. Undergraduate research does not have to be publishable in economics journals to be worthwhile. Most economists believe that their undergraduate students cannot participate in research because they don't have sophisticated econometric skills. That may be true if they are trying to publish in the Review of Economics and Statistics, but the benefit of undergraduate research is in doing it, not necessarily publishing it. That said, there are several outlets for student research.

2. Mentoring students in undergraduate research projects is a valuable teaching activity, and it should be rewarded as such. Assisting students with research will not be successful on your campus unless it is rewarded. Many departments allow faculty to bank the number of independent studies they offer until the number reaches the typical number of students that they would have in a regular class. When the number reaches that level, the faculty member gets a course reduction. An even better model is to offer research mentoring classes as part of a faculty member's regular class load. Administrators need to put their money where their mouths are, if undergraduate research is going to be successful. At UNF, our Undergraduate Research Grants give faculty members $1000 to mentor a student. It's not much but it comes in handy during the summer. When it comes to promotion and tenure, departments should count mentoring students as both teaching and research. Many faculty co-author papers with their students that get presented at conferences and published in peer reviewed journals.

3. The project can be modified for different learning environments. The project can be modified as a group project in a research methods class rather than an individual student's thesis. Classmates can divide the work into different working groups such as a subgroup that writes the literature review and the thesis statement, a subgroup that collects and formats the data and writes the IRB application (if necessary), a subgroup that runs the regression analysis, etc. Then the entire group can come together to analyze the regression results and write parts of the paper and present the results to the class.

Student understanding and implementation of the research process and achievement of Hansen's proficiencies is ascertained by the research proposal, report, and presentation, using Greenlaw's guidelines for preparing a research proposal and rubric for grading a research paper.

Greenlaw, S.A. (2006). Doing economics: A guide to understanding and carrying out economic research. Boston: Houghton Mifflin.

Zwick, R. (2002). Fair game?: the use of standardized admissions tests in higher education. New York: Routledge Falmer