×

logo of Northeastern State University

Provenance: screen capture of the university logo top left corner at https://www.nsuok.edu/

Using Artificial Intelligence in Introductory Programming Courses

Ernst Bekkering, Northeastern State University

Location: Oklahoma

Context

Northeastern State University is a comprehensive, regional university in Northeast Oklahoma with campuses in Tahlequah, Broken Arrow, and Muskogee. It is the oldest institution of higher learning in the state of Oklahoma as well as one of the oldest institutions of higher learning west of the Mississippi River. Current enrollment is 5,225 (Fall 2022). About 25 % of students identify themselves as American Indian. The Computer Science major is located in the Greg Wadley College of Science & Health Professions and has about 140 majors.

This CURE was implemented in CS2014 Computer Science I, the first programming course in the major. It is taught in C++, but this CURE could just as easily be taught in Java or Python classes. Typical enrollment in the class is 25-30 students who meet twice a week for 75 minutes lecture and one 50 minute lab over a 16 week term. Prerequisites for the course consist of one basic mathematics course (completed or concurrent), or a MATH ACT score of at least 23, and computer proficiency. Statistics and Composition courses are not required. Students have limited knowledge of mathematics and no knowledge of statistics, so instruction in these areas is part of the CURE.

Many CUREs are dedicated research courses, but this is not one of them. By design, the CURE only counts for 50 out of 1,000 points in the course, which comes to half a letter grade. It is also clear from the learning goals of the course that the CURE is only part of the curriculum.

As overall learning goals for the course, students will be able to: 
1. Use variables, arrays, and constants of appropriate data types 
2. Use expressions and conditions 
3. Use appropriate decision structures 
4. Use appropriate repetition structures 
5. Use simple file structures for external data storage 
6. Use functions for modular program structure 
7. Create relevant internal documentation 
Since inclusion of the CURE, the following learning goal was added:
8. Make appropriate use of artificial intelligence

Target Audience: Introductory, Major, Non-major
CURE Duration:A full term

CURE Design

The research theme is evaluating how artificial intelligence can assist learning in computer programming. Students can generate their own data set from the textbook used for the class, or textbooks for other introductory programming courses, or find other sources of programming exercises to use. Students can choose a question that is more straightforward (comparing if the outcomes for two AI engines using book exercise descriptions function correctly) or more complex (using multiple AI engines with their own custom scoring mechanism). This allows more advanced students to seek out a greater challenge and stay engaged while also allowing other students not to feel overwhelmed by this - essentially - unstructured task. Students are also more likely to be successful because they are choosing their own focus.

Students work alone or in pairs of two to set up and perform the study. They are given the tools (programming exercises, access to AI engines, training in constructing measuring tools, statistical analysis, reporting formats, reference software, etc.) to set up and perform the study. Students have access to computer labs outside lecture and lab hours if they do not have their own computers. During the class lectures, time is set aside for getting feedback from other students in the class. After each deliverable, students are encouraged to present their intermediate results to the class.

Some students work better alone, others work better in groups. Students have the option to work in pairs, but with the added requirement to analyze the data for inter-rater reliability. They are required to formulate a plan to independently evaluate the data, without allowing simply duplicating the results or splitting the data analysis.

Throughout the course, emphasis is placed on the idea that there is no such thing as a "bad" result - that not finding statistically relevant differences may well be a meaningful conclusion as long as the study uses a sufficiently large data set. In this way, all students are able to achieve the goals of acquiring skills and experience with research design.

Since the students are relatively early in their college career (the course is at the sophomore level), the structure and time table are well-defined to encourage continuous work and discourage procrastination.

Students in other CS classes can benefit from the experience. This cannot be shared directly between students in CS2014 and other classes, but the instructor teaches other classes and carries experiences forward, and students often talk with each other especially when they work in informal study groups. Students who took the class with the CURE may encounter students without this experience in later courses.

Core Competencies:Analyzing and interpreting data, Asking questions (for science) and defining problems (for engineering), Constructing explanations (for science) and designing solutions (for engineering), Developing and using models, Planning and carrying out investigations
Nature of Research:Applied Research, Informatics/Computational Research, Translational Research

Tasks that Align Student and Research Goals

Instructional Materials

The Blackboard site for each course has a separate section "Research Resources" with the following supporting materials:
- The research process
- Research methods: experiments
- Research methods: simulations
- Research methods: surveys
- Research methods: statistical analysis
- Research in Computer Science
- Artificial Intelligence

Students who do not have their own computer, can use the lab computers in our building. All have Microsoft Word, Zotero, and Excel with the Analysis Tools installed.

Specific materials for this CURE:
- The current syllabus
- Zotero reference library
- OneDrive folder with resources such as files with programming exercise descriptions

Students can also search for other sources of programming exercise solutions on the Internet. For instance, this resource provides programming challenge solutions. They can also find them at this site. Some websites with programming exercise solutions have free trial periods. See for example this site.

Instructional Staffing

The instructor prepares the labs. No additional staffing is required. Grading is done by the instructor in the class. Since the college is very supportive of CUREs, I am considering recruiting a more senior CS major to assist as a teaching assistant specifically for the CURE.

×

picture of Dr. Bekkering

Provenance: Ernst Bekkering, Northeastern State University
Reuse: This item is offered under a Creative Commons Attribution-NonCommercial-ShareAlike license http://creativecommons.org/licenses/by-nc-sa/3.0/ You may reuse this item for non-commercial purposes as long as you provide attribution and offer any derivative works under a similar license.

Author Experience

 

Ernst Bekkering, Northeastern State University

Our college is introducing CUREs as a means to improve the retention in the STEM disciplines. I was one of the first cohort of four faculty members in the Sciences who participated in NSF-sponsored research to evaluate the effectiveness in doing so (link). This CURE is my first attempt to develop a full-semester student research study embedded within the regular programming course. It was first taught in the Spring 2024 semester.

CS2014 is the first programming class in our Computer Science curriculum. Historically, only 50% of students continue in the CS major and progress to CS2163, Computer Science II. CUREs have been reported to generate student enthusiasm and interest, so this is also an attempt to keep more students in the CS major.

Before AI gained the ability to generate computer code, my focus on homework was on having students practicing typing skills, since I considered typing one of the critical success factors (CSFs) in Computer Science. With the introduction of AI for programming, typing has been replaced by using AI in software development. Many aspects are still unknown and the use of AI in CS will develop rapidly. This CURE has been kept as flexible as possible by allowing students to select their own AI engine(s) and having them develop their own measurement instruments.

Advice for Implementation

The class is a mix of majors, non-majors, and undeclared students. Not all students are motivated to do a research study. Explain that the skills practiced for the study are transferable to other majors and environments.

Since students have limited or no experience in using AI (some are actually afraid of it), use ample demonstrations of using AI engines in the regular part of the course.

Break down the study by setting intermediate goals and deliverables. This discourages procrastination, which tends to be a problem still at the sophomore level. Discuss what is submitted as soon as possible to allow for early corrections.

Virtually no students have had exposure to experimental designs, so it is important to do plenty of scaffolded, skill-building activities. Make sure that computer labs have all software installed for those students who do not have access to their own computer, but also for active practice during lectures and labs.

Allow the use of paid subscriptions for AI engines, but provide sufficient information to free versions.

Iteration

The key to a successful research outcome lies in repetition: returning and referring to the hypothesis, original idea, methods, and data analysis, which helps develop accuracy and provides future direction. Using a test data set to check how well the scoring mechanism works helps to improve the accuracy of the instrument. Based on the results of the test data set, students are advised to adjust the data collection as needed. If the experimental results fail to support the hypotheses, redesigning may help to address the initial question.

Students can work on the CURE during lab time, but there is sufficient time allotted for the CURE in the work at home. Reference the Course Workload item.

For this course, the workload is calculated as follows: book pages minutes hours credit hours per crh mins at

• chapters 1-8 490 1960 32.7 weeks at home home
• assignment 1 240 4 16 4 120 7680
• assignment 2 240 4 
• assignment 3 240 4 
• assignment 4 240 4 
• assignment 5 240 4 
• assignment 6 240 4 
• CURE 2700 45 
• study for final 600 10 

Total 6700 111.6667 
% homework time utilized 87.2% 
hours homework left unused 16.3

Relevant tasks are practiced before they are used for the larger study. For instance, students practice with simulated data sets to learn statistical analysis with Excel, and they practice locating literature online and saving this to the reference library in Zotero.

Students can ask for feedback from other students during the class lectures, and use this feedback to modify their research designs.

Within each semester, students have multiple opportunities to improve upon use of AI using iterative approaches. For example, the instructor uses prompts for AI to generate some class examples during lectures, and students can choose between traditional problem/solution exercises or creating novel solutions with AI prompts during labs. Students also have the choice between one-step prompts designed to be fully correct with one iteration, or to use an incremental approach in conversation fashion.

Since AI goes through rapid development, repetition of the studies will still produce new results.

At the end of the semester, students also reflect on future directions of the project, which is beneficial for the next cohort of students in the course.

Using CURE Data

Students' data are saved in the form of written reports. Students can also present their research findings at the Oklahoma Research Day or the university's Undergraduate Research Day each year using PowerPoint presentations and posters.

Reports, presentations, and posters will be posted on the course Blackboard site in future courses. In the instructions for the paper, the name(s) of the authors are left open for purposes of anonymous review, and this also serves to keep the materials anonymous. Students are given the option to have their papers stored in a new shared Zotero repository for future students in the class.

The Research Resources section has a link to the starter ChatGPT library in Zotero. This repository is updated by the instructor in the class, but students are encouraged to submit papers they find especially helpful.

If students are interested in continuing this research leading to presentation at a professional conference, we can set up a special topics course for one or a few students. If the paper is accepted, students get a one letter grade upgrade in any of my courses. If conference attendance is in person, we try to find funding. The conference also has a virtual component if students do not wish to travel.

Resources

Due to the rapid developments in AI, relevant literature is a moving target. The current selection is available to anyone at the Zotero repository link. In the future, students in the class will also have access to some (anonymized) papers written by students in previous cohorts. The link to this Zotero repository is available to current students only on the Blackboard course site. Finally, general research resources are available in all courses with a CURE.