Peer Networks and Instructor Resources as Complementary Support Systems in Six CS Classrooms

This is an overview of the research paper Peer Networks and Instructor Resources as Complementary Support Systems in Six CS Classrooms. The work was led by former master’s student Noah Ravetch and undergraduate students Naomi Nayman and Jack Herberger at Cal Poly San Luis Obispo, and co-advised by my colleague Theresa Migler.

• Background
• Context
• Data
• Who participated in the peer networks?
• What was the relationship with academic performance?
• How did use of instructor resources factor in?

We studied the interplay between peer collaboration networks in six CS classrooms, usage of instructor-provided help resources, and performance on summative assessments. Students who were unconnected to their peers scored relatively lower on summative assessments. Students who were connected to their peers or used instructor resources performed dramatically better than those who used neither.

Background

A classroom is a social system at work—students are participating in a network consisting of their peers and the instructor. Having strong connections in this network is a form of social capital: it can facilitate advice about courses, policies, and study materials, and can lead to downstream improvements to academic persistence and performance. However, some students may lack this form of capital for a number of reasons, e.g., fear of breaching academic integrity policies, being new to the institution, or factors that prevent the student from interacting with their instructor (e.g., anxiety or external obligations preventing one from visiting office hours).

Evidence also mounts that general availability of large language model-powered chatbots like ChatGPT are contributing to a general erosion of the social fabric of classrooms. To add to that, I offer two bits of anecdata: first, I and my colleagues have experienced a steep dropoff in students attending office hours—no surprise, since it’s far more convenient to ask ChatGPT a question than to visit office hours on campus. Second, I advise the computer science peer tutoring center in our department at Cal Poly, where we’ve experienced a nearly 50% decrease in visits since LLM-based tools became mainstream.

In this study, we investigated whether use of social help resources—such as collaborating with peers or seeking help from the instructor—was associated with better academic performance.

Context

The study took place at Cal Poly San Luis Obispo. Some salient info about our context:

• We have small-to-medium class sizes (in this study, 29–35 students in each class). A large lecture-hall-style class might observe markedly different peer network dynamics than we did.
• The courses that we studied have a prominent lab component that is led by the instructor, and is given as much time during the week as lectures. This results in a significant amount of time available for students to work on course activities in proximity to their peers and to the instructor. It is in these lab sessions that we expected peer networks to develop.

We studied four courses (six classrooms): an introductory programming course taken mostly by Graphic Communication majors (referred to as Intro), an intro programming course taken by Engineering majors (Eng), three sections of an object-oriented programming course (OOP), and an upper-division software course (SE). Noah taught Eng and I taught the rest.

Data

We studied three variables:

• Performance on a summative assessment: In Intro, OOP, and Eng, this was measured as a 0–100 score on a cumulative final exam. In SE, this was measured as the average performance on programming assignments.
• Use of office hours or the online forum.: In the OOP sections, I kept track of office hour visits using a simple online form. Only the frequency of visits was tracked, not the length or purpose. For online forum usage, we downloaded analytics data from the course EdStem platform.
• Participation in the classroom peer network: We used self-reported survey data to learn about peer collaborations.

The network survey was distributed to students toward the end of the term in which data collection took place. The survey asked for informed consent and demographic information, and then prompted the student to list any classmates with whom they had worked that term. For each collaboration they were asked to choose one or more of the following options:

• I helped them
• They helped me
• We studied together
• We collaborated on an assignment
• We discussed course logistics
• Other (free response; no students picked this option)

The vast majority of students (over 95%) picked at least one of the bolded items above, indicating a bidirectional flow of help. We used these responses to construct for each classroom an undirected disjoint graph in which edges indicated any collaborative relationship between two students.

Since students were reporting on their collaborations, we enacted the following protections for surveyed students.

• It was made clear that questions were assumed to be about coursework on which collaboration was explicitly allowed.
• Raw survey responses were not accessible to course instructors before anonymisation.
• If a survey taker (i.e., a consenting student) listed a non-consenting student as a collaborator, we included that edge in the network, but removed all information about the non-consenting student from the dataset (i.e., their grade and office hours and forum usage). We did not exclude the edge entirely because this might falsely label the consenting student as “isolated” if it was their only connection.
• If multiple survey takers listed the same non-consenting student as a collaborator, those survey takers received edges to two separate anonymous nodes, so no information about the non-consenting student would remain in our dataset. In effect, all that remained in these cases was the fact that the consenting students “had a friend” in the classroom.

The peer networks so constructed are depicted below.

Who participated in the peer networks?

Our first focus was on students who were isolated in the peer network, i.e., those who did not list any collaborators and none listed them. About a quarter of the students were isolated, and men were more likely than expected to be isolated. They were ~60% of the surveyed students but just under 80% of isolated students.

What was the relationship with academic performance?

We used hypothesis tests to check for a relationship between a students’ connectedness in the network and their performance on the course’s summative assessment.

How did use of instructor resources factor in?

In the three OOP classes, we collected data about office hour visits and use of the online EdStem forum. The distributions were remarkably similar:

• 58% of students did not visit office hours, and 55% of students did not post on EdStem.
• 19% of students visited office hours once, and 17% of students posted once on the EdStem.
• Only a handful of students visited office hours or posted on the forum more than once.

Hypothesis testing suggested a significant relationship between likelihood of posting on the forum and visiting office hours: students were likely to use both resources or neither.

Finally, we turn to the interplay between use of instructor resources, participation in the peer network, and summative assessment performance. We looked at four groups of students: those who had peer collaborators, those who used instructor resources, those who used both, and those who used neither.

Given our results, we recommend that instructors take steps to broker social connections in the classroom, e.g., through in-class activities or collaborative assignments. Because peer connections seemed associated with comparable score improvements (to say nothing of other benefits of peer connections that we didn’t study), and because office hours impose real costs on students with external obligations, we place more emphasis on building in-class connection than on driving up office hours attendance.

For more details and discussion, see the paper.