← Ayaan M. Kazerouni

CSC 513 Computing Education Research and Practice

As taught in Winter 2023. If you’re a current student in this course, please refer to our course Canvas, not to this page.

Contents

Acknowledgement

The course structure was heavily inspired by conversations with Zoë Wood and by Mark Guzdial’s blog post detailing a similar course.

Syllabus

Welcome!

Course: CSC 513 Computing Education Research and Practice

Overview: An overview of the current landscape of computing education research and practice, covering key findings about how people learn computing, diversity and equity in CS, and a discussion of how and why computing goes beyond computer science.

Pre-requisites: Graduate standing or consent of instructor.

Course info

Motivation: A basic understanding of computing has been called a “21st Century Literacy”, allowing a person to better engage with and reason about the digital environments in which much of our personal and professional lives take place. As computer scientists, not only are we are uniquely situated to thrive in this digital era, we are also in a good position to help democratise this potential. To do that, there is a need for computing educators and computing education researchers to push forward the science of computing education at all levels. This course aims to present students with the landscape of What We Know about how people learn computing, to train them to critically examine research in the area, and to propose new work that expands our knowledge of the subject.

Learning goals

After this course, you will be able to:

  • Discuss the current landscape of computing education research: what we know, what we don’t know, and grand challenges
  • Interpret research findings within the context of learning theories in engineering education and computing education
  • Discuss evidence-based teaching practices like peer instruction, subgoal-labelling, and pair programming
  • Understand causes, effects, and considerations for underrepresented and overrepresented demographics in computing education
  • Come up with your own answer to the question: Should everyone learn computing? Why or why not?
  • Identify an open question in computing education research; design and propose a study to answer the question

In pursuit of the goals above, this course will involve a fair amount of:

  • Reading papers, book chapters, and articles about various related topics
  • Critical discussion of the assigned readings, often guided by prompts or activities in class
  • Writing summaries and critiques of the assigned readings, and a research study proposal
  • Presenting your chosen research questions as a “lightning talk” introducing the question and soliciting feedback

Grading and assessments

Grades will be distributed across the following items.

  • Reading reflections (30%)
    • Short reflections on our readings, typically once a week, sometimes twice.
  • Paper sleuthing assignments (20%)
    • Read a from the perspective of a given role (see below for details). Submit a write-up based on what you find (1–2 pages).
    • 2 over the course of the quarter.
  • Teaching activity + reflection (10%)
    • You will each participate in a teaching activity, where you’ll get a chance to apply what you’ve learned this quarter.
    • Details will come later, but from a high level: you will teach a lesson in CSC 203 (in groups).
  • Research proposal (30%)
    • 5% — Lightning talk (3–5 minutes + discussion)
    • 10% — Research question white paper (1–2 pages)
    • 15% — Research proposal write-up (6–10 pages, due in the last week)
  • Participation (10%)
    • This is a discussion-based class, so participation is an especially important component. This includes participation in class and on the discussion boards.

Details about these assessments are below.

Reading reflections

Each week will involve “readings”. These will most often be research articles, but sometimes they may be blog posts or videos, etc. To “seed” the discussion, these will often be accompanied by reflection prompts. You can find these under “Reading reflections” on the assignments page. Please plan to spend some time on these reflections—some weeks may take more time than others because the prompts involve tasks in addition to a short written response.

Most importantly, read (or watch, or listen to) these artefacts critically, connect them with what we’ve learned from this class and from your own experience (after all, you’ve been CS students for years), and come to class ready to participate in the discussion.

  • Frequency: about 1 per week, but some weeks have 2
  • Responses need not be very long, but they should be substantive. You can say a lot in 250-300 words.

List of reflection prompts

Paper sleuthing

Read and analyse a paper with a specific role in mind. Write up your analysis in 1–2 pages (see further below for formatting instructions). There are two possible roles (below); you should make 1 submission per role, and each submission should cover a different paper or topic. The roles are described below and each has some specific instructions.

You may choose any papers from our reading list for these submissions, or papers that are not on our reading list provided they are related to computing education research (see more information below). There’s no ordering requirement on roles or papers chosen. Note that the first one of these is due during Week 4—you are not required to stick to papers or topics from before that date. Feel free to read ahead, or pick a topic of your choosing (related to computing education).

Detailed guidelines and examples for each of these roles are available.

Roles

  • The archaeologist determines where the chosen paper sits in the context of previous and subsequent work. Find and report on at least one older paper cited by the current paper, and at least one newer paper citing the current paper. Draw a clear research connection between the past paper, the current paper, and the subsequent paper.
  • The faculty member is assessing the plausibility of applying the current paper’s findings or methods in our department’s CS courses. This should be written with the rest of the faculty (or the department chair or college Dean) as the imagined audience. This submission is trying to answer the questions: Can and should we do this in our courses? Which course(s)? What would it take to do it? If you are not familiar with Cal Poly’s undergraduate CS courses, you can write in broad terms like “CS 1”, “introductory courses”, or “data structures”.

At the top of each submission, please clearly include your name and the role under which you’re writing. The submission should include a reference for the paper you’re writing about.

  • Frequency: 2
  • First is due Feb 1st
  • Second is due March 8

Research study proposal

Identify a novel research question and design a research study to answer that question. The final proposal should contain the following elements:

  • Describe why the question is answerable and ought to be answered.
  • Concretely describe an experiment to help answer the question. This means actually designing and/or prototyping the research instruments or tools you might need. Designing a survey? Include the survey questions in your submission. Designing educational technology? Include initial prototypes, lo-fi or hi-fi if time permits.
  • Discuss implications of the study. How would our understanding or our teaching methods change based on the results of the study?

Research studies can take the following forms:

  • A testable hypothesis and an experiment to test it: e.g., playing death metal in the hallways improves learning outcomes in CS courses. The hypothesis should be well grounded—is there a theory of cognition/motivation that suggests that death metal helps learning? Or perhaps there is evidence that rock ‘n roll helps learning, and we would like to see if this transfers to death metal.
  • An experiment that tests a theory: e.g., Spatial encoding strategy (SpES) theory from Lauren Margulieux attempts to explain the relationship between spatial reasoning skills and STEM achievement; you could design an experiment to help support or refute SpES
  • A one- to two-week module to teach a topic of your choosing; your submission would include a description of the design process as well as a proposed experiment to evaluate the module
  • An educational tool or modification to an existing tool, e.g., a pedagogic IDE, a plugin to make Eclipse or IntelliJ easier on novices, a dashboard for learning analytics, visualisations for CS concepts, etc.
  • A reasoned critique of a CS education tool, module, or pedagogy based on relevant theory, and suggestions for improvements
  • A systematic literature review, e.g., A systematic review of assessments of software tests—note that this is not simply a summarisation of existing research, but rather a synthesis of existing research to provide new insights and potential future research
  • A valid and reliable instrument to measure learning, self-efficacy, motivation, belonging, or other aspects of interest in computing education

Deciding on your project. You can choose your own research topic, area, and methodology within computing education. There are a couple of “milestone” submissions (see below) which you can use to solicit feedback and approval for your project topic; this is mostly to discuss scope rather than the actual topic. But you need not wait till submissions to get feedback; feel free to get in touch with me over Slack or office hours to discuss projects.

There are several deliverables to this project:

  • A 3–5 minute lightning talk about the question and its importance. Due Feb 8 or Feb 13 depending on scheduling.
  • A 1–2 page white paper about your research question—this should contain the question itself, the motivation, and theoretical basis. Due Feb 15.
  • A 6–10 page proposal describing the question and the research study designed to answer it. Due March 22.

Report formatting instructions

All written submissions will be PDFs written using the standard 2-column ACM conference paper format.

  • Reports will be written professionally, as if for an academic publication. First-person pronouns (“I”, “we”) are okay—what’s important is be to clear, concise, and precise.
  • Don’t use contractions like I have done throughout this syllabus :-)
  • Avoid making claims without (1) citing supporting evidence, or (2) making it clear that you believe something might be true.
  • When you do make a claim supported by evidence, be sure to include context. For example, student performance in a CS course can mean very different things depending on the level of the students, the environment (was it an accelerated intro to CS for AP CS takers?), the pedagogy and programming language used, etc.
  • All references (papers, books, blog posts, YouTube videos, software) should be cited in a bibliography. The sample Word doc contains toy references, and LaTeX users should use BibTeX.

For certain assignments you will need to conduct literature review on your own. E.g., if you’re looking work that’s related to one of the paper’s in our reading list, or if you’re looking for work related to a topic that’s not on our reading list (encouraged!).

If you’re looking for work related to a known paper, you can track down papers that are in the known paper’s bibliography, or you can look for subsequent work that cites the known paper. You can do this by searching for the paper in Google Scholar, and clicking on the “cited by XX”. This brings up a list of paper’s that are citing the paper. Not all of these will be relevant!

If you’re conducting a fresh search (which you will do for your research proposal at the very least), you’ll be looking for work that’s related to computing education, or at least education in general. The following conference proceedings and journals are good places to look. Note that this is a non-exhaustive list; you’re encouraged to look farther afield.

Articles within these venues should be available for free if you’re on campus or on the campus VPN. Alternatively, for many articles you can access them through the Cal Poly Library website.

Communication

  • You can communicate with me and your classmates using the Slack channel I have set up for this course.
  • Although there are assigned reading reflection discussion items in Canvas, but you should feel free to raise other interesting papers or topics of discussion in Slack.
  • You are also encouraged to seek feedback about project ideas in Slack—that way you can get feedback from me + 20 other smart and motivated people!

Accessibility and inclusivity

I know that everyone is unique, and I may have unintentionally overlooked something that limits access to some materials or activities. Please let me know if you cannot access any content. If you need additional accommodations to complete the required course work, please contact me as soon as possible! You should also contact the Disability Resource Center. Here is a link to the Cal Poly Accessibility Policy and the Accessibility within Canvas statement.

SensusAccess is a self-service, alternate media solution made available by Kennedy Library to automatically convert files into a range of alternative media including audiobooks (MP3 and DAISY), e-books (EPUB, EPUB3, and Mobi) and digital Braille. The service can also be used to convert inaccessible files such as image-only PDF files, JPG pictures, and Microsoft PowerPoint presentations into more accessible and less tricky formats. This service is available at no charge.

We expect every member of this class to contribute to an inclusive and respectful culture. Students in this class are encouraged to speak up and participate in different forms. I recognise that our individual differences can deepen our understanding of one another and the world around us, rather than divide us.

In this class, all students are strongly encouraged to share their rich array of perspectives and experiences. If you feel your differences may in some way isolate you from our classroom community or if you have a specific need, please speak with me as soon as possible so we can work together to help you become an active and engaged member of our class and community.

Course schedule

Assignments come in three flavours: Reading reflections, Paper sleuthing, and submissions related to the research study proposal.

WEEK TOPIC READING ASSIGNMENT DUE
1 Course introduction   Introduce yourself in Canvas
1 Computing for everyone Good (and Bad) Reasons to Teach All Students Computer Science (Lewis) Reading reflection #1
2 How people learn Chapters 1 & 2 from How People Learn (Bransford, Brown, Rodney, & Cocking)
Learning: From Speculation to Science
How Experts Differ From Novices		  
2 Introductory programming Chapter 2 from Learner-Centered Design of Computing Education (Guzdial)
The Challenges of Learning Programming		 Reading reflection #2
3 Notional machines Sections 1–4 from Notional Machines and Introductory Programming Education (Sorva)  
3 Contextualised computing education Chapter 4 from Learner-Centered Design of Computing Education (Guzdial)
Media Computation and Contextualised Computing Education
Computing with CORGIS: Diverse, Real-World Datasets for Introductory Computing (Bart, Whitcomb, Kafura, & Shaffer)		 Reading reflection #3
4 Programming environments What is a Pedagogic IDE? (Krishnamurthi)
Comparing Block-based and Text-based Programming in High School Computer Science Classrooms (Weintrop & Wilensky)		  
4 Meta-cognitive scaffolding
Coursenotes		 First Things First: Providing Metacognitive Scaffolding for Interpreting Problem Prompts (Prather, Pettit, Becker, Denny, Loksa, Peters, Albrecht & Masci)
Executable Examples for Programming Problem Comprehension (Wrenn & Krishnamurthi; 📽️ Talk recording)		 Reading reflection #4
Paper sleuthing #1
5 Educational data mining; AI in computing pedagogy Toward Data-driven Example Feedback for Novice Programming (Zhi, Marwan, Dong, Lytle, Price & Barnes)
The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming (Finnie-Ansley, Denny, Becker, Luxton-Reilly & Prather)
Exploring the Learnability of Program Synthesizers by Novice Programmers (Jayagopal, Lubin & Chasins)		 Reading reflection #5
5 LIGHTNING TALKS    
6 Learning a 2nd PL Understanding Conceptual Transfer for Students Learning New Programming Languages (Tshukudu & Cutts)
📽️ Talk recording		  
6 Spatial skills
Coursenotes		 Investigating the Relationship Between Spatial Skills and Computer Science (Parkinson & Cutts)
Spatial Encoding Strategy Theory: The Relationship Between Spatial Skill and STEM Achievement (Margulieux)		 Reading reflection #6
Research question white paper
7 ACADEMIC HOLIDAY    
7 Diversity & equity Diversity Barriers in K–12 Computer Science Education: Structural and Social (Wang & Moghadam)
Learning to Program: Gender Differences and Interactive Effects of Students’ Motivation, Goals, and Self-Efficacy on Performance (Lishinski, Yadav, Good, & Enbody)
Alignment of Goals and Perceptions of Computing Predicts Students’ Sense of Belonging in Computing (Lewis, Bruno, Raygoza, Wang)		  
8 Spoken language and programming languages
(topic chosen by students)		 Native Language’s Effect on Java Compiler Errors (Reestman & Dorn)
Relating Natural Language Aptitude to To Individual Differences in Learning Programming Languages (Prat, Madhyastha, Mottarella, & Kuo)		 Reading reflection #7
8 Pedagogic programming languages
(topic chosen by students)		 An empirical comparison of the accuracy rates of novices using the quorum, perl, and randomo programming languages (Stefik, Siebert, Stefik, & Slattery)
Hedy: A Gradual Language for Programming Education_ (Hermans; Also, try the Hedy language)
The Pyret programming language. Why Pyret? and Examples		 Reading reflection #8
9 CSC 203 REVIEW    
9 Misc (topics chosen by students)
Machine learning education
Software engineering education		 Learning Machine Learning with Personal Data Helps Stakeholders Ground Advocacy Arguments in Model Mechanics (Register & Ko; Abridged write-up available)
Experience Paper: A First Offering of Software Engineering (Davis, Amusuo & Bushagour)		 Paper sleuthing #2
10 NO CLASS. CSC 203 TEACHING ACTIVITIES    
10 Cybersecurity education
(topic chosen by students)		 Teaching Cybersecurity with Networked Robots (Lédeczi, MarÓti, Zare, Yett, Hutchins, Broll, Völgyesi, Smith, Darrah, Metelko, Koutsoukos, & Biswas)
Evaluation of Peer Instruction for Cybersecurity Education (Deshpande, Lee, & Ahmed)		 Reading reflection #9
Finals week     Final report due