Past and Present Research Efforts

Current Projects

Evaluating the Impact of MiSTEM Regional Allocation Grants on Student Outcomes: A Longitudinal Study

Folks Involved: D. Kinsman (lead), D. Caballero

This study investigates postsecondary STEM readiness - a collection of markers traditionally used to determine the preparedness of students for higher education in STEM. Through complementary modeling analyses that offer comparative and regional lenses, we will be able to unpack the state of Michigan’s progress towards greater and deeper college readiness for all students. The first project applies regression-corrected streamline modeling to longitudinal standardized assessment data to characterize regional variation in student trajectories and identify patterns associated with postsecondary STEM readiness. The second project uses machine learning models to estimate the relative importance of demographic, academic, and contextual factors in predicting postsecondary STEM readiness and STEM credential attainment. Findings will inform future investment strategies and provide empirical evidence to support data-informed decision-making by MiSTEM leadership and state education policymakers. This work is supported by the Lappan-Phillips Foundation and the CREATE for STEM Institute.

Developing a Survey of Student Epistemologies and Expectations About Computing in Physics Courses

Folks involved: A. Reynolds (co-lead), R. Roca (co-lead), D. Caballero

Over the past two decades, computing has become increasingly integrated into undergraduate physics courses, from introductory labs to upper-level theory courses, and from courses for physics majors to general education courses for non-majors. Instructors want to know whether these integrations are working, but the wide variety of learning goals, coding languages, and course contexts means no single concept inventory could assess the specific skills taught in every course. Instead, we are developing a survey of students’ epistemologies and expectations about computing within their own course context, modeled on the E-CLASS (Zwickl et al. 2014), which has successfully assessed students’ views across the diverse landscape of physics lab courses. To design an instrument flexible enough for this landscape, we first need to characterize it. We are interviewing instructors from a wide range of computationally integrated physics courses, including those at two-year colleges and Minority Serving Institutions, about their motivations for integrating computing, their course contexts, and the computational skills they emphasize. We analyze these interviews using physics computational literacy (Odden & Caballero 2023) alongside frameworks for physics identity development (Hazari et al. 2010; Irving & Sayre 2016). Our early case studies show that instructor motivations range from building disciplinary problem-solving skills to engaging students in authentic research practice, and that the material, cognitive, and social dimensions of computing look quite different depending on course level, institution type, and instructional goals. Future work will pair these instructor interviews with student interviews from the same courses to examine how instructors’ goals shape students’ beliefs and expectations, grounding the development of the survey instrument. This work is supported by the National Science Foundation (DUE-2337055).

Characterizing Computing Practices in Graduate Astronomy Research

Folks involved: S. Chitchyan (lead), D. Caballero

Computing is central to modern astronomy, from simulating the earliest galaxies to reducing and analyzing telescope observations, and the field has a strong open-source software culture built around community tools like Astropy and yt. Yet there is little astronomy education research on how professional astronomers actually engage in computing, which makes it difficult to prepare students for this work. In this project, we interview astronomy graduate students in both observational and computational astrophysics about the workflows of their research projects, using the Computational Thinking in Mathematics and Science taxonomy (Weintrop et al. 2016) as a starting point to identify which computing practices they use and where. Our preliminary findings suggest that the Weintrop taxonomy captures many of these practices, such as programming, troubleshooting and debugging, choosing effective computational tools, and developing modular solutions, but misses an important social dimension of computing in astronomy: learning tools through mentorship, debugging with peers, working within community-developed codebases, and communicating the computational decisions behind scientific results. Building on this work, we aim to develop a framework for computing practices specific to astronomy research, drawing on additional perspectives such as physics computational literacy (Odden & Caballero 2023), to inform how computing is integrated into astronomy education. This work is supported by the National Science Foundation (DUE-2337055).

Leveraging Institutional Data and Knowledge to Assess the Impact of STEM Course Transformation on Student Outcomes

Folks Involved: C. Lem (lead), H. Kim, D. Caballero

Over the last few decades, research in higher education has discovered that a concerning percentage of students who enter college as STEM majors switch to non-STEM majors or drop out of college altogether (Seymour et al. 2019). Additional research has suggested that gateway STEM coursework is a significant factor in the high rates of attrition (PCAST 2012). In response to this, higher education institutions have attempted to “transform” these courses to result in improved outcomes for students. For instance, at MSU, there has been an effort over the last several years to transform gateway course sequences in physics, chemistry, and biology using the 3-Dimensional Learning (3DL) framework. Our work aims to use interviews with faculty, institutional data, and machine learning to assess the impact of this course transformation on retention in STEM at MSU, which will build on existing findings of how transformation has correlated with student success in these courses (Matz et al. 2017). Ultimately, this work provides an evidence-based account of how large-scale course transformations shape STEM retention and offers a model for how institutional data can be used to evaluate similar initiatives. This work is supported by the Lappan-Phillips Foundation and the CREATE for STEM Institute.

Exploring the Impact of Unsupervised Clustering Methods in Systematic Literature Reviews

Folks Involved: E. Bolger (lead), D. Kinsman, A. Kosinski, D. Caballero

About 15 years ago, education researchers conducted a systematic literature review (SLR) on change strategies for improving undergraduate STEM education instruction (Henderson, Beach, & Finkelstein, 2011). Analyzing 191 articles from 1995 to 2008, the researchers identified four broad categories of change strategies through a comprehensive interdisciplinary literature review: (1) disseminating curriculum and pedagogy, (2) developing reflective teachers, (3) enacting policy, and (4) developing a shared vision. With recent developments in STEM education practices, it is imperative to conduct a follow-up SLR comparing the effects of change strategies and associated student success. Similarly, with the influx of scientific articles published in recent decades, it is time consuming to conduct comprehensive SLRs without the assistance of machine learning (ML) analysis techniques. Working with qualitative researchers, we investigate the impact and ability of using ML to assist in the analysis pipeline through implementing Computational Grounded Theory. This work is supported by the National Science Foundation (DRL-2201795).

Students’ Approach to Problem-Solving Computational Tasks with Functions

Folks involved: R. Frisbie (lead), K. Sachithanand, E. Tobias, D. Caballero, D. Silvia

Students with a variety of prior knowledge and experience with respect to computation engage in our interdisciplinary introductory computation courses. We strive to create a curriculum that brings together students’ disciplines and experiences with computer science concepts, data analysis, and computational modeling. This project encompasses two different research avenues. (1) Qualitative interviews with students centered on computational tasks involving functions and (2) Exploration of the impact of research findings integrated into introductory curricula. The data generated consist of student interviews where they describe their thought processes aloud while engaging in tasks presented in a Jupyter notebook. In particular, I compare the characteristics of the pathways that students take to arrive at a solution to a particular task.

Self-efficacy in interdisciplinary computing

Folks involved: P. Hamerski (lead), P. Tourangeau, D. Silvia, D. Caballero

This project focuses on characterizing self-efficacy for interdisciplinary computing, from the perspectives of students in an introductory CMSE course. We have carried out data generation by conducting semi-structured research interviews. We are working on data analysis, which entails theory-driven and data-driven coding for self-efficacy, developing a codebook, and shaping findings around how students described their course experiences and how those descriptions align with self-efficacy.

Past Projects

Using Resource Theory to Understand How Students Think About Indexing

Folks involved: T. Finzell (lead), D. Silvia, D. Caballero

Students struggle with how to loop through lists in Python, often mixing and matching their strategies between looping by value and looping by index. At the heart of this issue is the concept of indexing, which can be a challenging abstraction for students. My work uses Resource Theory, a cognitive framework based on DiSessa’s Knowledge in Pieces, to identify concepts and procedures that students activate when solving problems involving indexing.

Characterizing Community Interactions Among Faculty Using Network Analysis

Folks involved: E. Bolger, M. Nwobi, D. Caballero

Using Social Network Analysis (SNA), we analyze the interactions of physics instructors over Slack involved with the Partnership for Integration of Computation into Undergraduate Physics (PICUP) – a faculty learning community. We analyze these relationships with traditional SNA metrics, such as reciprocity. However, to statistically quantify the values of these metrics in our networks, we need to create our own sample by generating random networks similar to the original. In this project, we compare the degree distributions of the random networks from various generation techniques. The choice of random generation method will allow us to make statistical claims about the nature of the interactions between the physics instructors and provide insight on how the virtual environment supports faculty in integrating computing into their classrooms. This work is supported by the Lappan-Phillips Foundation and the CREATE for STEM Institute.