Beyond test scores: How one educator preparation program is improving students' attendance

Attendance is one of the most important components of educational success. Higher attendance rates are associated with increases in student achievement and have been linked to improved longer-term outcomes, such as high school graduation and college enrollment. We know that educators can directly impact a variety of student outcomes, although we often focus primarily on measuring their effects on academic achievement directly. Measuring the effect that teachers have on students’ attendance—in addition to academic achievement—helps broaden our understanding of the different ways that teachers can influence positive student outcomes.

An organization that Education Analytics (EA) has partnered with since 2017, the Relay Graduate School of Education, is also interested in understanding the impacts that teachers can have on attendance rates. Relay is an accredited national non-profit institute of higher education focused on educator preparation and development. They have 19 campuses across the U.S. focused on training aspiring teachers via a two-year residency program and providing advanced certification and master’s degree programs for current teachers. In 2020, Relay wanted to explore how their programs impact student outcomes beyond achievement on state summative assessments. Specifically, our collaborative team set out to answer this research question: Do Relay teachers reduce student absences compared to non-Relay teachers?

What We Did

To answer this question, we used data provided by the New York City Department of Education from 2014 to 2019. The dataset we used included student state test scores; student yearly attendance rates; student demographics (including race, gender, English learner status, free and reduced-price lunch status, and special education status); teacher demographics; teacher years of experience; and linkages between students, teachers, and courses. We also used data provided by Relay identifying which teachers had graduated from or were currently enrolled in their program. Our dataset contained a total of 2.6 million observations spanning 5 years of data, giving us the potential for a rich analysis with large statistical power.

There are a few reasons why estimating the impact of the Relay program on students’ attendance is not as simple as it might sound. First, it’s difficult to isolate the effects of a program like Relay when so many environmental and social factors outside of a teacher’s control can influence students’ attendance. If Relay teachers were assigned to classrooms across New York City public schools randomly, then these factors would be easy to account for statistically. However, teachers are not randomly assigned—in fact, we found that, compared to non-Relay teachers in New York City, Relay teachers tend to teach more students of color (specifically Black and Latinx students), more students from economically disadvantaged backgrounds, more students with disabilities, and more students who are English learners. Our analysis shows that these students tend to have significantly higher absence rates. As a result, directly comparing attendance rates of students who are taught by Relay teachers and those who are not taught by Relay teachers would be misleading—and it could actually underestimate the impact that Relay teachers have on attendance outcomes.

A second challenge is that there are school- and grade-level factors, such as school safety, school culture and climate, or increased independence as students progress into higher grades, which can all influence students’ attendance. Our analysis shows that the assignment of Relay teachers into different schools and grades is also not random, and so we must control for this in our statistical model at these different levels. Although this is a straightforward model for us to implement, it requires a bit more sophistication in our analytical approach, including a longer panel of data and a larger sample size, to have sufficient statistical power.

In order to take this second issue into account in our models, we include what’s known as “fixed effects” at the student level and control for differences across schools, grades, and years. The way we set the model up allows us to estimate the impact of Relay teachers for each individual student; in other words, we can estimate how each student’s attendance behavior changes over time as the proportion of their courses that are taught by Relay teachers changes. This allows us to properly isolate the effect of Relay teachers on students’ attendance outcomes.

What We Found

Our analysis resulted in three key findings. First, as mentioned above, we found that Relay teachers teach more students of color, students with disabilities, students who are English learners, students from economically disadvantaged backgrounds and students in higher grades (i.e., Grade 6-8), and these students have more days of prior absences compared to their peers. For example, students of Relay teachers missed an average of 1.3 additional days of school and had lower test scores in the prior year, compared to students of non-Relay teachers. This is why we couldn’t simply directly compare students’ absences between those two groups.

Second, we found that Relay teachers are more effective than non-Relay teachers in improving students’ attendance. We compared students’ attendance outcomes in years where they took a higher number of courses with Relay teachers compared with their own attendance outcomes in years where they took a lower number of courses by Relay teachers. Results showed that when students were taught entirely by Relay teachers (i.e., for 100% of their courses), they were less likely to be absent in a school year by 0.2 days (0.025 standard deviations), compared to when those students who took no courses by Relay teachers (i.e., 0% of their courses).

Third, we found that Relay teachers’ effects on students’ attendance are statistically significant among students of color and students from historically marginalized backgrounds. For example, the impact of Relay teachers is 0.2 days (0.026 standard deviations) among Latinx students, but statistically not significant among White or Asian students. Results also show that Relay effects are larger among students from economically disadvantaged backgrounds. In addition, Relay teachers’ effects on absences were approximately four times larger for students who were English learners (specifically, for those students who were designated as English learners in all years of the data we have). Taken together, these results suggest that having a Relay teacher could contribute to closing achievement and opportunity gaps.

What's Next

There are many other student outcomes—beyond test scores, which are typically the easiest and most common to measure—that matter for student success. For instance, student suspension rates, GPA, and on-grade progression (i.e., on-time promotion to the next grade) are shown to be predictive of students’ long-term outcomes—including high school graduation, college enrollment, and labor earnings. Exploring whether Relay teachers also have an effect on these other student outcomes could broaden our understanding of the impact that Relay teachers can have on a range of student success factors. With access to data on even longer-term student outcomes, such as high school course-taking (e.g., enrolling in an advanced math course), high school graduation, and SAT scores, we could examine whether access to a Relay teacher might have a lasting effect on students.

Given our findings that Relay teachers in New York City are more likely to teach students of color and students from disadvantaged backgrounds—and that Relay teachers have positive impacts on these students—we also plan to examine whether first-year Relay teachers are more racially and ethnically diverse (i.e., consisting of more non-White teachers) than other novice New York City teachers. If they are more diverse, and if we find that students of color are more likely to be taught by first-year Relay teachers of color (especially teachers of the same race/ethnicity), then we can explore whether this match between Relay teachers’ and students’ backgrounds may be the cause of improved attendance (or other) outcomes for these students. We are eager to continue supporting Relay as a partner as they probe these important questions and explore the potential causes of their teachers’ success with students.