Reimagining accountability in K-12 education through an equity lens
Equity is at the heart of EA’s mission to develop rigorous, actionable analytics. We are committed to the use of data and evidence to help close opportunity gaps for the most vulnerable and historically marginalized students. Aligned with this goal, EA was selected in October 2022 as one of four research teams to participate in the K12 Research for Equity Hub (The Hub) to explore how accountability systems can better promote equity in schools and districts. The Hub is a multi-year initiative intended to advance national policies on accountability and assessment in K-12 education, and it is managed by EduDream, a Latina-founded, women-owned research and consulting firm.
Traditional systems of accountability often have been deficit-oriented, have overly relied on test scores, and have reflected outcomes that are too late in a student’s education experience to meaningfully improve. Yet when designed well, accountability can be a tool for equity—by highlighting where inequities are occurring and informing actionable strategies to combat them. Because persistent inequities in education can create barriers that hold some students back while others succeed, we at EA have worked to identify leading indicators of student readiness and of being on-track to graduate and pursue postsecondary opportunities, with a focus on the supports needed to realize these opportunities. Through The Hub, we are building on this work by bringing together what we know about predictive analytics and early warning systems with accountability systems—and needed modifications of these measurement approaches—to transform accountability systems from threats to tools for improvement.
Predictive analytics are often integrated into early warning systems to identify which leading indicators are most indicative of attaining later outcomes. Early warning systems in turn are intended to highlight these leading indicators early on, so that educators can target supports to help students achieve readiness and success. Accountability systems, on the other hand, serve as a retrospective report card to evaluate schools against a set of outcomes at the end of the year.
EA seeks to import the best of early warning systems into school accountability and to broaden the set of student and school outcome measures included to create a richer, more equitable, and more actionable accountability system. Including predictive analytics and early warning measures in accountability systems could identify opportunity gaps and root causes of inequities early enough for educators (and families) to take action, so that more distal outcome gaps in high school graduation rates and even post-secondary well-being can be addressed while schools, students, and caregivers still have a chance to make real change. An improved and equity-focused accountability system could act less like a verdict on student and school performance and more like an actionable roadmap to improve students’ life outcomes.
Combining early warning indicators with accountability systems
It’s not a simple task to align early warning and accountability systems while ensuring equity is embedded throughout. In partnership with two large urban school districts, EA is working to expand upon these systems to include a broader set of student and school experiences and outcomes, such as participation in career and technical education, post-secondary outcomes, and social emotional learning and other non-cognitive outcomes. Many standard measures used in early warning systems can be brought into the accountability framework, but with some tweaks.
For example, course grades are often used as indicators of whether students are on-track or ready to succeed in college or the workforce. A student who receives As, for example, is likely more ready than a student who receives Ds in relative terms. Yet, if course grades are embedded in a system of evaluating school quality, schools may be incentivized to artificially inflate grades. One way to use this valuable metric for accountability while not incentivizing grade inflation is to make school-level adjustments, so that students are “compared” against their own school’s average (rather than schools being compared across all other schools in the district). In this way, grades are always relative within a school, and thus it would not benefit a school’s standing in the accountability system to increase grades within that school. This does, however, present another challenge—we’ve now removed the ability to evaluate differences between schools using course grades. This highlights the need to have a broad set of metrics within the accountability system that can feature differences both within individual schools and across schools.
Other metrics that can be borrowed from early warning systems and redesigned for an actionable accountability system include indicators like course levels (used to indicate the extent to which students are taking challenging courses), attendance, or chronic absenteeism. Taking the example of challenging course-taking, in a reimagined framework, this measure could be used to go beyond a simple accountability measure telling us what already happened, but rather could allow us to uncover some of the mechanisms for change and to track the impact of school change interventions. If a new accountability measure for a school indicates, for example, that Black students are enrolled at lower rates in Algebra 1 in 8th grade compared to other students, additional indicators could assess whether the school increases the number of sections of Algebra 1, hires math teachers to increase teaching capacity in this subject, or takes steps to ensure that Black students are not counseled out of enrollment in Algebra 1.
Some of the work to develop an equity-focused accountability system builds upon more traditional measures like test scores and growth metrics, but with an equity focus, such as highlighting comparisons of certain student subgroups within a school against city-wide averages or with students in neighboring schools. EA is also currently deep in the research phase of developing a set of new equity indicators, including measures of school racial/ethnic diversity, disparity in student transfer rates between schools, and student, parent, and teacher responses to equity-related survey questions as well as related questions on school culture and climate.
Critical to the goal of redesigning an accountability system to better promote equity among historically marginalized communities is involving these communities in the process. EA is partnering with the Center for Equity for English Learners (CEEL) at Loyola Marymount University to gather input from parents and educators with the goal of having community engagement contribute to more equitable and responsive measures that reflect the values of these communicates for their students and children. Such a system could be used by parents and educators to inform their decisions at all points in students’ high school careers.
Things to keep in mind when embarking on this work
As exciting as this work is, it also comes with challenges and much to think through. As EA moves forward with this research, we seek to share our learnings with others who may also be on this important journey.
Consider how to modify measures to avoid unintended consequences. When incorporating readiness measures into accountability systems, it's crucial to ensure they provide the right incentives to schools to genuinely improve readiness and school quality, and that they do not provide incentives to “game” the system. As discussed above, an accountability system that includes useful predictors like course enrollment or grades could be gamed, for example, by delaying student enrollment in challenging courses or by inflating course grades. Alternatively, schools may take the approach of enrolling all students in challenging courses, such as 8th grade Algebra, which could have a negative impact on equity (e.g., if students who are not ready for such courses are enrolled without being adequately prepared).
Find ways to balance important priorities that are in tension. For decades, researchers and practitioners have worked to develop measurement systems that (1) identify students who are ready or on-track to graduate from high school or enroll in postsecondary institutions, (2) monitor progress and improvement toward getting ready, and (3) provide students the opportunities and support they need. These efforts provide a rich history to call upon that may be familiar to educators and families—but there is a clear need for improvement and innovation with these historical approaches, particularly as it relates to highlighting areas of inequity and driving change to support historically marginalized students. Accountability systems are also deeply entrenched in complex local, state, and federal political systems. It is important to consider ways to balance innovation with what stakeholders are familiar with, as well as balance the urgency to make needed change with the time and intentionality to manage this change well. Consider how we can build upon the past lessons learned to keep or improve upon what worked well, while adding innovative and important new measures that serve students and educators.
This work can’t be done in a vacuum. Critical to this approach of building an alternative accountability model that is strengths-based and equity-focused is gaining input from and authentically engaging families, educators, and the community throughout the process. Involving community stakeholders in developing the system—such as through townhall meetings, focus groups, and surveys—is key to building a system that is responsive to and reflects the values and priorities of the communities served. When embarking on this work, build community engagement into all steps of the plan and consider what it means to develop a locally-driven system while balancing the need for comparisons across contexts.
Consider far-reaching implications of the accountability system and potential misuses of such a system. Beyond considerations of how accountability measures—if not implemented well—could be interpreted to incentivize certain unwanted behaviors like grade inflation, it is also important to consider implications for how the system interacts with broader societal systems, governance questions, and local context. For example, could behavioral metrics be used to unfairly target certain students, rather than provide needed supports? When planning for this work, consider implications for highlighting certain inequities and the difficult but necessary conversations and decisions that are needed based on findings from such a system. If these systems are truly tools for equity, they must highlight where inequities exist and provide actionable steps to address them.
Consider including a broader set of outcomes, including workforce outcomes. By including predictors of workforce outcomes in addition to college outcomes, such a system would also serve often-overlooked students who could succeed in making an economically viable career through a variety of different paths. Expanding beyond the more easy-to-measure outcomes of high school graduation or college enrollment to outcomes such as college graduation is another important consideration. Data availability can present a challenge when looking to expand upon these outcomes, which speaks to the dire need for investment in statewide longitudinal data systems that leverage interoperable technologies.
What’s next?
As tempting as it is to try to do everything at once given the urgency and importance of this work, it’s simply not possible. We are attempting to start just big enough to be expansive and impactful, but modest enough to be manageable and implementable. Given the need to prioritize time and funds, EA and our partners at the Center for Equity for English Learners have begun our community engagement efforts focused on parents and educators. Future efforts would aim to expand this engagement to students and to other communities, as well as to other methods of gathering feedback from hard-to-access communities.
As research efforts continue to evolve, two interesting issues have emerged that EA is currently working through:
- Using data to determine how to assign weights for measures when they may conflict within a particular school. For example, if performance gaps between Black students and White students is large, but the gap between English learners and native English speakers is small, how should those results be weighted in such a system? In other words, we’re working through how to handle cases of conflicting metrics, such as when a school is strong on a metric for one group of historically marginalized students but not another. Engaging with our district partners and other colleagues is critical as we consider both the results from rigorous analysis and policy implications.
- Balancing predictive power with simplicity and transparency when considering indicators for inclusion in an accountability system. We are in the process of thinking through how to “roll up” certain indicators into one metric for ease of interpretation, while still maintaining a high level of predictive power. Similarly, we are developing a single composite post-high-school outcome rather than predicting separate outcomes (e.g., college enrollment, college grades, and enrollment in remediation courses).
Based upon these experiences of conducting rigorous analyses and deep partner and community engagement, we would like to develop a set of best practices to support the field in replicating efforts to build an equity-focused accountability system that is authentic and responsive to the schools and communities where the measures are being implemented.