OBF EQUITY TOOLKIT: LEVERAGING OUTCOMES-BASED FUNDING TO SUPPORT EQUITY
[Toolkit Series 2] OBF POLICY DEVELOPMENT & REDESIGN: PRIORITIZING EQUITY
Equity Metrics: How States Construct OBF Formulas to Improve Outcomes for Underserved Students
THE OBF EQUITY TOOLKIT provides practical lessons on how states, systems, and institutions work to address equity in the development and implementation of OBF policy. Broken into four Series focused on equity challenges in distinct phases of the OBF policy process, the Toolkit contains short, individual modules that focus on specific topics and provide lessons learned and recommendations for policymakers and institutional leaders to consider. Content is derived from in-depth study of six states (Tennessee, Indiana, Ohio, New Mexico, Oregon, and Kentucky) and 13 institutions in them. See the Research Methods section of the Overview for more information.
This research was conducted in coordination with the Center for Law and Social Policy (CLASP) and supported by a grant from the Bill & Melinda Gates Foundation. Opinions reflect those of the authors, not necessarily those of the Foundation.
THIS MODULE addresses equity in OBF at the state and student levels as outlined in our OBF Equity Overview. It will help state policymakers and institutional leaders consider:
Variations in how states construct OBF metrics; and
The degree to which institutional leaders perceive OBF metrics to be meaningful and effective.
Determining how best to utilize equity metrics in OBF
OBF equity metrics are measures that allocate additional funds for the success of underserved student populations. State policymakers, researchers, and advocates describe equity metrics as serving one or more of the following purposes:
To foster the success of underrepresented or underserved students and meet state completion, attainment, and workforce goals;
To disincentivize institutions from increasing their selectivity or reducing access as a means of bolstering their performance;
To distribute funding to institutions serving underrepresented or underserved students, in recognition that such student groups may need more resources to succeed; and
Equity metrics are increasingly common in OBF policies. According to HCM Strategists,[iii] of the 25 states that implemented outcomes-based funding during fiscal year 2018, 22 included equity metrics. RFA’s research focused on six of these states.
LESSONS FROM THE FIELD
States vary in their selection of equity metrics, and institutions vary in how they perceive their utility and purpose. This section presents an overview of these variations.
Variation in equity metrics
While OBF policies reward the retention, progress, and completion of all students at an institution, OBF equity metrics provide additional dollars when students in targeted, historically underserved populations succeed. Equity metrics can vary in three important ways: the populations prioritized, the approaches used to allocate funding, and the types of prioritized outcomes. Each factor is discussed below.
The process for selecting and defining priority populations is complex and varies significantly across states. Differences in definitions, demographics, state college attainment and workforce goals, the availability of data, and the policy development process itself can contribute to these variations.
Our six study states include a range of priority populations, including low-income, academically underprepared, first-generation, or rural students, adults, veterans, and students of color. For more information on how states prioritize and define underserved student populations, and to learn about factors to consider when selecting priority populations, please see Module 2.3: Selecting Student Groups to Prioritize in OBF Policies.
Approaches to allocating funding through equity metrics: direct vs. bonus
The Center for Law and Social Policy (CLASP) determined that states use two approaches to crafting equity metrics:
Direct equity metrics are embedded in the formula as stand-alone metrics that reward specific outcomes for target, or priority, student groups. For example, Indiana awards 15% of its OBF formula funds to degree completions of low-income students.
Bonus equity metrics act as a premium, or bonus, on a direct outcome measure for a certain population. For example, in Tennessee, postsecondary institutions receive a financial award for all degree completions. In addition, when a qualifying member of a priority population succeeds, the institution receives an 80% bonus. The bonus increases to 100% for students who qualify for two populations and, for community college students only, 120% for three priority populations.
Type of outcomes prioritized: completion vs. milestones
Equity metrics also vary in whether they reward interim milestones, such as credit accumulation, or long-term outcomes, such as degree completion.
Completion outcomes vary by state but can include certificates and degrees captured in the state’s broader OBF formula.
Milestone outcomes also vary by state and include a much wider range of progress-to-degree outcomes, including credit and course completions as well as progression benchmarks and transfers.
Table 1 shows the student groups prioritized, the approaches to weighting equity metrics (direct or bonus), and the type of outcomes (completion or milestone) awarded across all six study states.
Table 1. Equity metric for each state and sector defined by priority population, metric type, and outcome awarded
Table 1 shows:
Prioritized populations vary across states, but all six states include low-income students.
States utilize direct and bonus metrics at about the same frequency. New Mexico, Indiana, and Kentucky utilized direct measures, while the other three states applied bonuses for all or most of their measures.
Equity metrics are applied to milestone outcomes for two-year institutions more commonly than four-year institutions. Of the six study states, only Ohio and Tennessee applied equity metrics to milestone outcomes in the four-year sector.
Within states, equity metrics vary depending on the focus population. For example, Indiana awards milestone outcomes achieved by academically underprepared students in community college, but awards completions achieved by low-income students in both two- and four-year institutions.
The implications of these distinctions vary across states and sectors and are discussed below.
Are equity metrics working?
Research to date provides no definitive answer on how equity metrics impact institutional or student outcomes. Variations in how states with OBF define equity (i.e. selection of priority populations) and incorporate equity metrics into their policies (e.g., bonus v. direct, completion v. milestone), as well as differences in the relative value of the metrics themselves and their longevity present formidable challenges to measuring effects that might be considered generalizable or precise. Thus, the field will need to wait for an answer until the above challenges are addressed with more nuanced datasets and additional years of implementation. In the short term, the research presented in this module provides insight into whether and how institutional leaders perceive equity metrics to be working to close equity gaps.
Perceptions of the effectiveness of OBF equity metrics
Institutional leaders assessed the effectiveness of equity metrics in a variety of ways:
Leaders of open-access institutions believe equity metrics provide long-overdue recognition of the added costs of serving historically disadvantaged students. One leader said, “To me, it [equity metrics] is a way of compensating institutions for graduating those students who maybe require more resources or more assistance.” This sentiment was echoed among open-access and community colleges, with institutional leaders describing equity metrics as a reflection of their student body. One leader said: “I think it's rewarding us for the work we're already doing. We're just trying to serve our communities and our communities happen to be…underprepared and low income.”
Institutional leaders widely agree that equity metrics play a role in communicating a state’s equity goals. Some college and university leaders perceive them to be intrinsically valuable as a signal of the importance of closing attainment gaps, regardless of the actual dollars attached. One institutional leader explained:
It really strengthens the message that it's not just bringing these [underrepresented and underserved] students and teaching them. It's to make sure they are successful. And, so, having that success as part of the formula would give a stronger message to faculty, to administrators, to staff, so we can be more creative with what it would take for them to be successful.
Yet there was strong consensus that the effectiveness of equity metrics would increase if they were attached to more resources. Despite their symbolic importance, institutional leaders do not generally believe equity metrics produce enough additional revenue to fully incentivize comprehensive reforms. This quotation from an administrator at an open-access university was typical: “I've seen the formula and our [n-sizes] are so small… It's pennies. I mean, it’s nice—at least there's a little bit. But more would be better.”
Policymakers in Tennessee agreed. Under Tennessee’s first iteration of the OBF model, institutions were awarded a bonus (40% premium) on milestone and completion outcomes achieved by adult and low-income students. However, institutions and policymakers did not feel that the bonus was sufficient. One policymaker said, “There’s a general idea, over the five years of the first implementation, that 40% was not nearly enough. It did not reflect the cost that it takes to get a low-income student across the finish line.”
As a result, Tennessee doubled the bonus amount awarded to outcomes for priority populations, from 40% in the old model to 80% in the 2015-20 model. This change was based on feedback from campus leaders who “believed a higher premium level would be more appropriate.”[iv] In addition, Tennessee’s OBF model was also refined to employ elevated bonuses, or graduated premiums, to address the effectiveness of equity metrics. In 2015, a third priority population for community colleges was added, replacing the remediation success outcome with bonus funding for the success of academically underprepared students. Under the policy’s elevated bonuses, institutions are awarded an 80% bonus for outcomes completed by a student in one priority population, a 100% bonus for students in two priority populations, and, at community colleges, a 120% bonus for students in all three.
Leaders of large universities reported that weights were not high enough to incentivize an increased focus on priority populations. One administrator said, "There's got to be some more teeth to [equity metrics in the formula] if we're really going to continue to drive it." He elaborated that the amount of funding attached to his state’s equity metrics is enough for his institution “to do something with it.” However, the amount of funding driven through the metrics is “not enough” to compel larger institutions, less reliant on state dollars, to “focus on it.”
Some community college administrators also reported that equity metrics had not affected institutional practice. These institutions already enroll large numbers of historically underserved students, but many believe the metrics don’t go far enough. One administrator said, “I don't think [the equity metric] does enough. I can't think of a single institution that is really working to improve relative to that measure.” Despite this complaint, many community colleges reported significant reforms designed to improve outcomes for historically underserved students.
In addition, institutional leaders commonly evaluated equity metrics across two key elements – the approaches used to allocate funding and the types of prioritized outcomes:
Perceptions of the relative value of direct vs. bonus equity metrics
Across the six states, policymakers had no clear rationale for selecting either bonus or direct metrics. This quotation from an architect of one state’s OBF equity metrics is typical:
I think we were always trying to keep [the formula] as simple as possible. We never really took an approach where we split the outcomes out as a separate outcome to measure some sort of subpopulation. We just never really considered that [direct] approach, for whatever reason. Trying to keep things simple.
Institutional leaders do not perceive a meaningful difference between direct and bonus equity metrics. When asked why they selected one type over another, policymakers consistently reported that they saw little difference between the two. Leaders from all 13 institutions in our study were either unaware of whether equity metrics were bonus or direct or they were inaccurate in their understanding. Chief financial officers were often the only senior administration members aware of how the equity metric was calculated.
Perceptions of the relative value of milestone vs. completion equity metrics
Leaders of open-access institutions believed that milestones metrics are more effective than completion metrics. Institutions that enroll high numbers of historically underserved students note the importance of front-loading resources. Guided pathways, intrusive advising, developmental education reform, and more accurate placement policies have all been shown to increase student outcomes. However, they also require significant financial and human capital. Equity metrics that reward institutions for early success, such as retention and credit accumulation, recognize their importance and can drive much-needed resources to institutions in a timelier manner to reward and support these efforts.
A leader from an open-access institution described the rationale for prioritizing milestones:
I think that the spirit of the equity metrics is to help institutions like us. My concern is what they are measuring with at-risk is the awards--how many students have graduated. I would rather delete the awards for at-risk students and take that money and put it in [the milestone outcome], because the milestone is the one that is actually capturing the progress that the students are making.
These findings suggest that equity metrics can have positive impacts on institutions and students. However, it matters how metrics are constructed and how they are perceived at the institutional level. While most institutional leaders view equity metrics as a helpful, important acknowledgement of their mission, others reported that the relatively modest amount of allocated funding was not enough to incentivize extensive changes to institutional practice or policy.
As states and institutions consider how best to utilize equity metrics, we offer the following recommendations:
States should allocate more money through equity metrics. Institutional leaders consistently reported that equity metrics would be more impactful if more resources were attached. There is broad consensus that current levels of equity funding do not go far enough in incentivizing institutions to align with state priorities to improve outcomes for underserved students. States should, therefore, consider increasing their investment in the funds awarded through equity metrics. While our research does not identify what level of funding would be impactful, future studies examining the change to Tennessee’s funding formula may capture institutional perceptions.
State policymakers should include equity metrics that award interim progress in both the two- and four-year sectors. Many institutions invest in early interventions to quickly identify and address the barriers that can disadvantage traditionally underserved students. As these students progress, states can use milestone equity metrics to reward institutions. All six states in our study used these metrics in their community college OBF policies. In contrast, completion dominated OBF equity metrics in the four-year sector. States should consider utilizing equity metrics in both sectors to reward the progress of underserved students. In this way, the state would provide early resources to strengthen the pathway to a four-year degree for traditionally underserved students.
In designing OBF policies, state policymakers and institutions should consider equity metrics as an opportunity to communicate their commitment to the success of underserved students. The distinction between bonus and direct equity metrics proved to be more meaningful for researchers than for state policymakers and institutions. In contrast, the broader message sent by adopting equity metrics can be powerful. Even when the metrics do not result in significant additional resources, they can provide a foundation for strengthening programs and policies aimed at fostering the success of underrepresented or underserved students.
[i] Snyder, Martha, and Scott Boelscher. Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update. HCM Strategists (2018). Accessed August 7, 2018. http://hcmstrategists.com/wp-content/uploads/2018/03/HCM_DBO_Document_v3.pdf
[ii] Cielinski, Anna, and Duy Pham. Equity Measures in State Outcomes-Based Funding: Incentives for Public Colleges to Support Low-Income and Underprepared Students. Center for Law and Social Policy (2017). Accessed August 7, 2018. https://www.clasp.org/sites/default/files/ public/resources-and-publications/publication-1/Equity-Measures-in-State-Outcomes-Based-Funding.pdf
[iii] Snyder, Martha, and Scott Boelscher. Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update. HCM Strategists (2018). Accessed August 7, 2018. http://hcmstrategists.com/wp-content/uploads/2018/03/HCM_DBO_Document_v3.pdf
[iv] Tennessee Higher Education Commission. “Outcomes-Based Funding Formula Overview.” Accessed August 7, 2018. https://www.tn.gov/content/dam/tn/thec/bureau/fiscal_admin/fiscal_pol/obff/1_-_Outcomes_Based_Funding_Formula_Overview_-_One_Page.pdf