OBF EQUITY TOOLKIT: LEVERAGING OUTCOMES-BASED FUNDING TO SUPPORT EQUITY
[Toolkit Series 2] OBF POLICY DEVELOPMENT & REDESIGN: PRIORITIZING EQUITY
Increasing Completions Through Quality Certificates
THE OBF EQUITY TOOLKIT provides practical lessons on how states, systems, and institutions work to address equity in the development and implementation of OBF policy. Broken into four Series focused on equity challenges in distinct phases of the OBF policy process, the Toolkit contains short, individual modules that focus on specific topics and provide lessons learned and recommendations for policymakers and institutional leaders to consider. Content is derived from in-depth study of six states (Tennessee, Indiana, Ohio, New Mexico, Oregon, and Kentucky) and 13 institutions in them. See the Research Methods section of the Overview for more information.
This research was conducted in coordination with the Center for Law and Social Policy (CLASP) and supported by a grant from the Bill & Melinda Gates Foundation. Opinions reflect those of the authors, not necessarily those of the Foundation.
THIS MODULE addresses equity in OBF at the state, institutional, and student levels as outlined in our OBF Equity Overview. It will help state policymakers and institutional leaders consider:
What is known about the quality and market value of different kinds of certificates for students;
How states have approached rewarding certificates under their OBF formulas; and
How institutions have balanced maximizing rewards with awarding meaningful certificates.
Responding to OBF by increasing valuable certificates
Of the 19 states currently using OBF formulas to determine institutional funding, 17 reward certificate production. Because certificates are relatively quick and easy to produce, many institutions have turned to them as an efficient means of increasing completions and improving performance under an outcomes-based funding model.
However, the concentration of certificate completions among low-income and minoritized populations and the varied economic value of certificates raise concerns. As a result, certificate proliferation attributed to OBF is generally thought to widen equity gaps by disproportionately and negatively affecting historically underserved groups.
Yet some certificates are more valuable than others. This module provides a summary of research on the value of certificates and highlights some best practices to help states and institutions think strategically about maximizing their benefit in the context of OBF.
The market value of certificates: A mixed bag
Certificates are the fastest-growing postsecondary credential, often used as a milestone between a high school diploma and a college degree. Nearly two-thirds of students who begin their postsecondary educations with certificates go on to enroll in additional college courses, and a quarter earn associate degrees, bachelor’s degrees, or both.[i]
Certificates are commonly grouped into two broad categories: occupational, which are aligned to workforce fields; and academic, which are typically decontextualized from the labor market. This module focuses on the former, as academic certificates only constitute a small fraction of certificates awarded and are generally recognized to be of minimal labor market value[ii],[iii]. While occupational certificates are one of the main pathways into so-called “middle jobs,” those leading to a middle class wage but requiring less than a Bachelor’s degree, certificate recipients’ labor market outcomes vary based on race, gender, length of certificate, field of study, and whether the recipient works in their field.[iv],[v],[vi] For example, several studies have found that labor market returns are highest for both short- and long-term certificates in health-related and technical fields.[vii] With this variation in mind, earnings with a certificate can range from $22K with a certificate in cosmetology to $45K with a certificate in construction.[viii] A recent ranking of vocational programs also highlights this wide range of earning potential, as well as the income disparity that exists within fields depending on which institution granted the certificate.[ix]
Students are also earning certificates as milestones along a path of study, as certificates are often considered a stackable credential. While researchers recognize the potential of stackable credentials to improve completion and provide labor market advantages, there is little evidence of their impact. One study suggested that stacking motivated students to complete short-term certificates and pursue additional credentials, but the study observed racial disparities and limited labor market gains. Other scholars point to data limitations which hamper analyses measuring the impact of stacked credentials.[x]
Given the range in outcomes attributed to certificates and their disproportionate concentration among historically underserved populations, rapidly increasing certificate production to meet attainment goals may have a negative influence on students, institutions, and states. It is therefore important that states and institutions make evidence-based decisions about which types of certificates to include in OBF formulas.
LESSONS FROM THE FIELD
The six states in our study each took different approaches to rewarding certificates under their OBF formulas, attempting to strike the balance between state workforce needs, institutional financial needs, and student educational needs. Table 1 displays how each of our study states includes certificates in their OBF formulas.
Table 1. Treatment of certificates in state OBF polices
a. States in this category reward all certificates awarded to students, but the way bonuses are applied varies: we include states which reward bonus points for each certificate, states that set bonus weights for focus population, and states that include focus population metrics in their OBF formula. Focus populations are student groups that states identify as priority subgroups, e.g., low-income students, underprepared students, or underrepresented minorities.
b. States in this category reward all certificates and apply bonuses to certificates in certain high-demand fields. In contrast, states identified in the fifth column only reward certificates in the fields identified by the state.
c. New Mexico is the only state that applies a reward (bonus) metric of STEM and Health fields on all certificates but at the same time excludes short-term certificates that are not in the STEM and Health fields.
d. Ohio will reward two certificates earned by a student in the same year, but the second certificate is discounted by fifty percent.
Table 1 shows that states in our study vary in terms of the criteria they place on certificates rewarded under OBF formulas. While all six states incentivize certificate completion among focus populations, only some provide additional bonuses for certificates in high-demand fields, or weight certificates less than degrees, or require a minimum of credit hours to be eligible. Some states only reward certificates in certain fields of study, and others require that the certificate must be the highest credential earned (e.g., if a student earns a certificate in the same year as earning an associate degree, only the associate degree will be rewarded under the formula).
Determining how to reward certificates under OBF
Conversations with state policymakers revealed a range of rationales for whether and how OBF models included certificates. Primarily, policymakers considered the utility of certificates for each state’s workforce needs and the effects of certificates on student success.
Aligning certificates to state workforce goals. Policymakers emphasized the inclusion of certificates to meet workforce needs. As states and regions responded to different industries, high-impact certificates were recognized as playing a critical role in state and local economic growth. Policymakers credited the formula for helping the state think more strategically about how to align certificates with workforce needs, often making revisions following OBF implementation. For example, one state recently proposed a revision to its OBF model to reward high-impact degrees at community colleges. A policymaker said:
We have this new workforce-ready grant where we are, as a state, paying for certain certificate programs that our workforce development agency has identified as key drivers to what the economy will need now and in the future. They’re called four- and five-flame hot jobs. Our plan for our community colleges is to reward increases in specific certificate programs that have been identified as four- and five-flame degrees, pathways that are needed for our current economy and the future.
Considering whether certificates are meaningful for closing the achievement gap. Tying certificates to workforce needs, and ultimately student earnings and job placement, were important considerations. Some policymakers also considered the role of certificates in addressing the attainment gaps between student groups. One policymakers said consideration for the inclusion of certain metrics was driven by achievement data: “When we looked at the degree, certificate, and transfer rates, we saw just these enormous gaps between, say, minority students and the Caucasian students.”
Ensuring certificates are meaningful
Following OBF implementation, certificate production proliferated as institutions responded to the incentive by awarding more certificates. Policymakers and institutions alike noted the challenges to ensuring that certificates are meaningful credentials, and they acknowledged the need to strike a balance between rewarding milestones and ensuring their impact.
The need for clear policy and quality checks for institutions. Policymakers expressed concern about the proliferation of certificates. While they empathized with institutions that want to maximize their funding under the formula, they also wanted to limit the ways in which institutions might “game” the system.
One policymaker discussed the need to improve quality checks on certificates rewarded through the formula:
We’re by no means saying that institutions are just handing out certificates that have zero quality. We are saying, though, that we’ve seen a lot of movement, a lot of unprecedented growth from some institutions in the number of certificates that they award. So, what we’ve said this year is that we want to pump the brakes a little bit and look at those certificates and make sure that those certificates are the ones that the state wants to reward through the formula.
Another policymaker described revising the formula to address the proliferation of stackable credentials: “We’ve seen just a lot of increases in the stacking, especially multiple certificates in a single fiscal year, stacking of a certificate with an associate degree. While we know that those have high value, we want to ensure that we are being good stewards of state funding. We don’t want to be paying for the same thing twice.”
Ongoing review of certificate production. Given the variation in certificates, state policymakers took a closer look at the types of certificates being produced and whether these certificates are incentivized under the funding formula. One policymaker said:
There is one institution that saw a 60 percent increase in one-year certificates. A lot of the other institutions haven’t seen that sort of year-over-year growth. So, when we looked at it, we realized that these certificates were sort of general education certificates and that brought about questions of quality. In the institution’s defense, they said, ‘This is a part of our effort to get students to the finish line. This is a halfway point certificate that we’re awarding to these students to encourage them to continue.’ So, that’s one unintended consequence in that we’ve had to go back and revisit, ensuring quality and ensuring that we’re getting the degrees and rewarding the degrees that we really want.
Multiple states have revisited their OBF policy, adding additional eligibility requirements to certificates to mitigate against increases in lower quality credentials.
The shift to more pragmatic curricula design and student advising. Institutional leaders reported revising their curricula to support students earning a certificate on their way to an associate or bachelor’s degree. One administrator noted how changes in institutional curricula are helping students earn stackable credentials:
We've always talked about the stackable nature of our certificates and how they lead to associate degrees, but [OBF] has caused us to take a much closer examination of what that meant and just how stackable are they really? In many cases they were earning the certificate and the associate degree all at once and so if a student left early, they still weren't necessarily leaving with anything. We have started thinking through ways of structuring those [certificates] differently or do we advise students differently so that they really are stacking. And if a student does leave before earning their associate degree that they do have some type of credential to show for it as well.
Similarly, institutional administrators discussed how they have adjusted their advising to highlight the role of certificates along the path toward a degree. One institutional administrator described the shift: “In the admissions office, for example, when they meet with students, in the past, we just talked about channeling students into degree programs. And some would say, ‘Well, I'm thinking about business, or I'm thinking about communication.’ Well, pick one, because you're going to need it for financial aid. Well now, we have conversations, ‘All right, you're a little unsure.’ Then we introduce the idea of, ‘Have you thought about a certificate program to start that's stackable into a degree?’”
Conversations with state and institutional leaders suggest that there is no one right way to reward certificates under OBF. However, there do seem to be some strategies that states can use to help institutions respond in ways that create viable credentials for students and strengthen the quality of the state workforce. The following are key takeaways for policymakers and institutions regarding certificate production under OBF policies.
Discuss the rationale for awarding certificates during formula development. As states consider various approaches to awarding certificate production, policymakers and institutional leaders should identify the value of certificates. Cost to institution, labor market value, and use as a stackable credential on the path to a degree can all factor into whether a certificate has long-term meaning. Following OBF policy development, states should revisit the rationale for including certain certificates as costs, mark value, and degree paths change.
Consider how to award certificates in specific fields or relative to other outcomes. Valuing certificates at the same level as associate and bachelor’s degrees may provide an incentive for institutions to generate completions in the form of certificates, in a way that may not benefit students. However, some certificates are high value and may be considered a stronger credential than an associate degree in a lower-impact field. One institutional administrator said, “We're educating students in STEM fields and they're earning certificates and getting out [making] $71,000 a year. To me, that's a high-value degree. Maybe that should be worth more than an associate degree in general studies.”
Evaluate the potential earnings for various certificate pathways. Short- and long-term certificate labor market values vary overall and by student subgroups. This variation should be factored into OBF formulas to increase the possibility that certificates will help close earning gaps.
Prioritize state workforce needs and attainment goals. Utilizing workforce and educational attainment data, policymakers should identify meaningful certificate pathways that align with the economic and educational goals for the state.
[i] National Student Clearinghouse Research Center. “Snapshot Report: Certificate and Associated Degree Pathways.” https://nscresearchcenter.org/wp-content/uploads/SnapshotReport29.pdf.
[ii] Bailey, Thomas, and Clive R. Belfield. "Community college occupational degrees: Are they worth it." Preparing today’s students for tomorrow’s jobs in metropolitan America (2012): 121-148.
[iii] Xu, Di, and Madeline Trimble. "What about certificates? Evidence on the labor market returns to nondegree community college awards in two states." Educational Evaluation and Policy Analysis 38.2 (2016): 272-292.
[iv] Carnevale, Anthony P., Stephen J. Rose, and Andrew R. Hanson. “Certificates: Gateway to Gainful Employment and College Degrees.” Georgetown University Center on Education and the Workforce. 2012.
[vi] Tesfai, Lul, Kim Dancy, and Mary Alice McCarthy. “Paying More and Getting Less: How Nondegree Credentials Reflect Labor Market Inequality Between Men and Women.” New America. 2018
[vii] Bahr, Peter. R., Susan Dynarski, Brian Jacob, Daniel Kreisman, Alfredo Sosa, and Mark Wiederspan. “Labor Market Returns to Community College Awards: Evidence from Michigan. A CAPSEE Working Paper.” Center for Analysis of Postsecondary Education and Employment. 2015.
[viii] Carnevale, Anthony P., Stephen J. Rose, and Andrew R. Hanson. “Certificates: Gateway to Gainful Employment and College Degrees.” Georgetown University Center on Education and the Workforce. 2012.
[ix] Glastris, Paul. “America’s Best and Worst Colleges for Vocational Certificates.” Washington Monthly. September 2018. https://washingtonmonthly.com/magazine/september-october-2018/americas-best-and-worst-colleges-for-vocational-certificates.
[x] Giani, Matthew, and Heather Lee Fox. "Do Stackable Credentials Reinforce Stratification or Promote Upward Mobility? An Analysis of Health Professions Pathways Reform in a Community College Consortium." Journal of Vocational Education & Training 69 no. 1 (2017): 100-122.