The limits of naming and shaming in higher education

Accountability measures have an important role in higher education, but not all accountability measures have received extensive attention. The majority of prior research and public focus has been on high-stakes accountability measures like cohort default rates, financial responsibility scores, and heightened cash monitoring. Less explored are the low-stakes accountability policies that are not tied to institutions’ ability to access Title IV federal financial aid funds. These are generally policies that require institutions to submit information to the federal government that is then communicated to students, their families, counselors, and the institutions themselves. A primary goal of these types of policies, according to Drs. Katharine Meyer and Kelly Ochs Rosinger, is to help by “simplifying the college-going process and helping students make informed decisions.” One of the key federal low-stakes accountability policies for higher education are the College Affordability and Transparency Center (CATC) lists.

CATC lists

Since 2011, Congress has required that the Education Department (ED) oversee the creation of six annual lists focused on tuition and required fees—referred to as tuition for the rest of this essay—and average net price. ED explains that the CATC lists were created “with the goal of increasing transparency of college tuition prices for consumers.” Table 1 describes the lists, which highlight institutions near the top or bottom of various categories. The four “level” lists identify institutions that meet the tuition or average net price criteria for the most recent year within their sector (such as public four-year, private not-for-profit two-year). The two “change” lists identify institutions that meet the increase criteria over a three-year period within their sector.

Table 1: College Affordability and Transparency Center lists

If institutions are included on a level list, they are listed on the CATC website. If institutions are included on a change list, they are both listed on the website and an institutional administrator must complete and submit to ED the College Affordability and Transparency Explanation Form.

These lists have been contested in both the press and among institutional administrators. One of the key reasons is that it is not clear what being included on a list actually means. For example, in the 2017 list, Carver Bible College is included in the change-in-tuition list while Loyola Chicago is not. If the point of the list is the affordability of a postsecondary education, one might expect that Carver Bible College would be significantly more expensive than Loyola. In actuality, the 2015-2016 tuition and fees (which the 2017 list is based on) for Carver were $9,860 while that same year Loyola charged $40,426. The reason that Carver is included on the list while Loyola is not is that Carver had an 18.44% change in tuition from 2013-2014 while Loyola only had a 13.64% change (from $8,325 and $35,573, respectively).

This contention about the usefulness of these lists is exacerbated by the argument that most students and counselors do not know about the lists. With these hurdles, it can be difficult to understand how being included on one of the CATC lists would affect institutional behaviors or enrollment decisions.

New evidence on the effects of CATC lists

Using the Integrated Postsecondary Education Data System (IPEDS), I analyzed the effect of list inclusion on the change lists (since these lists involve the most sanctions). I examined institutions that just missed making the top 5% change list and compared those to institutions that just made the list. The intuition behind the analysis is that being in the top 4.9% and being in the top 5.1% is arguably up to chance in a way that being in the top 20% versus the top 5% is not. It is unlikely that institutions can manipulate whether they will fall just above or just below the arbitrary cut-off of the 95^th percentile that ED has selected. Moreover, it would be difficult for institutions to precisely manipulate their way off the list, since they would also have to guess the changes in tuition or average net price for other institutions in their sector. Therefore, institutions that fall just above and below the cut-off are likely to only differ in the institution’s inclusion on the CATC lists. If these assumptions are met, then my approach produces plausibly causal estimates. More details on the methods appear in the published paper.

I examined the relationship between institutions being on the change lists and their affordability and enrollment in the following years. I found little to no evidence of an effect on institutional behavior or students’ enrollment decisions when institutions are included on these lists. I explored this relationship in several different ways, including allowing for a lagged effect, examining individual years, and investigating heterogenous effects in enrollment by student race/ethnicity. I still found little to no evidence that institutions that barely made the list behaved differently or had different enrollment patterns than institutions that just missed the list.

Policy implications

Based on prior research, low-stakes accountability policies may be more effective when focused on earnings or when they provide individualized information directly to students. It seems clear that the type of information presented and the manner in which it is presented will influence the efficacy of accountability policies. In the case of the CATC lists, the current model requires the public to know about and access the lists on the government’s website or for the press to communicate about them. With no consistent evidence that students or institutions are responding to these lists along enrollment or affordability measures, however, future discussions could be warranted on whether these lists still need to be created. The calculation and publication of these lists place a burden on career staffers at ED as well as institutional administrators.

The recent study’s results do not mean that it is impossible for a similar information-based accountability policy to have an effect. However, at the least, the evidence shows that the current construction of the policy does not appear effective in changing institutional pricing or enrollment behaviors. Stopping the creation of the CATC lists would require an act of Congress; this could occur with Higher Education Act reauthorization policy conversations currently ongoing, though these conversations have stalled as a result of the ongoing COVID-19 pandemic. If more impact is desired from this type of accountability policy, revisions are needed.

A robust federal accountability framework likely includes both high-stakes and low-stakes accountability policies. Low-stakes accountability policies, by their structure, have less significant penalties attached to them and it would be reasonable to expect that they have smaller effects on student and institutional behavior. However, even with these smaller expectations for low-stakes accountability policies, there is little evidence that “naming and shaming” institutions about their affordability is an effective strategy. It is likely that the external and internal pressures around affordability—including the economy and state support for public institutions—are too large for these types of accountability pressures to influence pricing or aid-giving decisions. Also, even if students, their families, or counselors learn about an institution being on one of the lists, that does not mean that the students have other, better options to attend nearby. Together, these factors suggest that a low-stakes “name and shame” approach as currently envisioned is adding administrative burdens without substantially improving college affordability.