Back in the 1970s and the 1980s, there was a high level of concern about the concentration of social ills in poor neighborhoods. At that time, the devastation wrought by the crack epidemic, the rapid rise in out-of-wedlock childbearing, and the high levels of violent crime in the inner cities led a number of journalists and scholars to talk about the emergence of an “underclass.” This term was controversial, and over time has fallen out of favor. Nevertheless, the dangerously self-destructive behaviors that gave rise to the underclass debate, and particularly the geographic concentration of these ills in inner cities, were legitimate topics of public concern and led to a burgeoning of research on this group, including attempts to measure its size, composition, and location.

Defining the Underclass

In 1988, Erol Ricketts and Isabel Sawhill proposed a conceptual definition of the underclass and then produced some of the first empirical estimates of the size of the underclass using available census data. A number of journalists, such as Ken Auletta and Leon Dash, who had written about the underclass emphasized that it was not just about low income, but also about destructive or anti-social behaviors. When Ricketts and Sawhill considered which behaviors to focus on they chose four. To have a shot at the American dream, they argued, you needed to stay in school at least through high school; you needed to not have a baby before you were married or prepared to support a child; you needed to work or actively seek work; and you needed to be law-abiding. This is what most Americans expect other Americans to do. If you are poor, despite doing all of these things, then you are not part of the underclass and have a reasonable chance of making it, especially if provided with an appropriate set of social supports. But if you fail to behave in accordance with this set of societal norms, you are less deserving of social assistance and could be considered part of the underclass.

Instead of focusing on individuals, however, Ricketts and Sawhill emphasized neighborhoods because of the tendency of these dysfunctional behaviors to concentrate in certain areas and to become normative or quasi-normative for the adults, and especially the children, growing up in these neighborhoods. If many of one’s friends are teenage mothers, for example, then it is easy for any individual teen living in the neighborhood to see this as an acceptable, perhaps even a desired, lifestyle. But the development of such norms—norms that are at odds with mainstream expectations—leads to a diminution of social mobility for those growing up in such neighborhoods and imposes costs on the rest of society.

When it came to translating their concept of the underclass into something that could be measured with available census data, Ricketts and Sawhill had to make a number of compromises. They defined the underclass as all those living in census tracts with very high concentrations of certain behaviors. Specifically, the tract had to be one standard deviation above the mean for the country as a whole on all of the following indicators: proportion of teenagers dropping out of high school, proportion of women heading a family, proportion of households on public assistance, and proportion of prime-age and able-bodied men not in the labor force. (Data on crime are not available from the census.) The underclass is then defined as those living in these troubled neighborhoods, regardless of whether they themselves engage in such behaviors. Everyone living in such an environment is exposed to lifestyles that are not conducive to joining mainstream American society. In recent decades, this definition has meant identifying as an underclass neighborhood any census tract in which more than a quarter of the relevant population dropped out of high school, more than a third lived in female-headed families, more than a sixth were on welfare, and more than half the men were out of the labor force.

Size and Composition of the Underclass

Using this measure, a number of different researchers have now analyzed the size and composition of the underclass. The group is relatively small (2.2 million people living in 775 neighborhoods, according to an analysis of 2000 census data by Paul Jargowsky and Rebecca Yang). The underclass is disproportionately minority, and concentrated primarily in large urban areas, especially in the mid-Atlantic and Midwest areas of the country. Some people think of the underclass as synonymous with being poor, but this would be a mistake. For the past several decades there have been over 30 million people living in poverty, but even at its peak no more than 3.4 million people living in the kinds of neighborhoods we associate with the underclass. Moreover, not all of the residents of these neighborhoods are poor. Indeed, only 57 percent of underclass neighborhoods qualified as high-poverty areas in 2000 (meaning census tracts where more than 40 percent of the population was poor). In short, underclass behaviors and low income are correlated, but not nearly as much as has been commonly assumed.

The Decline of the Underclass

Perhaps the most striking finding about the underclass is its recent decline (see figure 1 on pdf). Using the Ricketts-Sawhill definition, various researchers have found that the underclass grew dramatically in the 1970s, edged up further in the 1980s, and declined quite sharply in the 1990s (although not to its 1970 level). The immediate reasons for the decline were reductions in the number of census tracts with high levels of dropping out of high school and high levels of public assistance receipt. The smaller number of tracts with high levels of welfare dependency undoubtedly reflects the reform of welfare in 1996 and the subsequent sharp decline in caseloads. The number of tracts with a high proportion of female heads continued to increase over the decade, although at a slower pace than earlier, and the number with a high proportion of men not in the labor force remained about the same. This last finding is somewhat surprising given the very strong economy that prevailed during most of the decade. In terms of where the declines in the number of underclass tracts were greatest, New York and Detroit led the way with Chicago, Philadelphia, and Los Angeles making impressive improvements as well. The declines occurred among all racial and ethnic groups but with whites and non-Hispanic blacks faring somewhat better than Hispanics.

Taking a Broader Look

As we have noted, the Ricketts-Sawhill measure is dependent on census data for its construction, and as such, misses some key behaviors that these two researchers as well as most other people associate with the underclass. Recall, for example, that Ricketts and Sawhill focused on law-abidingness as a key element in their definition. Although it is not possible to measure this in census reports, we know from other sources what happened to crime rates over this period. The pattern is similar to that reported above for the census-based underclass measure. The violent crime rate, in particular, increased dramatically during the 1970s, declined slowly in the first half of the 1980s but increased greatly in the latter half of that decade, and declined sharply after 1991.

Another element in the Ricketts-Sawhill definition was waiting to have a baby until marriage, or at least until adulthood when one is more likely to be able to support a child. In the census-based measure, this is proxied by looking at female-headed families and reliance on public assistance. But by turning to data from other sources, we can round out the picture by examining trends in the teenage birth rate and in the proportion of children born outside of marriage. Unlike our other measures, the teen birth rate declined rather than rose during the 1970s, but shot up in the 1980s, and then declined sharply in the 1990s. Overall, the teen birth rate dropped from 6.6 percent of all teens aged 15 to 19 in 1970 to 4.8 percent in 2000. Most births to teens occur outside of marriage and require that the teen turn to public assistance for support. But not all births outside of marriage are to women under the age of 20. To see what has happened to out-of-wedlock childbearing more generally, we can look at the proportion of all babies born outside of marriage. This proportion rose throughout the period but at a decreasing rate and then leveled off at around one-third of all births in the late 1990s. Finally, even among never-married mothers, employment rates have risen dramatically.

In sum, the conclusion that the underclass, and the behaviors that define it, have declined over the past decade seems unassailable.

What Caused the Decline?

The reasons for the decline of the underclass remain largely a matter of conjecture, but we can shed some light on this question by suggesting at least some of the major arguments that various experts have advanced as explanations for the trend.

One argument is that the underclass engages in what, from a middle-class perspective, seem like antisocial behaviors, but are in reality an adaptation to the tough environments in which poor people live. If true, we should see a fall in poverty rates preceding the decline in the underclass. Poverty rates did decline sharply in the 1960s and to a lesser extent in the early 1970s. Since that time, they trended upwards, at least until the mid-1990s. Thus, the timing of these two trends doesn’t provide much support for this explanation. This rather simple-minded analysis shouldn’t rule out the possibility that bad behavior is the result of low income, but it doesn’t provide any strong support for the theory. Nor does the disjuncture between underclass neighborhoods and high-poverty neighborhoods.

Another argument is that the successes of the 1990s were the result of an unusually strong economy. It is true that jobs were plentiful, that the unemployment rate had fallen to 4 percent by the end of the decade, and that earnings gains for those at the bottom of the ladder were quite impressive. However, almost all of this good economic news came in the latter half of the decade and thus likely post-dates the beginning of many of the trends described above. To be sure, some of the decline in welfare caseloads is attributable to the strong economy in the second half of the decade. However, the failure of the male labor force participation variable to improve significantly argues against a purely economic interpretation.

A third argument is that public policy has gotten smarter about linking assistance to personal responsibility. During the 1960s and 1970s, as a result of the welfare rights movement, along with changes in attitudes about sex, childbearing, and marriage, welfare caseloads tripled. This may have helped to shore up the incomes of poor mothers and their children, but it also enabled women to raise children on their own without working and probably contributed to a change in social norms within the neighborhoods where early childbearing and welfare dependence were most prevalent. The welfare reform law of 1996 changed all of that by requiring work and limiting the duration of federal assistance to five years. In addition, the expansion of the Earned Income Tax Credit, child care assistance, and Medicaid during the 1980s and 1990s helped to make work in a low-wage job more rewarding than welfare for the first time. In a similar vein, the public education system came under attack during the 1980s for its failure to educate students to succeed in the new economy. For example, A Nation at Risk, published in 1983, called on schools to be held accountable for what students were learning and led states to strengthen graduation requirements. For the first time, in a 1989 summit, the 50 state governors and the president set a goal of having 90 percent of students graduate from high school. Finally, in the criminal justice area, more police on the beat in local communities and longer sentences not only sent tougher messages to would-be offenders, but also took many criminals off the streets.

A fourth explanation centers on a sea change in federal housing policy—moving away from policies that concentrated low-income persons. The demolition of high-rise public housing projects and the greater emphasis on low-rise, scattered site public housing as well as portable Section 8 certificates acted to deconcentrate low-income persons. This reduces the underclass measure directly, by reducing the policy-induced creation of extreme neighborhoods. Moreover, by reducing the concentration of troubled families, it becomes far less likely that any given neighborhood will have the critical mass of social troubles that can lead to the development of self-destructive norms. The timing of this policy change is broadly consistent with the underclass trend.

A fifth argument is that the greater availability of abortion after the Roe v. Wade decision in 1973 led to fewer unwanted births—especially among poor black women who are young and unmarried at the time of pregnancy—with the result that there was a subsequent decline in crime and other antisocial behavior once the cohort of children born in the post-legalization period reached their late teens in the early 1990s. This argument has been popularized by University of Chicago economist Steven D. Levitt, but has been criticized by some other scholars.

A final possibility is that all of these reasons and perhaps others have worked in combination to produce the trends we have described. Human behavior is always difficult to predict and history cautions against any simple answers. It may be that risky behaviors such as crime, violence, unplanned teen pregnancies, and drug use move in cycles with each generation learning from the immediately previous one the devastating consequences of such behaviors, and then these lessons are lost on the next generation. Add to these normal generational swings some of the factors described above and you have at least a plausible explanation for what occurred.

Conclusions and Policy Implications

Not that long ago, the problems of the concentration of poverty, the underclass, and the inner city were unfailingly referred to as intractable. The 1990s, however, were a remarkable decade in which substantial progress was made against all these problems. A wide ranging set of forces undoubtedly contributed to these improvements, including the strong economy, favorable demographic trends, and several major policy innovations inspired by both the right and the left. While the relative roles of these different forces and policy changes is difficult to disentangle, on balance they had their greatest impact in the inner cities of our metropolitan areas.

The changes experienced by inner-city neighborhoods are quite profound and have received insufficient attention. Previous research has shown a precipitous drop in the concentration of poverty. The Ricketts-Sawhill underclass measure reveals even greater progress. It is possible that reducing concentrations of poverty has had a disproportionate benefit in reducing social problems. However, a strong conclusion on this front is probably not justified, given uncertainties about the direction of causality.

We can, however, safely conclude that many fewer neighborhoods now resemble the depressing descriptions of the inner city that were commonplace in journalistic or scholarly accounts of previous years, such as those by Ken Auletta, Leon Dash, and Alex Kotlowitz. While many neighborhoods remain troubled, and many older inner-ring suburban neighborhoods are showing signs of distress, far fewer neighborhoods than in the recent past have the confluence of low levels of education, few men in the workforce, and large numbers of unmarried women raising children on welfare. Crime rates have fallen, teen pregnancy and birth rates are down, and the rising proportion of children born outside marriage has slowed or leveled off. Progress on this scale—a relatively rare occurrence in the arena of social problems—should be celebrated.

That said, it would be a grave mistake to complacently assume that these changes will continue or even be maintained. Without public policies that discourage crime and encourage education, work, and delayed childbearing, as well as housing policies that facilitate spatial access to opportunity, these positive trends could easily reverse. The two groups most at risk are adolescents and low-income working families. These two groups need ladders into the middle class. For adolescents in distressed neighborhoods, this means smaller, more effective high schools, afterschool programs, and adult mentors or counselors. For low-income families, the success of welfare reform in moving many single mothers into the job market needs to be followed by efforts to keep them there with wage supplements, child care, health care, and employer-based training programs. Without strong rungs on the bottom of the ladder, they—or more likely their children—could just as easily become tomorrow’s underclass.