How Well Are American Students Learning?

Transcript

Ron Nessen: Good morning, and welcome to The Brookings Institution, a National Issues Forum titled "How Well Are American Students Learning?" And, at this forum we will release the first Annual Report on American Education from the Brown Center on Education Policy. I want to especially welcome our viewers who are watching this forum on CSPAN.

The format today will be that Tom Loveless, who is the director of the Brown Center on Education Policy, will review the report in considerable detail, explain what it's findings are and so forth. This is a report which examines achievement in American schools, evaluates whether any gains have occurred, and whether they're large or they're small, and it takes a look at the policies and practices which influence student learning.

After Tom has outlined the report, explained the report, we'll hear some comments from Jane Hannaway and from Checker Finn. Jane Hannaway is an organizational sociologist whose work focuses on the study of educational organizations. She is currently director of the Education Policy Center at the Urban Institute in Washington. She is also a senior researcher with the Consortium for Policy Research in Education. Her recent research has focused on comparisons of urban and suburban school districts and structural reforms in education, particularly reforms promoting competition and choice.

Checker Finn is -- has devoted most of his career to improving education in the United States. He is the John M. Olin Fellow at the Manhattan Institute, and the president and trustee of the Thomas B. Fordham Foundation. His primary focus is the reform of primary and secondary schooling. He has served previously as an assistant secretary in the Department of Education.

You'll hear their comments on the Brown Center report. There will be a discussion between them and Tom Loveless, and then the audience will be invited to participate with questions or with comments. This is obviously a timely topic. It's a big topic in the presidential campaign, and it usually finishes at the top of the list of public concerns. I should explain that this is the sixth in a series of National Issues Forums that we call Priorities 2000 -- P2K -- featuring discussions of the eight issues that we believe the presidential candidates ought to be discussing and that the voters ought to be interested in. And they're obviously interested in this issue of education.

Just briefly let me mention that there are more details about the Brown Center report available on the Brookings website at www.brookings.edu. You'll also be able to find a full transcript of this event and a full video of this event on the website. And Tom will have a live, online website chat on October 4th at 2:00 p.m. on this topic if you want to take part further in that. Tom is also taking this show on the road. After unveiling and discussing the report here for national audience, he will go during the next two weeks to New York, Chicago, Seattle, Los Angeles, and Houston to talk more about it, to have local experts and local educators focusing on the local situations in those cities.

Let me introduce you now to Tom Loveless who, as I said, is the director of the Brown Center on Education Policy and a senior fellow here at Brookings. His latest book is The Tracking Wars: State Reform Meets School Policy. He is also the editor of Conflicting Missions? Teachers' Unions and Educational Reform, published by the Brookings Institution Press. He -- immediately before coming to the Brown Center, he was at Harvard as both an assistant and associate professor of public policy at the Kennedy School of Government. Tom has one additional qualification for talking about education. He was a sixth grade teacher, so he knows about this from both the theoretical point of view and also the practical point of view. He taught sixth grade for a number of years in hometown of Sacramento, California.

So, I will turn the program now over to the director of the Brown Center on Education Policy, Tom Loveless.

Tom Loveless: Thank you, Ron. And let me also welcome all of you to Brookings today. We appreciate your attendance. Let me thank a couple of people before we begin. I want to thank two colleagues in the Brown Center -- Paul DiPerna and Judy Light -- for helping put the report together. And also, the firm of Lipman & Hearne, Chrysoula Economopoulos and Laurie Glenn, as well as Rodney Ferguson, who was unable to be here today. They helped put together the production of the report and also promoting the report.

Now, the Brown Center report really is organized around three sections, so let's get started. First of all, we're going to be looking at trends in student achievement. What we'll do annually is we will take the best data we can find and really say how kids are doing in reading and mathematics at both the elementary and secondary levels. Secondly, you'll notice these two things I'm going to cover today -- we have several points I'm not going to cover -- but I'll be talking about the center section, which the theme this year is mathematics. I'll be discussing some contradictory signals that we're getting on the pace of progress in mathematics from the NAEP test -- the National Assessment of Education Progress -- and I'll briefly touch upon math learning in the 1990s, what it is that kids are learning. Finally, I'll wrap it up talking about exemplary schools programs. These are programs that give awards to schools for being excellent schools.

So, let's take a look at student achievement. These are data from the NAEP test, the National Assessment of Educational Progress. It tests at three different ages. First of all, nine-year-olds along the bottom here, 13-year-olds and 17-year-olds -- these are reading scores. Basically reading scores have been flat. The test was first given in 1971, and as you can see, there are just minor gains. Nine-year-olds have gained just four scale score points since 1971. In fact, reading achievement among nine-year-olds peaked here in 1980 -- 215 scale score. You can see we're down to 212 in the last test, which was 1999. Thirteen and 17-year-olds also very little gains. The gains are only minuscule and for the most part reading achievement has been flat over the last 28 years.

Math is a much brighter picture. There are gains across all three groups -- nine -year-olds, 13 and 17-year-olds, and these gains look rather significant. All three groups in 1999 scored at the highest level they've ever scored on NAEP -- all three groups. And you can see in terms of the scale score gains, a 13 point gain for the nine-year-olds, the youngest kids, and large gain also for the 13-year-olds. The 17-year-olds, however, only a four point gain. So, the oldest students in NAEP aren't showing the same kind of progress in mathematics. Now, how can we quantify this gain? Is this a large gain or a small gain? What kind of gain is it?

Take a look at this first column. These are the standard deviation units -- it's the units statisticians use to measure variation. The reading gains are very small. Basically, we consider anything between zero and .20 to be a small gain; between .20 and .50 a moderate gain or a modest gain, and over .50 a large gain. Well, if you look down the first column, we don't have any large gains. We don't have any even significant gains at all in reading. In mathematics, at both age nine and 13, around a third of a standard deviation gain. That's a modest gain. So there has been progress in math achievement since the early 1970s.

Another way of looking at this even makes that gain look more impressive, and that's if we convert into years of learning. Take a look at the second column, and let's focus now on the nine-year-olds and the 13-year-olds -- that's where we have the largest gains -- and that each gained about one year's worth of learning since the early '70s. What that means is a nine-year-old today essentially knows about as much math as a ten-year-old did in 1973. And that's fairly significant progress. You can see the age 13-year-old has also gained about one year's worth of learning. That's pretty good.

Now, before we get carried away and start throwing ticker tape parades with 13-year-olds packed into cars to celebrate this, we need to think about something. There's another way of comparing that throws a little cold water on this, and that is if we consider these gains in relation to other countries in the world, say the highest scoring countries in the world.

Now, I want to focus just on 13-year-olds for a moment. The rate of gain for 13-year-olds works out in terms of a rate at about 1/8th of a standard deviation per decade, that they're increasing their achievement. Okay, about 1/8th of a standard deviation per decade. Now, if you take that rate of gain and you go back and you look at the TIMS test, which was an international test given in the mid-'90s, and you compare U.S. achievement of 13-year-olds to the highest scoring countries in the world, the question is, at that rate of gain, how long will it take for us to catch those highest scoring countries? The answer is a very long time. Those 13-year-olds are going to be 138 years old before we catch Singapore, and you can see to catch Korea or Japan. For our 13-year-olds to be performing at a level equivalent to theirs also will take a very long time.

So, what's the bottom line? I'd summarize achievement gains over the last three decades basically like this: In reading, there's been virtually no gain whatsoever. Scores have been flat. In mathematics there have been significant gains, but the rate of progress is really at a snails pace and over 26 years a snail can cover quite a bit of ground, so the gain over that 26 years is significant, but it's very slow, and we shouldn't celebrate too much because we still have a long way to go to match the highest scoring countries in the world.

Let's turn to the second section of the report, and this really deals with the issue of the quality of the evidence, the kind of evidence we're getting in math achievement from the NAEP test. We posed the question: How could one test give two different results? And the answer to the question is well, the NAEP really isn't one test. It really is two tests. There are two tests. There is what's called the main NAEP. This was started in 1990, and it was designed to reflect current practices and current curriculum. Those scores that I just showed you earlier, that's called the Long Term Trend NAEP, and that's designed to gauge achievement over time.

Now, here's the bottom line to this story, and I think it's a significant one. These two tests have diverged in the 1990s. One of the tests, the main NAEP, is telling us achievement is skyrocketing. Kids are learning a lot more mathematics. We just looked at the trend NAEP, and even though it shows gains, it's not showing huge gains. Take a look at the gaps that have developed. This is for grade four. And you'll notice you have on the main NAEP, that's this top line, the solid line, about 11 scale scored point gains on the main NAEP in mathematics, but on the trend NAEP in mathematics, really a one point loss. This is to '96, to 1996. We have trend data in 1999. And even though that bottom line ticks up I think one or two points, that gap is still huge. In fact, the gap is about as large as all of the gain over the history of NAEP testing. So we have two NAEP tests and they're telling us a different story. And you can see that also with Grade 8, you see the gap. And with the oldest kids, the 12th graders, again, a very substantial gap has developed between these two tests.

Now, why is this happening? Why the divergence of the two NAEP tests? They're both supposed to be assessing mathematics. There are really five reasons. The first is the framework that governs the NAEP is designed to change. It can be tweaked each time the NAEP is administered, and so the test is a moving target. It keeps changing. That's the test that's showing these dramatic gains, don't forget. Also, the content of the two tests is much different. The main NAEP was designed to reflect the recommendations of the National Council of Teachers of Mathematics. It's a reform oriented math content. The main NAEP emphasizes things such as geometry, problem solving, data analysis. The trend NAEP is more old fashioned math. It's more traditional mathematics, the one that's not showing much progress. Kids are given manipulatives to use while they take the test on the main NAEP. They're not on the trend. They're given little geometric figures when they answer questions about shapes. Kids are allowed to use calculators on the main test. They're not allowed to use them on the trend. There's a lot more partial credit given to answers -- even problems where the answer may be incorrect if the kid has set up the problem right on the main through extended and short response answers, response problems, than on the trend.

So the basic story here is the two tests are designed differently. The one that has all of these different features built in is telling us a very optimistic story, a rosy story about math achievement, but the long-term trend is not. It's saying you should be wary of that rosy scenario.

Another issue is when you dig down beneath these numbers, and I think this is one of the things we need to be concerned about if we get too optimistic given the main test scores, is if you look at what kids are actually learning, there are some disturbing things in these numbers. What we did is we took the items that are on the NAEP, released for public access on the NAEP website, we clustered them by areas of mathematics and then assessed how well kids were doing on them. And as you can see, the top two items here -- addition of whole numbers, subtraction of whole numbers -- you know, 13-year-olds have that totally under control, and that's great, because that's really stuff that's taught back in second grade. But they are scoring at very high levels -- around 90 percent proficiency. Data analysis which involves reading graphs and charts, again very good score, 86 percent in 1996. But then we fall off a cliff. And these very essential skills -- for instance, the skills that when I taught sixth grade, I was preparing kids for algebra, and the skills that my kids needed were things like decimals, fractions, percents, integers.

Take a look at where kids, the proficiency level with those -- decimals, 64 percent. Yes, we've a had a little growth, but we're still functioning at a very low level. These are 13-year-olds, so they're in the 8th grade, they're getting ready for high school. And I'm telling you, as a teacher, if these were the scores for my class, I wouldn't be able to sleep at night. There is no way these kids are ready for algebra -- most kids in the United States at age 13.

Take a look at fractions, 54 percent, and this one especially disturbs me because there was a decline from 1990 to 1996 in terms of three percentage points in the proficiency with fractions. If you don't know fractions, if you have to stop and think when you're solving equations -- if you have to stop and think about what to do with fractions, you are not going to go on to symbolic mathematics.

Let's turn to the final section of the report. We asked the question: Are exemplary schools truly exemplary? In the report we looked at two kinds of programs. We looked at the federal program -- that's known as the Blue Ribbon Schools Programs -- and we look at some state programs. Today I'm just going to talk about Blue Ribbon Schools. Now, what the Blue Ribbon Schools Program does is they have a great website, and if you're a principal of a school, you can log on to their website and you can download the application packet and apply for a blue ribbon, for this awards program. It's a self-selection program, in other words. Principals apply and school staffs apply to get the award. They submit their application packet to the state department of education that then screens and forwards on their state's nominees to Washington for selection. About half of the nominees that come from the states eventually wind up getting a blue ribbon.

Now, what we asked was, okay, that's fine. How are these blue ribbon schools chosen? In the packet, there are these eight criteria listed. And I have the bottom on in red font because that's the one that involves academic achievement. There are a number of criteria as the schools work through filling out their forms, that answer questions about and provide documentation for before they get to number eight, called "Indicators of Success," which has to do with how well the school is teaching their kids academic subject matter.

Now, there's a price that's paid for this being one of eight, and being number eight. And the price is -- is that high achievement is not a necessary qualification for winning a blue ribbon award. It's one of the criteria that the awards program has, but it's not the most important, and it's simply one of eight.

What we did, and let's take a look at some data, we went and we looked -- we were able to gather data from seven states in terms of test scores -- these are elementary schools and these are the 1999 blue ribbon winners -- we're looking at now 1999 test score data. And what we did was we adjusted the data by controlling for socio-economic status of the kids. So, we're comparing poor schools -- schools in poor neighborhoods to other schools in poor neighborhoods, and schools in middle-class neighborhoods to schools in other middle-class neighborhoods, et cetera. And we asked the question: Where do they fall? And the two categories -- let me explain them. The top ten percent, when I went into this, looking at these numbers -- and quite frankly, I had no idea what the answer to the question would be, I had no idea how they would achieve, but I assumed that most of them would be in the top ten percent of schools if they have a blue ribbon award. These are all winners of the award. It turns out only about a fourth are in the top ten percent of similar schools in their state. And don't forget, we controlled for socio-economic status in this analysis.

And then I thought, well, you know, the other side of the coin is how many are below average in academic achievement for similar schools in their state? And I thought there should be zero. These are sort of misfires on the achievement criterion. And it turns out about an equal number -- again, about one-fourth are in the bottom 50 percent. They're below average for similar schools, for schools serving similar populations of children. Now, the remaining half scored somewhere in between. They have good scores, but not great scores. That's the profile of achievement of the blue ribbon winners.

Now, the states have some flexibility here, right, because they screen before they send on the nominees to Washington. So, we also wanted to ask the question: Do the states look a little bit different? And they do. Some of the states appear to put a greater premium on achievement than other states. Take a look at Michigan and let's just look at the elementary schools. I've divided up the schools by deciles, so deciles one through five, those are low-achieving schools. One is the lowest; ten is the highest. And here are the nine winners and how they are distributed among those deciles. Now, take a look. There's three schools here in the fourth and fifth decile, so they're below average for schools like them. And there are actually three schools that are typical of the schools that are scoring below average in this program. What they are is they are very wealthy schools. They are schools in wealthy communities. They have won blue ribbons. And they have good test scores, but not great test scores. So, they're doing okay with student achievement but not great with student achievement. In Michigan they give a test called the MEAP test, for instance. Now, in these schools that are similar to these wealthy schools, the average percent of kids who get to the satisfactory level on the MEAP is 85 to 90 percent. There are lots of schools that get almost all their kids to the satisfactory level. These three schools get around 72, 73, 76 percent of their kids to the satisfactory level.

So, the bottom line is, on this one criteria, and I recognize we're giving these blue ribbons for lots of other things, but on this one criteria, student achievement, these schools are not -- those three schools at least are not extraordinary.

Here's California. They have lots of winners. You see a similar pattern actually to Michigan. You have nine schools below average winning blue ribbons, but they don't do as good a job as most schools do when serving similar populations of kids as they are serving. Take a look at one of the high schools. It actually scores in the bottom ten percent of schools in terms of their test scores.

Pennsylvania is a state, at least at the elementary level, that does appear to put a premium on high achievement. Take a look there. There are 10 award winners, and nine out of the 10 are in the top two deciles. So, nine out of those 10 winners score in the top 20 percent of schools. So, the states do show some variation.

Now, why is this happening? Well, I think the main reason why it's happening is the Blue Ribbon Program is set up -- and by the way this is not a partisan criticism. This is a program that was launched in the Reagan administration, was carried on in the Bush administration, it's still carried on in the Clinton administration. I think the program is structurally flawed. It's been structurally flawed throughout all three of those regimes.

But what's the problem with it? Well, the main problem with it is it judges schools by what they do instead of judging them by what they accomplish. It doesn't go out and find schools that actually do an extraordinary job of teaching kids math and reading and give awards to those schools, what it does is it says we have a whole bunch of practices that our experts here, we assemble the panel -- we have a whole bunch of practices that we think are best practices. They're cutting edge. They're innovative. And here's an application packet. If you can tell us how many of these you're doing, gee, we might give you a blue ribbon award. And that's basically how the program is set up.

Now, the problem with these practices is they're all fine. Look, we can't criticize them -- well actually I can criticize a couple -- I will. I have criticized a couple of them. But, for the most part -- I mean, providing health services and safety programs, that may be absolutely terrific for some schools to do. The point is it's not connected to achievement in either reading or mathematics.

Now, some of them are simply misguided. Take a look. Curriculum integration, which means having interdisciplinary curriculum, where you teach several subjects in certain projects, and take a look at student-initiated learning, which is encouraged in the application packet. Diane Ravitch just published a book called Left Back: A Century of Failed School Reforms. Well, that century of failed reforms are reforms like that. It's a century of failed progressive education reforms. Student-initiated learning, for instance, comes from the 1920s. There's nothing new about this whatsoever. Harold Rudd published a book in 1928 called The Student Centered School, and it was all the rage. And student-centered schools were all the rage. In the 1960s we had something called "discovery learning." It was all the rage. When researchers sat down and actually looked at the impact of discovery learning, it did not promote learning -- it hindered it. Today we have constructivism, which is another manifestation of student-initiated learning.

The point is, when we have student-initiated learning, when we allow students to be the captains of their own ship, a lot of those ships sink, and most of them don't even get out of the harbor.

So, here's the bottom line to our recommendations in this report. We urge the Blue Ribbon Schools Program to start awarding schools and award for teaching kids math and reading. The schools that do the best job of instructing children should be the schools that are winning this award. The second thing is it shouldn't be self-selection. You shouldn't allow schools to nominate themselves. This has an inherent bias towards wealthy schools, because they have the resources, they know how to gain application packets, and they have the resources to put together these nice, shiny proposals.

Let me sum up. I've covered four points this morning, and I hope you'll take the time to read the report. We have several other issues in it. First of all, growth in reading achievement is minuscule. Growth in math is significant but slow. American math performance must improve a great deal before matching the world's highest achieving countries.

Number two, the two national NAEP tests, and that's the main and the trend, show contradictory trends in math achievement. The main NAEP test is saying that kids are skyrocketing in their math achievement in the 1990s. The trend contradicts that. It says that achievement is essentially flat, maybe up a little bit.

Number three, student performance is poor on basic math skills and specifically 13-year-olds on the skills they need to succeed at algebra.

And number four, high academic achievement is not necessary to receive an exemplary school award.

And with that, I'll stop right there. And I look forward to hearing what Jane and Checker have to say. Thank you.

Jane Hannaway: I want to begin by applauding Tom and the Brown Center and Brookings for preparing this report. Given the accountability movement in education in the United States now, there's a tremendous amount of new information that's coming out in the performance of U.S. schools, and I don't think we can over-analyze these data. So, I think the more report we have, the more people we have trying to keep track of where the trends are going in U.S. education and some of the reasons for it, the better off we'll all be.

What I'm going to do is comment on a number of items in the report -- some of which Tom mention and some of which he didn't mention but which you can read in the report.

One point that is stressed in the report is why we don't see the gains in 17-year-olds where we see larger gains in the lower grades. I didn't find this surprising, and I didn't find it surprising for a number of reasons. And I think we have to think about this. First of all, much of the focus of reform in the decade -- certainly the decade of the '90s -- has been in elementary education. It has not been in secondary education. If you look, for example, at the comprehensive school reform models that are out there, most of them pertain to elementary schools. So I think the level of effort in reform has been focused much more directly and much more seriously on elementary schools than high schools.

Secondly, it's much more difficult to change secondary schools. They're highly differentiated organizations. They aren't run by a principal. Each department has a tremendous amount of autonomy, and I think they're very difficult organizations to change. So, that's a second reason.

A third reason is, when you look at the NAEP transcript study and try to interpret that, it appeared to me that many students do not take much math in high school. So we're testing kids, 17-year-olds -- seniors, basically, in high school -- often we're testing kids that haven't had any math for two years. A little over 60 percent of high school students in the United States are taking less than three years of math. So that's another reason why we wouldn't expect the math scores to look very good when we test these kids when they're 17 or in 12th grade.

Still, if you look at kids who are taking advanced academic math, that percent has more than doubled. For example, kids that took pre-calculus and analysis actually tripled from five percent to 15 percent between '82 and '98. But again, this is a very small fraction of the kids who are taking math in high school. So, we're seeing at the top some fairly rapid increases in kids taking advanced math, but again, rapid increases but starting from a very small base.

We also note that there is wider variation in student performance as kids get older. And while the overall summary statistics that are presented in the report are very informative, I think more informative information would be information looking at how different populations of students are performing. Are we in fact seeing widened discrepancies with trends over time? Are we seeing the bottom getting worse while the top is in fact getting better. We miss that when we just look at this summary, overall statistics.

Also, if there's greater variation in school quality in higher grades than there is in lower grades, we would expect any lump sum gain to account for a smaller fraction of the variance. This is getting a little bit statistical here, but the idea is simply that if there's more variation in quality at the 12th grade than there is in elementary schools -- which I suspect there is -- then similar gains at the 12th grade are going to show up as smaller standard deviations there, which is another thing to think about when we're thinking about the statistics here.

The issue of tracking versus non-tracking was an issue that came up on the report, which Tom has done quite a bit of work on himself but which he didn't talk much about now. He makes the point in the report that tracking cannot be an explanation for lower performance in higher grades partly because if you look at European and Asian countries, many of them track more severely than the United States does. And I would respond to this by saying the issue perhaps is not tracking or non-tracking, but what goes on in the low track and what goes on in the upper track so that the lowest track classes in Asian countries or European countries may in fact be demanding much more in terms of mathematical achievement than the lowest track classes in the U.S. So, it's not tracking versus non-tracking. It's actually the substance of what's going on that I think is the real issue. And an observation is made in the report that 8th grade math books in Japan don't include arithmetic, while U.S. books do. I think this may be a little bit of evidence that suggests that what goes on in different tracks, in lower tracks in other countries may in fact be a higher level of performance than what goes on in lower tracks here.

A caveat in all this is that we do not have longitudinal data. What we have with NAEP is we have cross-sections of the same grades over time. So, we're looking at different kids at each one of those slices. Now, the reason this is important is if the population of kids being tested is different at different points in time, and again the report does suggest that while drop-outs -- it's very difficult to get good measures of drop-outs, but if some people do think that drop-outs have decreased, drop-out rates have decreased in the United States -- if that's the case, then the kids who are in the 12th grade, the 17-year-olds, could be -- especially at the bottom -- different kids in 1990 than they were in 1972. Similarly, I think if you look at rates of immigration, that there are likely the pool of kids in the 12th grade today is very different than the pool of kids was 20 years ago. And we know very little what to do about immigrant kids coming with limited English in secondary school. I think it's been an area that's been terribly neglected.

Now, the report also looks as state tests, which corroborate in many ways the national results. An intriguing finding that came out in the report is that states that used their own tests, used custom-made tests, are showing bigger gains than states that are using the off-the-shelf national test. I think it's an intriguing point to be made, and for us to look at in the future. However, we have to be cautious here also because we know that when new tests come in, the second time they're given we see a bump. We see a bump because teachers get used to what's being tested. Many of these custom tests are new tests, and this could be -- what we're picking up here could be the initial bump as to real learning gains. You know, but again, it's an intriguing finding to be looking at.

I want to talk also a little bit about what some people call "mother NAEP" and "baby NAEP" -- with mother NAEP being the trend test and baby NAEP being the test that varies given what we're teaching in the country.

Tom analyzed the performance on particular math topics over time, which was an interesting analysis to do. And I thought there was a lot of good news in what he found. He found the greatest gains in geometry, problem solving -- topics that are emphasized in the new math curriculum. And whether or not you agree that these are the topics that should be emphasized, I think the good news is that we can focus student learning -- that when we focus the curriculum on certain topics, indeed, achievement increases in those topics. That, to me, is very good news.

The findings also suggested that the NCTM topics -- which are the ones that were being tested in the main NAEP -- were well targeted, that they were the areas in which there was, in fact, the lowest performance historically. So, they're fairly well targeted topics.

The bad news, according to Tom, is that basic skills are relatively low in many critical areas. And in some areas, like fractions, they're actually going down. And the report suggests that only -- I think it's states directly -- only if you master these basic skills can you go on to learn algebra.

Well, I want to tell you a little personal anecdote about that. My son, when he was in the 8th grade, I took him to take the SSAT, and so we come screeching into the parking lot and behold, we're early, which is unusual for our family, but we were a little early. So, I pulled out -- last minute, of course, probably the worse thing you can do -- the instruction manual. I said, "Let's go over a few of these items." So, we started going -- the worst thing you can do, right? Mea culpa, Mea culpa. So, we started going over a few of these items, and I asked him something, and it was clear -- he was in the 8th grade, he didn't know his times table. I went to St. Joseph's Grammar School, and my times table was carved into my soul before fourth grade. I could not believe this 8th grade, good student boy, did not know his times table. I said, "What's nine times seven?" And he couldn't get it. He said, "But I can figure it out," he said, "because I know three times seven. Three times seven is 21, three times seven is 21, three times seven is 21, if I add up those 21s, it's 63." I said, "Well that's great. Now what are you going to do on this timed test?" You know, get the kid really anxious before the test, but I was anxious at that point. I couldn't believe it.

He went in and he did all right. But I called him last night. He's now a junior in college. He's gone through advanced calculus, done math in college, deans list all the way through. I called him up last night about 9:00, and he picks up the phone in his dorm. "Yo." I said, "Jeff," I said, "What's nine times seven?" And he said to me, "64 or right around there." This is a true story. I told him I was going to tell this anecdote, and he said, "Ohhh."

But, you know, at least with the case of one, I think the proof is in the pudding that there are a lot of basic skills that may be -- some may be more necessary than others, but I think the reasoning part is definitely important.

Just one brief comment on the blue ribbon schools. I would be very interested, if you're going to, Tom, be looking at these blue ribbon schools any more, or if anyone else is, to not so much be looking at level of achievement but be looking at gained scores or value-added, because we may see a different picture. Some of these schools, even though their level may be low, may have made great progress over a short period.

Chester Finn: Well, it's the day after Labor Day, so somebody ought to say "Welcome back to school," except that it appears a lot of American schools have been in session for several weeks now. Others are opening today, and still others are worrying about whether they're going to be on strike tomorrow. I'm generally a little suspicious of documents that proclaim themselves to be the first annual -- as this one does -- but if Brookings and Tom can maintain the quality of the report we're discussing today, it will be worth looking forward to the second annual and the third annual, and so forth.

I think the most important feature of this report is it's honest. It doesn't try to be more than it is, or spin things in any particular way. And there's an important, I think, and unfortunate bit of context of this that's worth nothing. The U.S. Department of Education puts out an annual so-called back-to-school press release that came out last week, prepared by the National Center for Education Statistics. And this used to be an extremely valuable source of basic factual projections for the new school year. In recent years, it's become a political instrument, hyping the administration's current policy enthusiasms -- this year it's school construction -- spinning the data, and generally dispensing policy propaganda. Much the same thing has happened to another once valuable NCES publication, this one a large annual volume called The Condition of Education. It too is now more policy argument than neural, factual analysis. We're drowning in hype and spin when it comes to education data, and it isn't just coming from the government.

The College Board is up to the same thing. They put out the newest SAT results the other day. And they did something I think is fairly squalid. In order to be able to claim that SAT math scores were at their highest point in 30 years -- which is true -- the College Board selected as its baseline 30 years ago, namely, 1970, conveniently concealing the fact that those 1970 numbers came after six years of very serious declines from the all-time on SAT scores of 1964. This enabled the College Board to pronounce good news. But if they had look at a 36-year trend line instead of a 30-year trend line, it would have looked pretty different.

So, whom can you trust these days for basic education facts, for clearly stated facts, the kind that let the chips fall where they may? It may turn out that the best answer is going to be Tom Loveless and his Brookings colleagues taking the place of the federal government, the College Board, and a few other organizations here. Well, I think it's a huge pity that the federal government has lost some of its trustworthiness in this area, and that organizations such as the College Board have become spin doctors for their own data. Let's be glad that the Brown Center is hurling itself into this breech.

What do we learn from this report that it's important to know? Well, there is a lot of important material here. I'm going to emphasize three findings that caught my eye, and I wrote these notes before I realized which ones Tom was going to emphasize this morning. They may turn out to be three of his four, but I'm going to emphasize them nevertheless.

First and most important, obviously, we learn a great deal about U.S. student achievement in math and a fair amount about reading. And what we learn is slightly encouraging, but hardly cause for breaking out the champagne. The gains in reading, as Tom showed, are tiny. Those in math are somewhat larger, but most of them fade away by 12th grade. And they're pretty flat in the 1990s, particularly if you look at the trend data, the 30-year trend data. If we had been doing okay back when NAEP started around 1970, we might indulge in a sip or two of champagne to celebrate the modest improvement that has occurred since then. But we were a nation at risk then, and we're not significantly better off today, despite the many reform efforts and all the money we've hurled at this huge national project.

Certainly the most vivid fact in the whole report -- Tom put it on the screen -- is the data about how long it would take us to catch up with Singapore and Japan and other countries at our present rates of improvement. I suspect that our present prosperity is cushioning us from full awareness of the implications of this gap. And I suspect further that our own immigration policies are actually helping us cope with this gap by importing people who are more adept at these things than the home-grown variety. But, as Bill Bennett asked in yesterday's Washington Post, why can't we do better with our own kids in our own schools, and on the cushioning effect of the current prosperity, you ought to also see a very thoughtful op ed in today's Wall Street Journal by Michael Milken, of all people, on the degree to which we are a little deceived by the current state of the economy as to how our own home-grown -- especially high school graduates -- are doing, even in economic terms.

I've got a -- what wants to be a very small criticism of Tom and his colleagues with respect to their sort of difficulty in finding satisfying explanations for the stagnation in our scores, and for especially what they call the "middle grades slump," the report is kind of inconclusive in that regard. They hunt around. They explore a number of hypotheses and theories and they don't sort of settle on any --

[TAPE CHANGE]

-- look closely on one of them, not even any two or three of them seem to be able to explain the results. So you don't claim that you found what went wrong or what isn't happening or what ought to be happening differently if you actually don't know. And if the available data don't give you an answer to that question, it's probably better not to claim too much. On the other hand, that does leave us fumbling, sort of wanting to know why, and what we can do -- typically sort of American kinds of reactions to these kinds of reports -- well, why aren't we doing better or what we can we do differently? And the report doesn't answer that question -- perhaps, as I say, because it can't.

The second important finding to me -- albeit this one's of greater interest to policy wonks and number crunchers is, of course, that NAEP, the National Assessment of Educational Progress, on which so much of our policy analysis in this field depends so heavily is far more complicated than most people suppose. NAEP afficionados, which is a very tiny, tiny population, have known for quite some time that there really are two NAEP's, but I've never seen it so clearly explained, and illustrated, and documented, as in this report. Be aware -- and I speak as a former member of the NAEP governing board, that this poses a real dilemma for NAEP policy too, also for Congress during the upcoming NAEP reauthorization. Are we more interested in long-term trend data, or are we more interested in keeping NAEP tests aligned with the latest shifts in curriculum and instructional practice in this country? It's really impossible to have it both ways -- impossible to have it both ways, unless in fact you continue to run to essentially separate, parallel and discrepant NAEP's, that throw off different results, and thus adding to the confusion. I think it's very important dilemma to weigh in terms of NAEP policy.

One more tiny dispute with Tom and his colleagues. Tom a couple of times used the word "skyrocketing" to describe gains in math on the main nape over the first six years of the 1990s. Folks, those are ten point gains on a 500 point scale that he is describing as skyrockets. And, if your skyrockets on the Fourth of July did that, all of your guests would be scorched by exploding fireworks because they would never even get into the air. Let's keep these gains in some perspective, please.

The third finding was depressing and striking because once upon a time I was responsible for that part of the education department that runs the Blue Ribbon Schools Program. And, in a nutshell Tom and Paul and the rest showed that having a blue ribbon on your school doesn't necessarily mean the kids at that school are achieving at high levels. And the big reason for this, I've concluded, thinking about this in context of a lot of other things that are going on, is that this program, like so many other things we do in education, relies on what amounts to a peer review process to identify and evaluate winners. Essentially it asks educators to look at what's going on in their educational institution and decide whether they approve of it. Peer review. And it turns out, once again, that doesn't necessarily correlate with student achievement or with student value-added, as Jane pointed out, student gained scores. I think this is not only a cautionary tale with implications of significance for the government's Blue Ribbon Schools Program, I actually think it has implications throughout the many parts of American education that depend on peer review processes rather than on objective indices of actual effectiveness.

I mean, consider for example the National Board for Professional Teaching Standards, which evaluates teachers according to a peer review process rather than objective indices of effectiveness. Consider for that matter the entire accreditation process as it applies to colleges and also to schools -- essentially a peer review process, that looks at what they do and what they claim they're doing, rather than whether it works or not.

All of these things, in my view, should shift to a value-added approach to our judging what is worthy of a blue ribbon, or higher pay or an accreditation. And we should quit with the old fashioned attempt to judge whether we like what we see when we look at the place or the person, regardless of how well it actually works.

Well, that's undoubtedly enough by way of openers to the discussion to follow. There's much, much more in this fine report. My congratulations to Brookings, to the Brown Center, to Tom Loveless and his colleagues above all for an honest product. May it be the first of many. And maybe that is occasion to break out a little champagne even at this early hour of the day and celebrate. Thank you very much.

T. Loveless: At this time we'll take your questions. We'll open up the floor to discussion. We're going to take our chairs up here. And we all can entertain questions.

Q: -- I'd really just like to throw out the question to Tom and the panelists, why, with all the caveats that have been about the gains in math, perhaps being reasonable but modest, why have we been making gains in math and not in reading?

J. Hannaway: Well, I think some of the credit can be given to the National Science Foundation, which has been very active over the last decade in instituting math reforms -- a lot of it done at the state level and also at the local level -- and funding a tremendous amount of professional development for teachers in math. So there has been I think a concerted focus on math, and I can't give you the dollars behind it, but there have been big dollars behind moving math reform in the U.S.

T. Loveless: I think there's just been more effort put into math reform, especially in the early grades. And I think the main reason, actually, is a cultural reason. The culture as a whole has declined in terms of its literacy over the last 20 or 30 years, and I think that's beginning to show up and really hamper little kids' progress in learning how to read. But there's been an emphasis on math. If you go back and read, for instance, the opening paragraphs of the "Nation at Risk," it talks a great deal about math and science. And I think there has been a real emphasis in our culture on improving math and science. And I think that, really, we've neglected reading somewhat.

Another question? Yes?

Q: -- second to immigration as a factor was alluded to. And I think in many regards we're importing excellence, perhaps is one characterization. There was a Rand study about two years ago, that characterized the potential effect of immigration on education in a relationship about 25 years hence. They characterize it as a bipolar outcome, as the consequences of our immigration policy. It is because we're getting East Asian immigrants and Central American immigrants who are at vastly different levels of educational preparation and performance. So, on that, is immigration actually a positive factor? It is perhaps affecting reading scores and the rest, bilingualism issues -- and can you really tell, are there any ways in the data to tease out homegrown consequences as opposed to immigration features, and will that be a focus?

T. Loveless: Checker, why don't you take that since that was your point.

C. Finn: Well, there certainly is a bipolar immigration pattern, with one group of immigrants coming in to program computers and another group coming in to clean houses and to bus tables, and harvest grapes and things, and there's some in between. The data don't lend themselves to this analysis, both because none of these tests that I'm aware of actually ask what country you were born in, and because the policies on including or excluding non-English speaking kids are so discrepant from test-to-test and state-to-state, and place-to-place, so you're not even sure how many of the immigrants are participating in the tests that these data really show the results from. NAEP has tried to standardize LEP participation rates, but this is an endless, endless problem in terms of testing policy.

Q: I want to pick up on a point that Jane Hannaway made about getting beyond this kind of analysis to value-added and incremental gains sort of things. My understanding is that -- I understand what Tom did as a sort of value-added analysis, because basically controlling for SES, you're controlling for the imports that are brought to the school. You're controlling for the school situation. And he's looking at whether or not the blue ribbon schools over-performed or under-performed their circumstances. And to me, if you get more out of the kids that you're given -- given their resources -- you're adding value.

T. Loveless: Yeah, there are really two approaches to value-added. One is to do the adjustment that we did, which is an SES adjustment to control for the kinds of neighborhoods that these schools are located in, and the demographics of the school. That's the kind that we did. And another is to actually estimate a gain score, where you have scores from one year and you have scores from the second year, and you can see actually how much kids are learning within those two years. That's the kind we didn't have the data for. Hopefully, as more and more state tests come on line, we'll be able to measure the gains that these schools make over time. But we did do the first adjustment.

J. Hannaway: And it is an additional analysis. When you put in the controls, you're basically controlling for initial conditions. You aren't controlling for the actual contribution -- you aren't estimating the actual contribution that that school made. You'd have to get the value-added score to do that.

T. Loveless: And we wish we could have conducted that analysis. That data just simply aren't available yet, but they will be as more state tests come on line.

J. Hannaway: If you look at states now like Florida, Texas, California, North Carolina -- a number of them now are testing almost every grade, every year. And under those conditions, we can get measures like this.

T. Loveless: Another question? Yes. Could you please wait for the microphone to reach you before posing the question? Thank you.

Q: In terms of the comparisons that you'd like to make that we could learn from -- I mean, you talk about tracking schools year to year -- I wonder what other types of comparisons would be worthwhile. For instance, comparing among the states. We're always to foreign countries, and it's very hard, even with the TIMS data, to say if we're really comparing apples and apples when we do that. But what about comparisons between North Dakota and Louisiana, Connecticut and New Jersey -- school systems that spend a lot and achieve very little; school systems that spend a little and achieve a lot; parochial schools versus public schools. For instance, in the case of tracking -- I'm sure you've looked at this -- the parochial schools use tracking but try to keep a higher standard at the lower tracks, or at least that's my understanding

But I'm curious, when you're looking ahead to what, in assessing what has happened in education, what are the comparisons that we'd like to be able to make? What are the ones that you think would be most telling about what we could do to improve schools?

T. Loveless: Well, actually, I think the most telling would be to get to a much lower level than you just mentioned of comparison. I think the most telling thing would be to compare classrooms and teachers, because that's actually where instruction occurs. And when the day arrives when we can do that, I think we're going to learn much more about why certain kids are learning and why others are not. Why is it that some teachers get tremendous gains out of children -- even kids who haven't learned very much in the past and can be successful, and why is it that others are unsuccessful. So, that would be the level of analysis. That's not going to come from a federal test. That will come from a local test, perhaps, but that would be the most useful data.

In terms of state comparisons, we have, of course, state NAEP tests, and all of the things that I mentioned about the main NAEP tests in mathematics is true for the state NAEP test because they use the same test as the main test. So, if my warnings about the inflated scores on the main test are true, then we're also getting inflated ratings on the states in mathematics.

C. Finn: We can think about this as one set of studies that are amenable to looking directly at policy differences such as state level analyses. And there's been a lot of that lately. Rand just did one a couple of weeks ago on states to see which ones showed the greatest gain, and I'm tempted to trace these back to policy differences.

And then there's the local analysis that Tom is suggesting, which is less amenable to policy shifts, though it is in the long run, but much more importantly tied to instructional practices, teacher competence, teacher knowledge, how a classroom is organized, things like that. And both are well worth doing, because both are obviously going to have some impact -- or both, I think are going to have some impact -- on student achievement.

J. Hannaway: And the study that Checker referred to, the Rand study, the Grissmer study, did in fact show state level policy effects in North Carolina and Texas. And they attribute it to the testing and accountability systems that have been put into place in those two states. So, the NAEP data are sensitive enough to be able to sort out some of these state level policy effects, but this is all sort of all new things on the block right now.

T. Loveless: Right. It's interesting, too, in the Grissmer study, they estimated the rate of gain, just like I did, using standard deviation units. And on that main NAEP, the rate of gain is four or five times faster than we've estimated on the trends. So, there's really a drastic difference between these two tests.

C. Finn: But not skyrocketing?

T. Loveless: Well, four or five times faster is pretty skyrocketing my book --

J. Hannaway: That's a take-off point. That's a take-off point.

T. Loveless: Another question? Here up front.

Q: I'm Bob Hershey. I'm a management consultant. I did a book called How to Think With Numbers, which is on the main NAEP kind of thing of analysis. And I think it's important --

T. Loveless: We can't hear you. You're going to have to speak into the mike.

Q: Okay. I think it's important that you brought out the difference between the main NAEP test, which is mostly analysis, and the trends test, which is mostly computation. Many of us would argue that the computation end of it is a bit less important than it used to be many years ago since everybody has a calculator. And, it's not quite as necessary to be able to multiply fractions with pencil and paper when you can convert them into decimal fractions and use the calculator. And I wondered if there is any effort to get the public to understand the difference between the two tests -- one is computation, on is analysis, and maybe start using the analysis test as the trend?

C. Finn: He should apply for a position as your son. I'm sorry, I just think that's not right. I think that the computation comes first, the analysis follows. You have to be able to do both. And far too easily are we slipping away from the ability to do computation using the very crutches you've described, like calculators and computers, leaving people like Jane's son lacking in multiplication tables and things like that. Now maybe you don't think multiplication tables are important. I think they're important, and that schools have to do them.

In any case, you can't go backwards on NAEP tests. If it wasn't given before 1990, you can't turn it into a trend line that goes back before 1990. You could going forward. You could use it for the next 20 years. Of course, it's very likely that at some point during the next 20 years, math instruction in this country will go through another earthquake, and what is currently called main NAEP will be perceived as obsolete in terms of the curriculum and instruction that it incorporates. And there will be agitation from somewhere to update and modernize NAEP in order to conform to the latest thinking about curriculum and instruction.

J. Hannaway: Let me add something else here. I agree that I think we slipped a little too far away from some of the basics in schools. I think that correction has already started to be put into place. But I think the really good news is that we can teach problem solving, and kids are in fact learning it. And we know how to teach the basic skills. That's the easy part. It's practice. It doesn't take a whole lot of time. It can be built into a classroom very, very easily.

T. Loveless: I disagree a little bit. We can teach problem solving to a proficiency level to 40 to 50 percent, which I don't think is really that acceptable. In terms of computation, I'm not saying -- and I'm not trying to argue, and I certainly don't argue in the report that that's all that kids should be doing is just turning them into human computers. That's really not the point. The point is that children really gain their conceptual knowledge of mathematics as they learn how to compute -- and not just whole number computation, but how to work with fractions.

There's a wonderful book about elementary teacher skills by Lipping Maw [ph]. It's called Teaching Mathematics by Elementary School Teachers, something like that -- it's a great book. And here's a problem that Lipping Maw [ph] gave to a group of Chinese elementary school teachers and a group of American elementary school teachers. She gave them the problem, if you have the math problem 1 3/4ths divided by -- make up a word problem that will teach that to children. Now that's a basic skill, and that's a computation problem, and it's something that's tapped on the long-term trend. All of the -- virtually all of the Chinese teachers made up a word problem, and they made up a good word problem. Almost none of the American teachers could come up with a word problem, and half of them came up with the wrong answer to the problem of just computing because they confused dividing by one-half and multiplying times one-half.

So this is a serious issue. And computation gets to the heart of having a conceptual understanding of mathematics.

Another question. In the back. The gentleman standing, Dave, but he doesn't have the mike. Dave? Oh, I'm sorry. Go ahead.

Q: Okay. Thanks. I wanted to ask Jane a question, and I'm sure I'd like Checker to chime in too. It's a threefold question. Jane, you talked about the state tests. You said the custom tests made greater gain. I assume those are some of the larger states. And would you characterize why you think that happened. Is this in part because standards are driving these tests in these states that have designed their tests according to their standards, and there is a correlation between instruction and all of that? You said that some states do that rather than the off-the-shelf. And how many states do that? I suspect there are just a few -- I'm not sure. But you also said the state tests corroborate, I assume some of the gains that the NAEP indicate. And Checker, I think, knows this well. Mark Musik [sp] at SRAB [ph] did a sort of a chart indicating how states, when they rely upon their own tests, tend to have very, very positive reviews of student achievement, but when it's juxtaposed with NAEP, state tests and state achievement doesn't look so good. And so I'm wondering what you mean by the corroboration. Is that clear questions?

J. Hannaway: Well, I think there were a couple of questions that you had in there. First, why do I think these states with the custom tests are making greater gains? I think the report does suggest that one is alignment, that what teachers are teaching is what in fact is being tested. I mean, these tests are, at least theoretically, developed to reflect the curriculum that's been put into place. So, I think that is one reason. The caution -- what I was putting out was just a caution -- the caution is that most of these tests are relatively new, these state tests, and we always see in testing regimes that in the first couple years of a new tests there are gains as teachers and students learn the formats of the tests and what to expect. So, the extent to which there's real learning going on and the extent to which it's learning how to take this particular type of test is something we can only look at over time.

C. Finn: I don't think Mark's famous study had much to do with whether it was a home-grown state test or an off-the-shelf state test. You can have a home-grown, a designed, custom-built, designed-to-order state test that's very easy or that has a low passing score. I mean, there's currently a lot of griping going on in New York that the passing score on the Regents has been set very low now that it's required that everybody has to pass it in order to get a diploma. That's the quintessential, all-time most famous homemade, customized, state-specific test. But it may have too low a passing score now. And that's I response to intra-state political imperatives. What Mark was doing was using NAEP as a kind of external audit to say if you look at state NAEP results without respect and you don't have any political obligations there, lo and behold, a far smaller fraction of kids are achieving, at least at the proficient level on the NAEP test than are passing their state's -- meeting their state standards according to the state test. This is why I think it's useful to have both, by the way -- both have a state test and have a kind of external audit.

Now, that's led to another kind of backlash where people are claiming the NAEP standards are too high and really ought to be dumbed down so that more kids get proficient. And that's another whole topic which, fortunately, we haven't gotten into yet today.

T. Loveless: And I want to add too, that particular section of the report is one of the sections that use a lot of the weasel words that Checker was referring to. I qualify that finding up one side and down the other because the sample is very small and it only shows up in reading and it doesn't show up in math. The custom and the off-the-shelf tests have very similar kinds of scores in terms of mathematics, so it's just in reading where that occurs, and the samples are very small in states. So this is something that we're going to have to monitor and watch.

Another question? Yes.

Q: Can I go back to the first question that was asked, on the difference between the lack of progress in reading and the progress in math? I think many of us were expecting -- maybe there's credit due to the National Science Foundation -- but for somebody to say that there had been a dumbing down of the reading or literature curriculum that was particularly affecting SAT scores, that the top end of our students were reading Lord of the Rings instead of Shakespeare or whatever. I take it by the fact that nobody said that that's no longer considered to be a factor?

T. Loveless: We just don't have a good -- we don't have good data on that. We just don't have good national data on what was the curriculum in high school and in middle school 30 years ago and what is it today in terms of the requirements that are placed on children. So, that's the main reason why we don't know that much about it.

C. Finn: If you spend no time reading, it doesn't much matter whether what you're not reading is Shakespeare or Tolkein. I agree with Tom's earlier point about the cultural explanation. Kids are not picking up books in their spare time, and they're not watching their parents pick up books in their spare time. Between TV and computer games, and part-time jobs and all the rest, we are becoming a less bookish culture. And it's bound to affect -- I mean, look at -- I don't know about the Philadelphia Inquirer, but most newspapers are losing circulation because people aren't reading. And this is bound to affect kids' scores on reading tests -- it's just bound to.

T. Loveless: The way our teens use their time is quite different than teens around the world -- that is another issue. We have a tremendous number of teenagers who have part-time jobs. Our teenagers spend their time outside of school really on three activities: part-time jobs, hanging out with their friends, and extracurricular activities, mainly sports. And the amount of time is enormous. And as you ratchet up and increase the amount of time that kids devote to those three things, their test scores fall, and they fall precipitously. Most countries in the world take the attitude that for teenagers, they already have a job. It's called going to school, learning as much as you can -- that's your job. But in the United States, we have, around a half, I believe, of teens work 12 to 15 hours per week. So that's going to take a toll.

Q: I guess I'm a little surprised that, in terms of this discussion of reading versus math, that none of you have said anything about the reading wars versus the math wars. Yuri Treisman [ph], at a meeting here last spring talked about how in Texas, which has shown significant gains in math and not in reading, the math curriculum hadn't been really subject to math wars, and the reading curriculum had been. Do you think that's been a factor in this as well?

T. Loveless: I don't know, to be honest with you. The reading wars were settled earlier. The math wars are still, for the most part, raging. And there really hasn't been a settlement yet in the math wars, with the NCTM reforms and the believers in the NCTM reforms on one side -- and by the way, the National Science Foundation happens to be on that side; I happen to think that's the wrong side, but that's where they are -- and then on the other, people who believe that kids should be mastering arithmetic and essential mathematics so they can go on to abstract math. That war is still being fought. For the most part, most states in their state curriculum -- to the extent that that influences schools varies -- have rejected whole language and have endorsed some form of phonics instruction for younger children. But that should start paying off then with reading gains. And so far, at least, we haven't seen any big leaps forward in reading.

C. Finn: I'm not sure the reading wars are settled within the classroom. I think they're settled at the National Academy of Sciences and the NIH and in the books on the subject. It's not clear that that settlement, that that armistice, has made its way into the typical first grade classroom at all.

T. Loveless: I think that's right. Yes.

Q: The ultimate test, of course, is life. I wondered whether there's any data to correlate test scores and how people do in life, whether this correlation has changed over time, and does it compare to a correlation in other countries?

T. Loveless: I have no idea.

C. Finn: Well, of course, it depends on your concept of doing well in life. The -- some people would call that time at the beach, and other people would call it earning more money from your three jobs. The -- as Milken recycles in this big op ed in today's Wall Street Journal, we do know that people with more education in this society earn more, and we know that the gap is widening between those with more education and those with less education -- in terms of college degrees, for example, versus high school diploma , less than high school diploma. But I don't know anything about test score correlations with those kinds of life indicators.

T. Loveless: There was a wonderful little book -- it was published I think in the early '70s or the late '60s -- called -- do you remember that? And it actually looked at that question, how -- trying to correlate scores with various things such as divorce rates, levels of alcoholism, this kind of thing. And in almost all of those indicators, basic happiness, how satisfied are you, the people who knew more were happier and better adjusted. But I haven't seen any update of that study. So, if there are ambitious people out there who'd like to update that, it would be good to do that.

Q: I was wondering, does anyone remember Goals 2000? And do you think this report helps explain the embarrassing silence of Democrats and Republicans alike, particularly then-President Bush and then-Governor Clinton, who seemed to take a leading role in that? And another thing I'm wondering about -- I was very disturbed that this discrepancy between the blue ribbon schools and the achievements that you're citing -- would anyone have any other explanation except that the people at the schools that are achieving these -- not achieving, but receiving these awards -- is there any other explanation except that they must have some very good person submitting the application, that's very good at salesmanship, very good a public relations, very good at spin?

T. Loveless: Well, I don't think we should put the fault on the schools. And if you read the report, I really tried to avoid doing that. The fault is in the program itself. The program emphasizes what schools do, and it says here's a whole bunch of things we think are best practices. Now, tell us you're doing those things. And sure enough, there are a lot of schools out there that, you know, can write 50 pages, yes, we're doing those things. And that's what the award is being given for. So, you're right, a lot of them, especially these low-scoring schools in wealthy communities, probably have learned how to gain grant applications and applications like this. But the main story here is that we have a program that doesn't focus on student achievement. Student achievement is in the program, but it's just one of many things that that program is looking for.

Yes, Richard.

Q: You haven't mentioned the racial test score gap, which for the past 10 years has been roughly stable and maybe even widening a bit. If you look at the SAT and ACT scores, ACT nicely broke it down into so-called core curriculum, and minorities have been making sizable gains in catching up the core curriculum, and yet certainly in math the gap even widened. Any thoughts here?

C. Finn: Yeah, the ACT, the SAT and the NAEP trend data shows widening -- at least black-white gap during the '90s. And I don't know what, if anything the main NAEP shows in terms of race differences over that six-year period.

T. Loveless: I don't know about the main NAEP, I just know about the trend data, and it does show a widening. The black-white test score gap reached its narrowest point in 1988 and it's been widening ever since. And one of the most troubling things about it is that it appears to be driven, this widening, by a decline in test scores of black children whose parents went to college. It appears that well-off African-Americans, for some reason, their scores have been declining throughout the '90s on the trend data.

C. Finn: I know Tom doesn't want to turn this into a discussion of school choice, but I will note that we have a recent Paul Peterson study of vouchers in three cities that suggest it's possible to boost black test scores -- though not white test scores -- by sending them to private schools. So, we might be on the verge of a possible solution to the gap. But I don't think we're here to -- I don't know how to explain the widening gap. I didn't see very many politicians rushing to claim credit for it, whereas every time there has been any kind of an upward flip on any NAEP results in the 1990s the -- you could barely get to the mike, there were so many people seeking to claim credit for the upward flip.

J. Hannaway: It's a real puzzle. I've done quite a bit of analysis of SAT scores, and it is a very difficult puzzle to try to sort it out. And, you know, family background characteristics don't explain scores for blacks the same way they do for the whites, and we just haven't sorted it out yet. But I think it's a tremendously important problem and one that we really should be focusing more directly on.

T. Loveless: Another question? Yes.

Q: As I've listened to the report and the questions, the array of questions, the word keeps hammering in my brain about the interrelationships of everything we're saying here -- the curriculum, the training of teachers, the schools, black, white, poor, rich, whatever. But, I'm, I guess the Brown Center, then, has to lead some kind of other activity that's going to bring these pieces together -- immigration, other nations, their advancements, let's put it that way. What are they doing that we need to do? We need to look at our educational system and see where it's failing us, because it is. And it certainly doesn't start from the top with state policies and everything else being filtered down. It's got to start down at the bottom.

And I didn't stand up here to speak either, but when do we get to that point, where we're going to do something about it in an interrelated fashion?

T. Loveless: Well, I appreciate your comment. One of the things -- and Ron Nessen mentioned it in the introduction -- I'm leaving on tour this afternoon and we're going to New York and around the country. We've commissioned papers in three cities to take a local view of achievement -- one in Chicago, Los Angeles, and in Houston, Texas. In Los Angeles, David Klein is a mathematician who wrote a paper about three schools in extremely low-income neighborhoods -- these are predominantly Latino schools, they are 80 to 90 percent of the children are on free and reduced lunch -- and these schools have extraordinary math scores. They are success stories. Two of them are in Inglewood, California, which is right by the airport there, a very poor neighborhood. And what these schools are doing -- and by the way, none of these schools have won a blue ribbon award and they wouldn't possibly be able to win a blue ribbon award because what they're doing is, when California, for instance, adopted whole language, these principals did not go along with that and they taught phonics. When California went overboard with reformed math, these schools did not go along with that. The kids do not use manipulatives in their math instruction. They use textbooks. They don't have student-centered learning. They have teachers standing up and giving instruction to kids. And these are extraordinarily successful schools, and they've gotten kids up to levels to the envy of many suburban schools. So you don't need to adjust their scores. They have great test scores, and they're doing a very good job.

So that's some of the stuff that we're trying to do in the Brown Center to put a spotlight on successful practices and how these things work together.

Another question? Yes.

Q: Thank you. My children went to primary school in public schools in Switzerland and in Germany, and then we came back to California, and it was like two extreme ends of the spectrum. I think you cannot -- I'm so glad that you talked about some of the cultural issues, but there are some deep ones. Children working is a big factor, and I think there's a couple of big reasons for that. Here in the States we have three months summer vacation. You don't find that anywhere else in Europe. You also have -- and part of that, you know, is kids drive here at 16, and it's extremely important for them to have cars, so some of that money is going to that.

But I think the biggest factor is the cost of college education. We talk and talk and talk about how important it is, most of the achievement awards are -- or most of the achievement goal is to get kids into college, and yet college is so outrageously expensive here compared to any other country in the world. And Switzerland, which is not a socialized system, and in Germany also, have free college education for the kids who make it into college. Those who don't get into fabulous kinds of training programs, academies. I think that is a huge factor in our culture, the stress of money, of trying to save money from the time your kid's an infant. And teenagers in America are well aware of the cost of their college education coming up. I think you cannot discount that tremendous stress for kids. You know, we can say oh, they're just working for CD's, but they're not. Kids know how important college education is. It's a tremendous financial burden they're facing that they don't face in other cultures. You add to that our high divorce rate, where kids are not going to get support, as they should, from their families. I mean, I think the cultural thing cannot be expressed enough.

And I would like to see the Brown Center look at that -- the cost of college education, how that's affecting the stress, and eventually achievement levels. Is there any work being done on that?

T. Loveless: No, and it would probably be done by someone else, because we focus on elementary and secondary education. But, to the extent that it does affect secondary education, that certainly would be an important topic, and we'll keep that in mind. Thank you. Other questions? Yes.

Q: I want to go back to math, my favorite topic. You showed an interesting graph at the beginning of your talk about how it would take 85 years to catch up to Japan, or 135 years to catch up to Singapore. And then in your report on page 19 you say the mastery of arithmetic is non-negotiable. I mean, I think we all agree that basics come first and then you can move on to the fundamentals of algebra and geometry. And those countries that you showed on your screen tend to teach those fundamentals in the middle grades. And then in your report it seems like you are saying that a more sensible goal for U.S. students is to master arithmetic by the end of 8th grade. And I just wondered if you could clarify -- I don't think you're really espousing taking a step back and not pushing geometry and algebra, but I just want to hear that.

T. Loveless: No. Absolutely. I'm not pushing that. I'm not pushing a step back. And the sentence actually says, "if not before." The point is all kids should be able to master basic, essential mathematics. And I'm using arithmetic as a catch all phrase -- not just whole number arithmetic, but fractions, decimals, percents, integers -- all kids should be able to master that material by the end of the 8th grade so then they're prepared to move on into higher level math. A lot of kids will master that way before 8th grade. But the -- one of the issues that I do bring up that we did not discuss today is calculators in the classroom. And take a look at some of that data because those high-scoring Asian countries, for instance, do not use calculators in the early grades, and they do press the kind of rigorous mastery of basic arithmetic that I've called for elsewhere in the report.

Well, thank you all for coming. You've been a great audience. And we hope you come next year, same place. And let me thank Checker and Jane for coming today too. Thanks.

R. Nessen: Just one more reminder, you can see this again if you want to on the Brookings website and get a lot more information about education policy issues on the Brookings website -- www.brookings.edu. We have two more National Issues Forums remaining in the Priorities 2000 series -- on October 10th, on keeping the good economy going, and on October 18th on foreign and military policy.

Thank you all for coming.

[END OF EVENT]