Insight: New York Times Ranks Reed in Bottom 8th Percentile for “Economic Diversity” Among Selective U.S. Colleges

Why did the New York Times rank Reed #265 out of 286 for "economic diversity?"

In the latest version of The New York Times’ College Access Index — released in early September — Reed fell to position #265 among the country’s 286 “most-selective” colleges. The ranking, which placed Reed in the bottom 8th percentile of all selective US colleges for “economic diversity,” marked a 183-point drop from the college’s position at #82 in the Times’ previous version of the CAI, which was last updated in 2017 (although there are significant differences in methodology between the two versions).

The drop comes at a time when many private colleges in the U.S. are drawing closer to one another when it comes to admitting and supporting Pell students. As seen above, over the last ten years a significant number of colleges have increasingly clustered in the 10-20% band for enrolling such students, both dropping down from the 20-30% band and rising up from the 0-10% band. (For the more statistically minded: the interquartile range for private colleges has dropped by six points, from fifteen percentage points in 2011 to only nine in 2021.)

However, this is not the same as saying that, “most schools have seen their number of these students decrease,” as the Times stated. According to the Times’ own data, which Quest data reporters downloaded and analyzed, only 137 of the 286 colleges studied (48%) saw their percentage share of incoming Pell students decrease over the last decade, a minority of the dataset. At the same time, 126 schools (44%) actually increased the percentage of Pell students in their incoming classes, and 23 (8%) experienced no change.

If read literally, the Times’ introduction could be interpreted to imply that the raw number, instead of the percentage, of Pell students has decreased at most universities — although this data point is not included in the database published by the Times. The Quest reached out to the Times on Monday, October 9 to ask if this was the case, and if the Times might be willing to share the full data with student journalists, but did not receive a response in time for publication.

Unable to obtain the full data from the Times, the Quest set out to reverse engineer it from public data released by the National Center for Education Statistics, which the Times cited as their primary source. Using admissions data from the NCES Integrated Postsecondary Education Datasystem (IPEDS), reporters were able to successfully match and calculate raw numbers of Pell students for 215 of the 286 colleges studied (75%). Data for the remaining 71 colleges could not be calculated with any confidence in time for publication due to inconsistencies of spelling and naming conventions between the Times and federal databases, which make it difficult to accurately merge the two datasets.

Among those 215 colleges, 106 (49%) decreased their raw number of enrolled first-year Pell students between 2010 and 2020, while 109 (51%) saw an increase. It is, however, possible that the remaining 71 institutions for which the Quest was unable to calculate data would sway the balance toward an overall decrease in the raw number of Pell students.

Regardless of the statistic used, Reed falls near the bottom. In addition to being placed in the lowest 8th percentile for percentage of Pell students, the college has also decreased its share of first-year Pell recipients by seven percentage points, from 18% in 2010 to 11% in 2020.

This makes Reed the only Oregon college studied by the Times to have significantly decreased its share of Pell recipients over the last decade. The University of Oregon and Willamette University both increased their percentages of Pell-eligible students, while Lewis and Clark College dropped by only one percentage point over the ten years studied.

Editors' Note: An earlier version of this chart printed the arrows in the correct direction but the relevant years out of order. The Quest apologizes for the error and will print a correction in our next issue.

Meanwhile, the college placed in the 11th percentile nationally for decade-long change in the share of Pell-eligible first-year students, with 149 institutions (52%) experiencing either gains in this area or no change, and another 92 colleges (32%) suffering declines less significant than Reed’s (changes in Pell share between 0 and -7, exclusive.) Only 32 of the 286 colleges studied (11%) showed greater declines in the share of Pell-eligible first-years than Reed did over a ten-year period.

The Quest reached out to the admissions department for general comment on this data on Tuesday morning and received an acknowledgment on Tuesday afternoon from Dean of Admission Milyon Trulove, who said he was traveling for the college, making this “a quick turnaround,” and requesting a summary of the Quest’s findings. The Quest responded with a findings document within the hour but did not receive formal comment in time for publication on Wednesday night. 

Why this drop occurred is a significant but complex question that falls outside the scope of this story, and the Quest plans to investigate further in follow-up coverage.

However, it is important to remember that the Times based this year’s rankings exclusively on the percentage of Pell-eligible students in the first-year class — a significant change from the 2017 ranking process, which also incorporated the costs charged to low-income students.

While the percentage of Pell-eligible students at a college is an interesting way to assess that institution’s accessibility to low-income students, it is not, in the opinion of this writer, a compelling measure of “economic diversity.” The Times’ ranking model, for example, appears to simply order colleges in descending order of percentage of Pell recipients, which would imply that a college where 100% of the student body received the same Pell grants, and therefore had access to the same socioeconomic resources, would be considered maximally diverse. Such a model, which rewards homogeneity on one end of the curve, cannot be considered a useful measure of diversity.

To get a better measure of economic diversity, we’ll have to design a new model. The Quest picked the Simpson’s Diversity Index, an equation most often used in field biology, but which can also be applied to sociology (or journalism.) In essence, Simpon’s index measures the probability that any two randomly selected elements of a set, or members of a community, will be different in some chosen aspect. In this case, we’ll consider Pell status to be the aspect, and “Pell-Eligible Students” and “Non-Pell-Eligible Students” to be the two populations of reference.

The Quest implemented a procedure for calculating Simpson’s Diversity Index against Pell-eligibility data for US colleges in R — a procedure we’re making publicly available on GitHub alongside this story — and applied it to the Times data. Where the Times ranked colleges by how many low-income students they could admit, the Quest ranks them by the likelihood that any two students will have different Pell statuses — in essence, the likelihood that your class discussion partner comes from a different socioeconomic background than your own.

The results were surprising. Berea College, which the Times ranked #1 for highest economic diversity, fell to last place at position #233 under Simpson’s Index. (Only colleges for which the Quest was able to obtain 2020 enrollment data are included in the Simpson’s Rankings.) This is an exemplary case of how outcomes can be changed by different models — supporting students with “limited economic resources” is part of Berea’s mission statement, and 94% of their students are Pell-eligible. In a ranking system that rewards numbers, it rises to the top, but under Simpson’s, it performs no better than a school with only 6% Pell-eligible students would.

Our model instead placed Missouri’s College of the Ozarks in the top spot, with a more than 50% likelihood that any two randomly selected students will have different Pell eligibilities. It did not, however, significantly improve Reed’s standing — moving the college up only one grouping to the 9th percentile instead of the 8th (rank #212 out of #233.) Colleges like Reed, which have fewer Pell students overall, are, by definition, less likely to have Pell-eligible students in a randomly selected pairing, so the Simpson’s model only significantly changes the rankings toward the middle of the bell curve as institutions approach 50% Pell-eligibility.

Below, we provide a complete table of Quest rankings for the 286 colleges studied. Our methodology and the source code for our implementation of the Simpson’s Index are available on GitHub.

Insight is the Quest’s irregularly syndicated column of investigative reporting, big data analysis, and news commentary. Have a burning question about Reed, or the world around us? Send it our way at quest@reed.edu.