Let Us Rank Journalism Programs

The Downside

A fair assessment of rankings, though, must acknowledge the downsides. Critics argue that rankings are often arbitrary and misleading. Detractors attack the lists for falling prey to misreporting by schools. They argue that institutions, desperate to boost their standings, manipulate data with such techniques as boosting application rates only to reject most applicants to appear more selective. ⁵ Moreover, some suggest it is a fool’s errand to try to compare widely dissimilar schools—the tony liberal arts college with a half-dozen students per class is chalk; the big state institution with a broad mission and big classes is cheese. ⁶

The backlash against ratings hit a high-water mark in the mid- and late-1990s but continues sporadically. Students from about forty colleges and universities, led by students at Stanford University, banded together in the mid-1990s into the “Forget U.S. News Coalition” ⁷ to attack the magazine’s popular rankings. Stanford’s president at the time, Gerhard Casper, backed their efforts, writing the magazine in the fall of 1996 to say he was “extremely skeptical that the quality of a university—any more than the quality of a magazine—can be measured statistically.” ⁸ Later, in the spring of 1997, Stanford created a data site that it billed as an alternative to the U.S. News & World Report survey, and Casper urged the magazine to change its ways, saying it should “eliminate its attempt to rank colleges and universities like automobiles or toasters.” ⁹

Stanford withheld some information from the magazine, declining to answer questions regarding reputation. But it continued to provide objective statistical information and has worked with the magazine and other organizations to find common ground. It has cooperated with U.S. News & World Report and other publishers, such as the Peterson’s, to provide information about the school in a standardized format that would allow prospective students to make their own comparisons. It relies on the Common Data Set (CDS) to provide a wealth of detail on admissions statistics, class sizes, racial and ethnic profiles of the student body, and so on. Stanford officials on their website say their aim is to “improve the quality and accuracy of information provided to all involved in a student’s transition into higher education, as well as to reduce the reporting burden on data providers.” ¹⁰

Some schools, nonetheless, see no common ground and refuse to take part in what they deride as “the rankings game.” Administrators at Reed College over the years have been especially critical and on the school’s website explain why they have refused, since 1995, to provide information for U.S. News & World Report rankings. They hold that the magazine’s methodology is “hopelessly flawed,” contending that intellectual engagement in the classroom is something the publication “does not and cannot measure.” They do provide information, however, to college guides such as Barron’s, the Fiske Guide to Colleges, Peterson’s, and Colleges That Change Lives. Unlike the rankings, such guides provide helpful descriptions of Reed’s student culture and academic environment, Reed officials say on the site. ¹¹

St. John’s College, similarly, has long balked at taking part in rankings. On its website, the school says the diversity and distinctiveness of colleges “tend to be lost in a scale of ‘best-good-worse.’” While officials there concede that such information as class size and programs offered can be useful, they hold that the statistics used in rankings don’t offer such information. “How can the interaction between faculty and students be quantified? What kind of numbers tell you about the interests students discover as they explore new ideas and participate in scholastic and extracurricular programs?” school officials ask on a page devoted to explaining their rationale. “Do statistics reflect the skills in thinking, writing and analysis that students develop during the course of a well-designed and cohesive program of study?” ¹²

St. John’s does take part, however, in an initiative to provide data to parents and prospective students. Coordinated by the National Association of Independent Colleges and Universities, the University & College Accountability Network (U-CAN) provides a common template schools can use to volunteer data on the makeup of the student body, tuition, financial aid, majors, and so on. This objective information, St. John’s officials have said, is an alternative to rankings that they say are inherently biased. When the U-CAN initiative was launched in 2007, President Christopher Nelson touted it as “something better—factual, useful, and transparent.” He contended that the U.S. News & World Report survey and its “single-scale of ranking” implied “that there is just one standard for a good education.” ¹³

Critics have long inveighed against comparative lists. The Atlantic, commenting on the release in September 2013 of the U.S. News & World Report annual list, headlined its piece “Your Annual Reminder to Ignore the U.S. News & World Report College Rankings.” After calling the “predictable round of excoriating assessments from journalists, college officials, and others” this year’s “bray-a-thon,” author John Tierney listed what he said was a small sample of the best critiques of the past fifteen years. ¹⁴ Among them were an essay by Nicholas Thompson in The Washington Monthly in September 2000, ¹⁵ a report by the University of Florida’s Center for Measuring University Performance in 2002, ¹⁶ an essay by Reed College President Colin Diver in 2005, ¹⁷ a report by the Center for College Affordability and Productivity (CCAP) in March 2009, ¹⁸ and an essay by Malcolm Gladwell in The New Yorker in February 2011. ¹⁹

Tierney admitted that the U.S. News & World Report ranking “provides a rough guide to the institutional landscape of American higher education,” suggesting it is helpful to people who know little about America’s colleges and universities. But he held that using them for “any more exacting purpose is about as good for you as eating potato chips and Gummy Bears for dinner. With maple syrup.” While criticizing U.S. News & World Report, furthermore, Tierney did defend as “far more nourishing and beneficial” rankings by The Washington Monthly. That magazine rates schools based on their contribution to the public good in three categories: Social Mobility (recruiting and graduating low-income students), Research (producing cutting-edge scholarship and PhDs), and Service (encouraging students to give something back to their country). ²⁰

Beyond assailing efforts to rank universities overall, critics fulminate about professional school lists. They fault the weights raters give to different variables as arbitrary and hold that many important variables are left out. They bemoan efforts by schools to game the lists by, for example, urging students and alumni to respond only positively to survey requests or, worse, by submitting false data. Tulane officials were embarrassed by recent revelations that information about their business school was misreported to U.S. News & World Report for years. Misreporting of information about undergraduate schools similarly turned up at Claremont McKenna College, Emory University, and George Washington University. ²¹

One common criticism is that some raters measure a host of variables that together do no more than provide a false sense of rigor and thoroughness. “The rankings of American medical schools published annually by U.S. News & World Report have the appearance of objectivity and scientific integrity yet fall short of both goals on methodologic [sic] and conceptual grounds,” thundered William C. McGaghie and Jason A. Thompson in a 2001 critique in Academic Medicine. ²² Still other critics hold that magazines that rate schools tweak their measures routinely to produce different rankings each year or so when the quality of the schools, in fact, changes little from year to year. Such critics suggest raters do so simply to sell magazines rather than improve the quality of the rankings.

Moreover, critics suggest professional schools waste time and money trying to boost themselves a few notches in the lists. “U.S. business schools are locked in a dysfunctional competition for media rankings that diverts resources from long-term knowledge creation, which earned them global pre-eminence, into short-term strategies aimed at improving their rankings,” complained Harry DeAngelo and Linda DeAngelo of the Marshall School of Business at the University of Southern California and Jerold L. Zimmerman of the University of Rochester’s William E. Simon Graduate School of Business Administration. ²³ And still other critics fault schools for attempting to buff up their images with marketing materials aimed at influencing peer ratings, ²⁴ which some say are arbitrary in any case.

Journalism programs, whether graduate or undergraduate, have been spared the heat in this debate because rankers pay them little mind. Undergraduate programs are mentioned by raters such as Frederick E. Rugg, author of Rugg’s Recommendations on the Colleges, who groups journalism programs into three broad categories based on where he lists their parent schools or universities: most selective, very selective, and selective. But there is no ranking. As for graduate schools, the professional school rankings assembled by U.S. News & World Report, Bloomberg Businessweek, Forbes, and The Wall Street Journal bypass journalism. TVWeek.com’s NewsPro magazine in 2013 surveyed Radio Television Digital News Association members to produce a top-20 school ranking, but it’s purely reputational and skewed toward electronic media.

U.S. News & World Report ranked some graduate journalism schools once in 1996. ²⁵ It simply surveyed journalism practitioners and academics to develop its list, rather than applying a broad array of criteria as it does with other disciplines. For business schools, for instance, the magazine surveys deans and directors and corporate recruiters and also uses information such as job-placement success, mean starting salaries, student selectivity, mean GMAT and GRE scores, undergraduate GPAs and acceptance rate. It assigns different weights to each of the many factors it measures.

The Challenge

But the magazine has not targeted journalism. Robert J. Morse, director of data research for U.S. News & World Report, in an October 2013 interview with me pointed to difficulties in gathering data on which to base a ranking. While law and business schools report detailed data to accrediting bodies or the Graduate Management Admission Council, administrator of the GMAT, Morse, said such standardized reporting does not exist for journalism schools. Furthermore, while placement offices play a large role in business schools, they are not as prominent or do not exist in many journalism schools. So information on where students work after graduating and how much they earn is more difficult to come by. “There don’t seem to be standards of reporting at journalism schools,” Morse said.

The absence of rankings on either the graduate or undergraduate level does not mean there is not demand for such information, however. Daniel Reimold, an assistant professor of journalism at Saint Joseph’s University in Philadelphia, publishes a blog, College Media Matters, on which he runs a list, updated annually, of the fifty undergraduate journalism schools and programs in the United States that he considers to be the best, the “places I would strongly consider enrolling if I woke up tomorrow back in high school.” ²⁶ He assembled the list, he said, in part because many readers of his blog often asked for advice on which schools to choose. His list, he told me in an October 2013 interview, gets “amazing amounts of traffic.” He attributed some 40 percent of the 140,000 hits his blog gets each month to his most recent and past lists.

Reimold acknowledges shortcomings to his top fifty list. It is “not meant to be all-inclusive or objective,” he said. He said it is based on nothing more than his personal knowledge of the reputations of various schools and departments, faculty, associated student media, classes, internship, and study abroad connections and feedback he has gotten from students and some alumni. He added, “It is strongly biased in favor of programs exciting me in the digital journalism realm and in some way aligned with quality campus media and professional publishing opportunities.” ²⁷

Reimold does apply some criteria, though. Nearly all the programs on his list are accredited by the Accrediting Council on Education in Journalism and Mass Communications (ACEJMC) or are housed within an accredited school or college. And the institution must feature at least a journalism major. He also focuses solely on undergraduate journalism programs, leaving out programs such as those at Columbia University, City University of New York (CUNY), the University of California, Berkeley, and Stanford. Separately, Reimold provides a comprehensive list on his site of U.S. colleges and universities that feature journalism schools, programs, majors, minors, divisions, and concentrations.

Reimold does not rank any of the programs on his blog, including the top fifty. Instead, he merely lists them alphabetically. He said he is not opposed to ranking programs but said he does not have enough data or the wherewithal to amass it for a ranking. Rankings, he argued in our exchange, should not be treated as “sacrosanct” but can be helpful to prospective students and their parents who wrestle with a “glut of information and yet no real starting point.”

Rankers face a high hurdle in gathering information for any such list. In its overall rankings, the CCAP works with Forbes magazine to rank 650 schools, culled from over one thousand that it examines. The center builds a score for each school based on these measures: student satisfaction, as reflected by student evaluations from RateMyProfessor.com, actual and predicted freshman-to-sophomore retention rates; post-graduate success, as measured by the salary of alumni tracked in the PayScale.com, listings of alumni in Who’s Who In America and on a Forbes/CCAP corporate officers list; student debt; the four-year graduation rate; and academic success, as measured by students winning competitive awards. Each category is assigned a different weight, though the first two—student satisfaction and post-graduate success—together account for 60 percent of the score (25 and 35 percent, respectively). ²⁸

The information that CCAP and Forbes gathers is publicly available and does not involve surveying employers, students, faculty, and administrators as other rankings do. Forbes’ ranking of graduate business schools, by contrast, relies on surveys of alumni and is based solely on the return on investment (ROI) achieved by graduates. Magazine staffers compare alumni earnings in their first five years out of business school to their opportunity cost of attending the school for two years to come up with the ROI measures. ²⁹

Increasingly, rankers are zeroing in on what the New York Times called “dollars-and-cents tabulations,” seeking to measure the return graduates get from their investment in the programs. ³⁰ The newspaper reported on October 27, 2013, that Queens College in New York ranked second in the 2013 Washington Monthly list of colleges that offered “the best bang for the buck.” With its annual tuition of $5,370, Queens stood out in meeting the magazine’s standard of doing the “best job of helping nonwealthy students attain marketable degrees at affordable prices.” Queens, part of the CUNY system, placed between Amherst at first on the list (despite its $63,000-plus all-in annual cost of attendance) and Bernard M. Baruch College, another unit of CUNY, at third. ³¹

If a ranker looked chiefly to a single factor, such as alumni salaries, the calculation would be simpler, of course. The challenge then is gathering accurate and broadly representative data. To gather its data, PayScale, Inc., surveys employees about their pay, but it leaves a lot of people out. It surveys only full-time employees, excluding self-employed people and anyone with a degree higher than a bachelor’s degree. Harvey Mudd College topped its 2013 list of “Colleges Worth Your Investment.” It cost $221,700 to attend Harvey Mudd for four years (as of 2012), but its bachelor’s degree-only graduates could expect to net $2.1 million over thirty years of working. (Queens College ranked 338th. Despite its low four-year cost, of $67,520, graduates could expect to earn just $664,000 over thirty years, PayScale reported.) ³²

ROI calculations like these could become more important, and soon. Federal policy makers plan to tie their new ratings to some $150 billion the government doles out annually in student financial aid. As of November 2013, no one knew how those ratings would be developed. The New York Times reported that Education Secretary Duncan does not plan to develop a first-place-to-last place system, but rather one with ratings that compares schools that are similar in their mission. ³³ In his September 20, 2013, TIME Summit address, he said he will include measures of average tuition, scholarships, and total debt and will weigh them against graduation and transfer rates, alumni satisfaction surveys, earnings graduate achieve, and the advanced degrees graduates go on to earn. ³⁴

For now, it is unclear how the Department of Education will draw a distinction between a “ratings” system, as Duncan calls it, and a ranking. Fearing the worst, however, some critics have already “gone on the attack” against the plan, the secretary noted. Appearing to sympathize with them, Duncan took shots at rankings. His department’s system, he noted, “won’t highlight trivial differences between elite institutions or heavily reward schools based the number of students an institution turns away.”

Despite the controversy over ranking, publishers certainly have found it lucrative to rank schools overall, as well as to single out graduate programs in business, law, and medicine. Students and their parents eagerly buy magazines that carry such lists and, perhaps more important, schools are willing to advertise in the publications that run them.

But journalism schools do not make the cut for the for-profit publishers. Richard K. Vedder, an economics professor at Ohio University who directs CCAP, speculated that the financial calculation is different for journalism programs. There may not be enough money involved, especially now when media are so hard-pressed, to make it worthwhile for publishers to develop rankings. Few journalism schools may advertise, especially undergraduate schools that are part of state universities. Publications that rank business schools, an especially competitive area, “reach an audience that can afford to pay a lot of money,” Vedder said in an October 2013 interview with me.

The Upside

Lack of interest among publishers, however, does not mean that a ranking of journalism programs would not be valuable. Comparing schools based on certain objective measures could help students evaluate programs. Moreover, a ranking would help faculty and administrators at schools see how their schools compare with others; it could provide barometers by which they could gauge themselves and help them set goals if changes are needed. Despite the flak that ratings draw, the demand for them suggests they are useful and government interest in the area suggests they are going to get still more attention.

The Association for Education in Journalism and Mass Communication (AEJMC) ought to get out ahead of the issue. Indeed, the AEJMC is well positioned to fill the gap left by publishers. Operating through a committee that represents a cross-section of journalism programs, the association could develop the criteria by which schools would be statistically compared. It could then team up with the accrediting body, the ACEJMC, to survey accredited schools, with the ACEJMC requiring that schools provide the needed information as a condition of accreditation (nonaccredited programs could take part voluntarily if they would like to appear in the ranking). Both associations could then publish the ranking periodically, perhaps every five years or so to make the lists meaningful (schools do not change all that much in a year, despite the impression that the many rankers leave by publishing their lists annually).

Deciding what the criteria are would not be simple, of course. Does one measure a journalism school by the number of awards students win in the Hearst Journalism Award Program, the National Student Advertising Competition or similar contests? Is it relevant how much faculty publish in academic journals or how much the professors earn? Are class sizes worth considering? Should schools survey graduates on their pay levels a year or five years out of school, or measure alumni satisfaction levels? Should the costs of attending, as well as the availability of scholarships, figure in? Are the numbers of faculty and students, broken down by area of study (print, broadcast, advertising, public relations, etc.) relevant? Would the number of faculty members with doctorates or long tenures in industry count?

Although difficult, such questions could be answered. And the measures could be weighted to make comparisons meaningful in a single overall list, a single first-place-to-last ranking, as well as in subsets that would zero in on issues such as ROI. Graduate programs should probably be measured differently from undergraduate programs. In all cases, though, administrators and faculty would find value in both the overall ranking and the subsets. And in all cases the criteria would have to be spelled out, so any reader could see precisely which grounds are being used for comparison.

Would such rankings be enough for a student to make a decision on enrolling? No. Nothing can replace campus visits, where students can see if they and the school are a good fit. No rankings could measure personal chemistry between a student and an institution. For faculty and administrators, too, the decision of whether to emphasize certain types of programs over others is a complex issue.

But could rankings offer a good start, a broad way to measure one’s programs against others? Unquestionably. The benchmarks they would provide would give students useful information as they begin their college or graduate school searches. They could equip the students to ask better questions on campus visits. For faculty and administrators, who often know little to nothing about schools in neighboring cities, much less distant states, the comparative data could be a potent tool. The information about the competitive landscape could help them improve their programs.

Once schools can see how they stack up against one another, schools might find it worthwhile in time to try to outdo a rival school. Such competition could be healthy. Too often, faculty at any given school now know little more about competing schools than what they can find on college websites or what they glean from AEJMC meetings. For everyone with an interest in the schools, rankings, together with the underlying data that go into them, would bring transparency to information that now is at best opaque. Bringing such transparency and openness is one of the core missions of journalism and it should be for journalism schools, as well.