A rating of graduate programs across the nation is causing concern for University administrators who say that one of the rankings does not do justice to the school.
The rankings, published today in U.S. News and World Report, shows that in the engineering category, the Institute of Technology fell dramatically, dropping 17 places in one year. In this year’s issue, the school is ranked 29th, down from 12th last year.
“I’m troubled for IT,” said Mark Brenner, dean of the Graduate School. “The rankings do not reflect the quality of that program. IT did not fall apart this past year. If anything, the program has gotten stronger.”
H. Ted Davis, the dean of the institute, blames the problem on the data that was sent to the magazine, which he said was inaccurate.
“Something went very wrong with the numbers,” he said. “There’s no way you could drop (by that much) in one year. I think it is mistakes made on our part in getting them the correct data.”
Compounding the problem, he said, the incorrect data was in key categories that are heavily weighted. “So if you have poor data going into one of those categories then that’s going to hurt you big time, and that’s what happened to us.”
One of the categories that Davis identified was faculty resources. “An example of how bad some of the data were in that category was that the number of Ph.Ds per faculty were entered into the data as 1.03, and in reality it is 3.0,” he said.
Another category, he said, was engineering research funding, which the ranking showed as dropping almost $6 million. Actually, Davis said, the number remained the same at $46 million.
Davis said he has not had the opportunity to find the source of the misinformation. “We have not analyzed all the data that was sent in,” he said. “There are inaccuracies there that we have to trace backwards and find so we can prevent inaccuracies in the future.”
If correct data had been sent in, Davis said, the school would have seen very little change. As proof, he cited the school’s reputation among academics and practicing engineers, one of the five categories that determines the overall rating. The academic and professional reputation rating remained the same at 15th.
Mel Elfin, the editor of special projects for U.S. News and World Report, said although it is very rare, it is possible that the information sent to the magazine was incorrect. “The data that we get is self-reported,” he said. “Sometimes there is an individual who sends in the data that is not authorized to do so, or may not have the latest information. It is possible.”
However, Davis also questioned the value and legitimacy of U.S. News and World Report’s ratings. He pointed to the National Research Council Rating, published about every 10 years by the National Academy of Science and Engineering, which he said placed the school in the top 10.
The data in that report are “much more carefully compiled and thoughtfully and scientifically ranked,” he said. “The U.S. News and World Report’s is not a carefully done rating. There is very little quality control that goes into them.”
As an example of the unreliability of the rankings, Davis pointed to the school of education, which rose from 25th in 1995 to 7th in 1996.
“Now do you believe that they changed in quality from 25th to 7th in one year? These are the kinds of fluctuations that are possible in the U.S. News and World Report’s ratings.”
Davis said the school of education merited the better 1996 ranking, and that it was 1995’s poorer showing in the rating that was inaccurate.
However, Elfin said the rankings are very reliable indicators of the status of a school. Of the tens of thousands of rankings they do, “less then a handful” have complaints, he said.
“Our system is reliable, but it’s not perfect,” he said. “If you were to grade us on our performance we’d be well ahead of the curve.”
Brenner said he agrees with Davis’ evaluation, and attributes the other minor slippages in part to a lack of legislative support. “It reflects the difficulties we’ve had in state funding for the past several years,” he said.
The University, Brenner said, will have to work to change to ensure that the inaccurate engineering rankings do not hurt the school. “It’s going to be very important for us to spend an extra effort to present IT as the quality institution that it really is,” he said.