Given renewed interest in federal performance measures (ratings) for universities evidenced by last week’s announcements, this reflection of August 21, 2012 is worth a second look.  So are the Department of Education’s pronouncements:  http://www.ed.gov/news/press-releases/public-feedback-college-ratings-framework.

No two universities are the same. No university was the same last year as it will be next year.  Ditto for students.

“The only man I know who behaves sensibly is my tailor; he takes my measurements anew each time he sees me. The rest go on with their old measurements and expect me to fit them.” – George Bernard Shaw

____________________________________________________________________________

When US News & World Report publishes its annual college rankings, two things happen. First it is the most widely read issue of USNWR. Second, without fail, the editors tell the readers not to pay too much attention to the rankings because there are so many “intangibles” in student/university fit.  No matter how many data points there are, too few exist to answer every question for every student and each university.

Academics occasionally look askance at the USNWR College Rankings:   Institutional leaders are wary of the findings. Interestingly, a university that comes out near the top of any heap rarely hesitates to trumpet that on its homepage. What was previously considered with disdain becomes a point of pride.  Yet, it was reported by AP last week that Emory University has been cooking the books for a decade in an effort to increase its standing in the disdained ratings.

Meanwhile, state boards of higher education are clamoring for performance measures, so that they might appropriately direct resources to a state’s assemblage of tertiary institutions.  Sometimes boards use too few measures of “quality” assessment. Worse yet, university leaders follow boards in fear of budget implications.

The marketplace works. USNWR sells so many copies of the annual college rankings because reasonably smart people review these and find some value and consistency in them as one means to comparatively assess quality: But universities must display integrity in reporting data, otherwise no ranking of any kind has any value.

Diane Natalicio, president of the University of Texas at El Paso, asked the right question when she entitled a talk “Graduation Rates: Whose Success Do They Measure?” She correctly points out that 70% of UTEP’s graduates are not counted in UTEP’s graduation rates. Transfer students, part-time students, returning students, and those who enroll in the spring semester are all under the radar. Only students who are first-time freshmen, enrolled full-time, seeking a degree, and start in the fall are counted.

Many non-traditional students attend universities. A six-year graduation rate may be absolutely meaningless to a single mom or dad working full time and spending a decade earning a degree while supporting a family and holding down a full-time job.  “Late” graduation is not a failure, but a stunning attainment of a life-long ambition:  A success for university and student alike.

University graduation ceremonies on military bases are a sight to behold.  I have been there.  Not a single graduate in his or her 20’s, and by the clanging pots and pans and air horns, those in their 40’s and 50’s graduated with a tsunami of pride from spouses and children, coworkers and friends. A six-year graduation rate may have great value in measuring a university’s success in dealing with an 18-year-old who just graduated from high school.  Any university that tends to the needs of non-traditional students will likely be hurt in this graduation-rate derby.

Graduation rates as measures of success germinated in the NCAA as a means to monitor the academic performance of student athletes. This is most likely a valuable measure for student athletes who are almost always members of the group of students that we call first-time full-time freshmen. An intercollegiate athletics vehicle for measuring academic progress has leached into public policy.

When institutional leadership offers no alternate, more appropriate, measures for university performance, things sink to the lowest common denominator in the quality equation.

Performance expectations are not only legitimate but absolutely essential.  The quality and effectiveness of an institution’s success must be measured for audiences internal and external. Oversimplified, sound-bite driven assessment is just that.

Most confounding and disconcerting is that a measure such as six-year graduation rates  misses the changing demographics of college goers, and works on a goldfish-swallowing, raccoon- coat-wearing, notion of who goes to universities, and why.

A constant and deliberate “tailored” and evolving effort to collect data and look at trends grounded in a vision is required, not isolated snippets. Comparisons to peers from around the nation are absolutely appropriate, but peers must be true peers. A state land grant university, for example, should never be compared to a state teachers college on any dimension.

Performance measures respectful to the needs of our universities and their students are required.  However, tailored-fit is essential. All else matters little.