“Measure what matters” is the name of a very popular business book written by John Doerr which talks about using the right data points and metrics to evaluate how well the company is performing. This concept was first introduced back in the ’70s by Andrew Grove of Intel. The crux is about bringing everything to a single number.
This article is not about business but specifically about college admission. The place is where we are in dire need of proper measurement. The status quo is so ridiculous that I cannot put words to it.
First, I am not going to end up glorifying the JEEs, NEETs, The US admission system, or the European Admission system. However, I will be addressing them all. This article is about what we should measure rather than any specific measuring system.
The biggest problem is in quantifying what one should be measuring for admissions into colleges. But a few things all can agree on are the measuring metric must be fair, meritocratic(more on that in a later article), objective, and meaningful. The question is about the meaningfulness of that metric.
What I mean by this is that the metric that people use to judge you should be meaning something and must be confirmed for reliability and validity. i.e whether the metric when assessed to yield comparable results with multiple attempts (reliability) and whether the metric correlates with success in the university (validity). To simply put, the metric must objectively measure the potential for college success.
Let’s come to the issue with validity later. First reliability. Indian admissions systems are specifically designed to be not tested for reliability because it is a one-shot, one opportunity type system. But one can reasonably suppose that a guy/girl who got into mechanical engineering IIT madras will probably get into IIT Gandhinagar or Varanasi if he/she takes the test another time or vice-versa. Same with TNEA.
Another aspect of reliability is “game-ability”. Let us say "game-ability" as the ease at which the system can be gamed. In that measure, all the Indian admission tests fare poorly. In my class of around 50-55 merit students, at least 40-43 students came from either Nammakal schools or similar systems. What it says is that if you are not educated in Namakkal schools, the odds of you entering PSG tech is reduced ten-fold if you are not educated in the almost jail-like Namakkal type schools. This problem is much worse with the IITs. The official figure is that 96% of the students in IITs are trained by some coaching institute to get specifically into IITs. This translates to twenty-three – fold lesser odds for someone attempting to get into IITs without the aid of coaching institutes. Worst of all is that I have good reasons to believe that this figure is under-reported.
What is happening is that these institutions (coaching classes for IITs, Medical colleges, and Namakkal schools for TN engineering) know how to manipulate and game the metric, which takes the meaning out of the metric. In an ideal world, once something is gamed, the yield should not be of much value. Similar to card counting in poker (what Alan does in hangover) or using formulae for solving Rubik’s cube in 20 seconds, it is both unfair and meaningless.
Now let’s come to the validity part. Things get bad for TNEA here. Sure, the weighted mean Math, Physics, and Chemistry should have meant something if the assessment was well designed. But the reality is much different.
I have always had profound discontent and doubts about its validity with TNEA admissions. And boy, it did not disappoint. PSG tech is supposed to the best engineering college outside of the IITs and NITs in Tamil Nadu. Or at least that is what its reputation is. And if you measure in terms of the facilities, Alumni, Faculty credentials, reputation, and the opportunities, PSG Tech is certainly right up there.
But the question is whether that is the right measure to evaluate the quality of an institution. I believe it is not. The quality of the college, at least for students, should be a direct function of the quality of the current student pool. Because, at the end of the day, it is them who has the most impact on you because you are directly learning from them, and competing and compared against them. In that way, saying I was disappointed is an understatement. Not to be disparaging on anyone, most are good people, but engineering potential and goodness are two different things. i.e this is not an attack on people but the system.
One of the more vivid memories of my experience with this is during my third year in college in our Formula student team. We preparing a question paper to recruit students into the team (Mostly first and second years). One of the questions was to calculate the approximate mass of the moon. Sound hard, but hear me out. I gave the mass of earth, the radius of the orbit, the velocity of the moon’s revolution around the earth, and the universal gravitational constant. Think about how you would solve it.
In a sample size of around 150, only one got the method right. Another got the answer right, but there was only the answer, so wonder how he would have got it. This is ridiculous. Because the students who don’t know the formulae for this problem will never have passed the 9th or 10th standard science exam, let alone getting state topping marks in physics and maths, that is necessary to get into PSG tech on merit. And if I had framed the question in another way, almost all would have arrived at the answer.
For example, let's say we have two objects in space A&B, and body B is revolving around body A at so-and-so m/s. Body B is gravitationally attracted to body A. Find the mass of body B, given that the mass of body A is XYZ and the radius of the orbit is XYZ, such that the system is in stable equilibrium.
The difference is that in the second case it is quite clear that you have to equate centrifugal force to gravitational force. But in the first case, it is implicit. The problem is too easy for just 1% of the supposedly brilliant PSG tech population to solve correctly. The IIT JEE, in this measure, is not bad per se but certainly could have been much better.
On the other end of the problem, there is this huge operational issue. The population of India and the number of aspiring engineers in India are making it unsustainable for any other more equitable systems like that the ones in the US or Europe to be implemented. That said, the status quo is certainly a bad place. Could and should be better.
Comments