Archive

Archive for June, 2008

Iq Test or Ia Test?

June 19th, 2008
Steve Gillman asked:


An IQ test is supposed to measure one’s “intelligence quotient,” to provide a basic rating of how smart a person is. When we try to measure people in this way, we hope it will predict how well they will perform in either their profession or academic work. Of course we know that this doesn’t work very well. We can all think of examples of highly intelligent people who are less successful than those with “weaker” minds.

In the world of academics, studies have shown that a student’s habits of self discipline are far more likely to predict high grades than the score on an IQ test. Obviously the usefulness of the latter kind of testing is limited, but what is the alternative? On possibility is the “intelligence application,” or “IA test.”

The IA Test

The idea here is not to measure how well you can use your brain on “paper problems.” This measuring of “brain potential” is what the IQ test is about. An IA score would be a rating of how well you actually apply your intelligence in your life. As far as I know, nobody has yet developed a systematic test of this sort. What would it consist of, then?

Most likely it would start with a standard IQ test of some sort. That measure of “what you have” would then be used in conjunction with a measure of “what you do with it” to come up with an IA score. The first number (your IQ score) might be added to or subtracted from according to measures of “application.” But what would we use for these? There are many possibilities.

If we used income as an “intelligence application measure,” we might start with zero for an average income for that particular IQ level, and add or subtract points for being above or below that. This is based on the idea that being more intelligent should lead to a higher income if one is applying that intelligence well. We could divide IQ scores into ten levels, and for each, find the average income or create a scheme for what ones income “should be.” Then we could add a point for each level a person is above their “presumed” income, or subtract one for each level below.

Of course, this measure assumes that higher income is either an objective value, or at least a goal of the person being tested. That brings up the first serious problem with this new kind of test. Do we want to use an objective set of values against which to measure how well people apply their intelligence, or should we measure according to the person’s own stated goals? There are tough challenges with both approaches. The first requires some agreement on what is universally valuable. The second assumes that people really know what they value.

This is the problem with the all the possible measures. We can create measures of some sort to see how well a person does in relationships, health maintenance, creative output, and even happiness, but we cannot so easily say what the value of those things is, or how much the person being tested really desires those things. At first this problem of standards and measurements may make it seem that we cannot develop an intelligence application test. But we can choose a standard (or develop two tests). As for the measuring, we already do it.

If you’ve ever said about a person, “He doesn’t use what he’s got,” or “She’s so smart, but she doesn’t do anything with it,” you were measuring. You can’t say such things without having some idea of what a person could be doing better. These kinds of comments are a measuring of performance against that idea or standard. Granted it isn’t precise, but it acknowledges that such things are measured.

The IA test would simply make it more precise. A statistician will tell you that when things are difficult to measure, you don’t give up: You just measure more. For example, if all you have is flawed rulers to measure a door with, you don’t use one once. You can get closer to a true measurement by using them all several times and taking the average. If you are measuring the relative happiness of people in various groups, whether the groups are determined by country of residence or IQ level, you can to some extent overcome the flaws in measurement by making enough of them. When a group consistently scores higher after thousands are tested using many methods, the data becomes more accurate and useful.

At the level of the individual, then, we would also want to measure many things in many ways, because of the uncertainty of any one measurement. We might measure income, as noted, but then measure again according to what income the person thinks would be ideal. We could also adjust this for age. We might measure success in relationships in six different ways, and the application of intelligence to household chores in several ways. Using all of these various measures, we might arrive at a IA test which can assign a score that actually means something. Or maybe not.

Perhaps the idea of “intelligence application” is most useful not for developing a new test, but for pointing out how flawed some kinds of tests can be. I’ve seen the same man score 70 on one IQ test and 140 on another, and I know of a millionaire who can’t read. If after generations of these tests being used they cannot be more accurate or predictive than that, there may not be too much hope for an IA test either. Life itself is the real “IA test,” and perhaps for a long time to come the closest measurements we’ll get will be the intuitive ones that lead to the comments, “He sure knows how to use what he’s got,” or “She just isn’t living up to her potential.”



Rodney

iq , ,

Psycho Babble – Psychometric (Personality) Assessment Testing

June 9th, 2008
Kim Jones asked:


I’ve never been that big a fan of psychometric testing (the “science” of personality assessment), since Jim, Maria and myself were tested and found to be unemployable in any normal company due to a collective tendency to stroll along to the beat of our individual drums. So we retaliated by setting up an advertising agency named after a strange looking animal with no relevance to advertising on the 13th of the month in the middle of the worst recession in 20 years.

Which pretty much proved there was nothing odd about us, we like to think.

Nevertheless, as every one of Britain’s 100 largest companies and some 80% of all the rest, use some sort of personality test on prospective employees, we thought it would be a good idea to have a look at the subject. Especially as the Sunday Telegraph did a big article on it which we could cull and make ourselves look dead clever.

There’s no doubt there’s a backlash against tests in the USA, evidenced by a mass of lawsuits from people who claim their lives have been damaged by them, particularly by invading their privacy and casting aspersions on their integrity. The pronouncement by the American Psychological Association that the vital “honesty” tests to sift out the potentially dishonest have a particularly high error rate hasn’t helped, while a newish book, “Cult of Personality”, argues persuasively that most profiling methods are flawed and damage the interests of all involved – the idea that you can capture the essence of a personality in a 20 minute test is nonsense given how complex people are, how moods can vary while being tested and, more importantly, how honestly the questions are answered. Re the latter, most testing groups claim to have anti-cheating (”lie-scales”) measures built in which attempt to measure consistency by asking the same question in different ways. That’s all very well but not all lies are conscious – there’s substantial evidence that people see things differently according to mood, and even day-to-day, and one body of research suggests that people taking an identical test a second time will be given a different personality type up to 75% of the time.

Psychometric fans, on the other hand, argue that the tests are not only a reliable guide but an acknowledgement that personality is critical in building a successful workplace. Which is why psychology has spent so long trying to crack the key to personality (and sold every resultant little fissure to business at every opportunity). The big break was the Myers-Briggs Type Indicator (MBTI), the most successful test in the world, used by 89 out of the Fortune 100. Loosely based on Jung’s view that we all have a predisposition to a certain personality type, the 20 minute tests assign people precise personality designations – for example, you might be an ESPF (Extrovert, Sensing, Perceiving, Feeling) type. All good stuff, if we do indeed all fall into precise personality types, and we don’t allow for variable attitudes. For example, I’d answer one question about whether you’d rather work for a good natured, inconsistent boss (me) or a sharp tongued, logical one (Maria) with “It depends on the circumstances and why does sharp tongued rule out a good nature?”. Other questions to ponder are whether personality is set in stone anyway and whether all downsides are accounted for – most ad agencies would, for example, be desperate for E (Oy! I meant Extroverts) which is great, except E’s are prone to selfishness. While I also wonder whether looking for certain types leads to a dull, uniform workplace without any of the creative tension we all so know and love (”Write that ad in the next five minutes or I’ll thump you a good one with my menacing stick” – creative tension, as created by a certain member of Giraffe).

Given the disparate views, why have the tests become so popular? Mainly because of the pressure in the 1970’s for workplace equality forcing employers to look for more democratic methods of selection. They then became a fad in the 1980’s “Helping to make people in HR feel wanted” according to one expert (don’t even think of shooting the messenger) while in the 1990’s the cost of, and problems involved in, removing unsuitable employees finally entrenched testing in the workplace. So much so that two years ago we were all entertained by the sacking of Carl Filer, a star salesman at Britain’s largest DIY chain who was on the fast track until a test revealed the shocking news that his favourite colour was blue – resulting in his removal for lack of dynamism. Doesn’t beat the story of Prof Glenn Ellenbourg of New York who, on profiling the personality of a corpse using a test that gave credit for non-responses, found the cadaver had an IQ of 45 – and was likely to enjoy a good measure of popularity round the office.

At Giraffe, we tend to go by instinct when hiring, which may explain one or two little things that go on round here, and have never tested anyone else in our lives. So the 20 years Katharine Briggs, a Washington DC housewife, spent studying Jungian theory in order to understand what her daughter Isabel saw in her boyfriend, Clarence Myers – which is how the Myers-Briggs tests came about (See. You didn’t know that, did you?) – were pretty much wasted on us.



ARAV

iq , ,