Jonathan's Corner
(Search & Sitemap)
(Search & Sitemap)
> Writing >
Essays >
The Hayward Nonstandard Test: An Interesting Failure
Previous
1
2
3
Next
Printer-Friendly Version
Charles's response was twenty-seven pages of wheels within wheels. From the first page I was met with nuance that let me know I hadn't taken everything in on the first reading, despite it being well-written. He claimed not to have any distinctive achievements. This modest remark was followed by no fewer than eight pages of dense summaries of some of his theories. These theories were subtle. They had a logical and scientific character and a spark of something interesting that stretches outside the bounds of science. He used a nonstandard format that made their logical structure clearer--successfully modifying a familiar format to make an unfamiliar format that works better, which is difficult. In the pages of his response I met an edifice of thought which impressed me and which I knew I didn't understand. (I say this as someone who has put a lot of effort into understanding other people's belief systems.) His response to that question reminds me of a passage in my current novel:
The woman looked at me briefly. "What languages do you know?"
If anything, I sank further back into my chair. I wished the question would go away. When she continued to listen, I waited for sluggish thoughts to congeal. "I... Fish, Shroud, Inscription, and Shadow are all spoken around my island, and I speak all of them well. I speak Starlight badly, despite the fact that they trade with our village frequently. I do not speak Stream well at all, even though it is known to many races of voyagers. I once translated a book from Boulder to Pedestal, although that is hardly to be reckoned: it was obscure and technical, and it has nothing of the invisible subtlety of 'common' conversation. You know how--"
The man said, "Yes; something highly technical in a matter you understand is always easier to translate than children's talk. Go on."
"And--I created a special purpose language," I said, "to try to help a child who couldn't speak. I did my best, but it didn't work. I still don't understand why not. And I--" I tried to think, to remember if there were any languages I had omitted. Nothing returned to my mind.
I looked down and closed my eyes. "I'm sorry. I'm not very good with languages."
Charles listed approximately fifty different interests--which is less significant than it sounds, as he broke his interests down in more detail than the other normees, but the detailed breakdown strikes me as significant independent of its content. He was the one normee who answered the Myers-Briggs question in the mathematical format requested--which does not mean that he is the only normee who could do that task, but may suggest that he was the one person who didn't take a shortcut by "just using adjectives". I wrote the test to listen for a certain accent in how people respond, and his sense of humor showed that accent loud and strong.
He wrote a complete test which seemed to have a low ceiling, but was polished enough that I wouldn't be surprised to see something similar on the web, and he showed self-criticism in writing the test, acknowledging that it was culture-biased. The completeness and level of polish for that answer caught me off guard.
I was looking to be surprised in a certain way, and for reasons discussed above Charles gave me the kind of surprises I was looking for.
David's response was twenty pages. He provided an extended writing sample, and (to my surprise) a complete transcript of grades from childhood. His answers were by far the most polished; they give the impression of finding, out of a large space of things that could be said, a microcosmic gem that encapsulates the whole space. Most of his responses were short; the twenty pages stem from the length of his answers to a small number of questions.
Question 11, requesting Myers-Briggs personality type, contained a hidden question. I was interested in Myers-Briggs type, but most interested in whether the normee would question the test or talk about not fitting in the frame the Myers-Briggs test provides. David told his type en route to making a dismissive remark about the test. In other words, he was the one respondent who questioned the test. The most cherished creation he gave was one that showed a certain kind of mental fireworks, reminiscent of the dialogues in Douglas Hofstadter's Gödel, Escher, Bach: An Eternal Golden Braid (New York: Basic Books reprint, 1999).
David also surprised me, and I heard an accent of brilliance.
What are the distinctive features of my test? I would like to describe them below.
The way Western culture is shaped means that psychology tries to know its subject-matter with the same kind of knowing as physics has of its subject-matter, in other words I-It rather than I-Thou knowing that is depersonalised and banishes tacit knowing as far as possible. (Banishing anthropomorphism is appropriate when you're studying rocks. It's more debatable in trying to understand people.) When I was thinking about how to write up the experiment, before I looked at prior scores, one of the things I intended to compare was writing samples. Brandon offered a clever placeholder in place of a "real" composition. Adam provided some poetry that reminded me of fifth grade English reading; I objectively recognized quality but felt no subjective emotional response. Charles provided poetry that I wasn't sure I understood but none the less felt like something powerful was washing over me, and I was sorry when it ended. David sent a fiction excerpt that filled me with despair. The tone of the writing was not despairing; I felt the despair of being shown writing so perfect that I despaired of ever attaining that standard.
Why am I talking about my subjective emotional reactions instead of objective assessment? That is why I chose this specific example, instead of examples of thought that would have more to justify them from the framework that understands knowledge in depersonalized and objective terms. I choose it because I paid attention to subjective emotional reactions. I believe that they are tied to tacit and personal ways of knowing: I experienced subjective emotional reactions because I was responding to different pieces of writing that were not of the same quality. Subjective emotional response is one of several things that can be a cue worth listening to.
(I am intentionally keeping the philosophy brief; the philosophical dimension involved in this topic is one that admits very long discussion.)
In most tests, there is a suite of questions meant to map out where a person's intelligence breaks down, and scoring is how many points total are earned. In this test, the questions do not represent a direct attempt to present difficulty in answering. The intent is rather to obtain a composite picture, and shed indirect light on how bright a person is. The assumption is that different levels of giftedness will leave a definite mark on a person, and that that definite mark is discernible through understanding the person. For one example, above a certain level, a person is so different from the majority of people that there is a social disconnect; children above IQ 170 tend to feel that they don't fit in anywhere. That kind of social disconnect was clearly discernible in all but one of the responses; Brandon clearly articulated it.
To some extent, that is corroborated by the data. I identified all of the normees as significantly gifted--which I had no reason to anticipate. The first norming of the Mega test had fewer than 10% of normees successfully answer any of the questions. (People who are emotionally insecure often attempt difficult tests to get an answer that may feel special; as the number of emotionally insecure people vastly outweighs the number of people at that level of giftedness, they "should" have been a small minority.) So I was able to recognize giftedness in all of the normees when I was not expecting it. That stated, the evidence does not warrant the conclusion that my test usefully discriminates among the normees.
As this test, or at least this norming, has been a failure, it's worth paying attention to what went wrong.
I have not done any real statistical analysis because there is no basis for analysis, and the statistics would only give a more precise quantification to the statement, "The measurement error exceeds the difference measured." Even if the four normees represented an optimal 120-140-160-180 spread, four points would be questionable. As is, the only conclusion I can confidently claim from prior test data is that all of the normees are at or above standardized test ceilings. In other words, data from previous tests do not provide a basis to claim that my test discriminates (and what correlation exists is negative).
Giftedness affects personality, but it is inadequate to simply say, "Giftedness is personality." There is diversity at each stratum of giftedness, and the normee pool did not permit the kind of two-dimensional analysis that would be needed to properly interpret responses (if there is a proper interpretation to be had).
This test is invasive. It's painful and offensive. There is probably a way to attempt a similar operation much more gently and delicately. My guess is that this, more than anything else, is why I only had four responses in three years. If this principle were put to serious use, it would have to be rethought so that it went about its aims with a far defter touch. (Or perhaps just remove certain questions.)
One question which I wonder is whether this offensiveness, which is partly an unedited form of giftedness, was the main reason why only brilliant men responded. The test's form may have been a powerful selector. So it would have put most people off. But that is not the whole story. Keep in mind that "reading" on a conscious or unconscious level is a two-way street, and the test reveals something significant about me as well as requesting revelation of the normee. A few very bright people, however, might be bothered by the invasiveness, but they recognize and respond to a voice that feels like home. It connects. That, at least, is speculation which seems plausible, but which I don't see how to support without writing a gentler test.
In one sense, this test was personal, too personal--it probed bluntly into things that are not polite to ask. In another sense, though, it related to the normees as objects to be studied, trying to dissect them as people but still dissecting them. It moves partway from I-It to I-Thou, but I believe it is possible to have a fuller I-Thou knowing, although I don't know what a fully I-Thou approach would be like. It could be argued that the questions are offensive because the test was not personal enough. In other words, the test reflected an attempt to understand people but not in a personal way. Furthermore, some of the philosophical merits to a personal approach may bear fruit if there were a more genuinely personal approach.
Jonathan's Corner
(Search & Sitemap)
(Search & Sitemap)
> Writing >
Essays >
The Hayward Nonstandard Test: An Interesting Failure
Previous
1
2
3
Next
Printer-Friendly Version