Exams and handwriting

Mary had a little lamb
Its fleece was white as snow
And everywhere that Mary went
The lamb was sure to go

fast 20 - 24 sec
standard 25 - 29 sec
slow 30 - 34 sec
very slow 35 - 39 sec
extremely slow 40+ sec

Exams, Competencies and Psychometrics

"In combination, therefore, the two studies support the existence of a general memory ability, independent of IQ, and indicate that this ability accounts for between 10 and 20% of the variance in exam performance. Surprisingly Cattel IQ did not contribute to GCSE performance in either group. There was of course a large difference in both GCSE performance and Cattel IQ between the groups, but the multiple differences between them make it unwise to ascribe the differences in GCSE performance solely to Cattel IQ."

"... an increasing number of studies were published which showed that traditional aptitude and knowledge content tests, as well as school grades and credentials:

1. did not predict job performance or success in life (see McClelland, 1973, for a review of this literature); and

2. were often biased against minorities, women, and persons from lower socioeconomic strata (Fallows 1985)."

"While changing motives and traits is possible (McClelland and Winter, 1971), the process is lengthy, expensive and difficult. From a cost-effectiveness standpoint, the rule is 'hire for core motivation and trait characteristics, and develop knowledge and skills.' Most organisations do the reverse: they hire on the basis of educational credentials (MBAs from good schools) and assume the candidates can be indoctrinated with the appropriate motives and traits... in the words of one personnel manager, 'You can teach a turkey to climb a tree, but it's easier to hire a squirrel.'"

• Can competency assessment methods help?
• How useful are standard exams?
• How useful are interviews?
• Has any research been done?

Answers from Elizabeth Valentine:

> We thought you might be able to answer a question prompted by
> your research: What relationship exists between "memory power"
> and "problem solving ability"?

I suspect that no simple relationship exists between 'memory power' (whatever that is) and 'problem solving ability' (what kind of problems?) A good SHORT-term or 'working memory' would be an asset for problem because this would enables you to hold information in temporary storage while working on it.

The nature of the 'memory power' might determine whether or not a good long-term memory was a help or a hindrance in problem solving. If it was a 'rote' sort, based on mnemonic techniques where the focus was on superficial non-meaningful aspects of the material, then, as your correspondent suggested, this might be deleterious. On the other hand, if the superior memory was of the sort cultivated by the mathematician Aitken who tried to keep his mind limpid so as to absorb material and sought MEANINGFUL relationships in material, then this might well be an advantage in having a rich domain of associations on which to draw.

> A reader has suggested that, in his experience, there is at
> best a poor correlation between academic ability (measured
> by class of degree) and the ability to solve problems on the
> job. Has there been any research done that might throw light
> on this?

This wouldn't surprise me as obviously many other ingredients such as motivation and political nous come into 'real-life' practical success. The only specific piece of research I know about is that which demonstrated a poor relation between remembering verbal material and remembering to do things.

> He also speculated that a good memory may be detrimental
> to problem solving because it encourages a "rote learning"
> mind set ...

Now, don't ask me any more questions or I'll be tempted to spend time answering them!

"UNIVERSITIES are to recruit students by postcode under a new system in which preferential treatment will be given to applicants from poorer backgrounds."

"Alan Smithers, the professor of education at Liverpool University, condemned the move, saying: "There is an obvious danger that bright students with affluent parents will be discriminated against unfairly. Admission to higher education should be on the basis of ability, not quotas."

• Can "exam technique" be taught?
  


• How well is "academic ability" measured by the exams for university entrance? (You will see the notes above questions on exams and writing speed, writing style, memory, exam panic etc.)
  


• Has "academic ability" a strong relationship with the co-operative problem solving said to be required in many modern knowledge-based industries? (cf. "They learn some from highly paid professors who now teach a way of writing that sounds clever whatever you are actually saying." posted elsewhere on faxfn.)
  


• Has much research been done to look at demographic groups by their educational characteristics? You may also recall the report by John Clare, the Telegraph's Education Editor, "'Leafy suburbs' dominate places at universities". Clearly there is a strong postcode effect in university recruitment. And demographic clustering of postcodes does show groupings of residential neighbourhoods where significantly different types of people live. (eg "Urban venturers", "Country Dwellers" or "The Hard Pressed".) You may still have colleagues at the University of Liverpool that have worked extensively on these systems.
  


• Finally, is proficiency in "exam technique" related to postcodes?
  

Dropout rates at some universities are an astounding 40 per cent, according to figures to be published next week which herald the first official league tables for higher education.

Professor Alan Smithers, of Liverpool University, was worried about taxpayers' money going down the drain. "I am startled that so many people who have gained entry are leaving in such a short period of time," he said. "It may not be good value for money for universities to be open to everyone if you then have 40 per cent of them leaving. This is worthy of systematic inquiry."

• Are graduates "finished products" and dropouts wasted "work-in-progress"?
  


• "Is there any serious work assessing the real value of degree courses?"
  


• Do you have any comment on any of these pieces?
  
  

  My Degree: 3 weeks work, 87 weeks drunk
  Graduates with the social skills of a caterpillar
  They learn then forget
  Degree results cannot be taken seriously

  
  

... an increasing number of studies were published which showed that traditional academic and knowledge content tests, as well as school grades and credentials:

1. did not predict job performance or success in life (see McClelland, 1973, for a review of the literature); and
2. were often biased against minorities, women, and persons from lower socioeconomic strata (Fallows, 1985).

Professor Alan Smithers at the University of Liverpool's Centre for Education and Employment Research regards the tests like astrology.

"They can be a very useful starting point for a discussion and might draw young people's attention to careers they had not thought of before. But the danger is that people limit their ambitions as a result. Tests might be a useful guide, but there is a greater variety to human talents and creativity." (The Independent, 15th November 1999)

On 06mar2003 The Telegraph had a story Universities told not to rely just on A-levels . They quote Charles Clarke, the Education Secretary

"A-levels are an important guide to educational achievement but they are not a perfect test of everything," he said.

"I'm not saying 'here's a model', but other indicators of merit and ability are essays, interviews and the Scholastic Aptitude Test used by American universities. Those are the sort of things we need to look at."

See also The Guardian on the same day which has a story Clarke attacks Bristol boycott . It too reports Sir Howard Newby

He told MPs that independent schools were good at getting high A-level grades for their pupils, but that did not necessarily mean those young people would do better at university than their state school peers: "When they come through to university, it is not always the better coached student that performs better."

In Evidence and Education in our Education and Training Section there is a quote from Alison Wolf's book "Does Education Matter"

...we cannot conclude ... that the skills that employers are actually using and looking for are indeed the ones gained late in the day. The most valuable could have been acquired much earlier - by age fourteen, sixteen or eighteen - and we have seen strong suggestions that this may indeed be the case.

In contrast to this research published by the UK National Statistics in Labour Market Trends does show graduates get, on average, good value for their investment in education but that for some types of degree, especially arts degrees, there is a negative rate of return.

On 07mar2003 The Independent had a story School leavers with two A-levels earn more than arts graduates which reported this.

Graduates of arts courses would be better off if they had quit full-time education after A-levels rather than studied for a degree, research published yesterday indicates.

"Feeling warm about literature doesn't pay the rent," Professor Walker [of the University of Warwick]said. "Maybe an average arts student knows he or she is not going to do very well. Maybe they do not. Education is a risk individuals take. We need to make sure people have the correct perceptions."

There are large average returns to education. There is a significant variance in returns across individuals. There is no evidence that the recent expansion in higher education has resulted in the financial returns falling – implying that the expansion in supply is just keeping up with growing demand. The effect of education on wages actually does work via higher productivity. However, while it has been shown that the private returns to education are large this is not enough to answer the question of the extent to which this education should, or should not, be subsidised.

The authors, Ian Walker of the University of Warwick and Yu Zhu of the University of Kent, base their conclusion on the fact that when the school leaving age was raised in 1973, only a few extra pupils stayed on at school past the increased minimum leaving date. They consider two explanations of the relationship between earning and education:

To investigate this important issue it is necessary to look at it in a roundabout way. If people choose education in order to distinguish themselves from others then, if a low productivity group were to raise its education for some policy induced reason, the more productive would also want to invest in more education in order to continue to distinguish themselves from the less productive. On the other hand, if education simply makes people more productive then educating one group more has no effect on the decisions of others.

What is not clear from the paper is to what extent the authors reject the hypothesis that employers pay wages according to the qualifications of their employees in the (mistaken) belief that these are a good guide to productivity.

Clearly the publication of this paper in Labour Market Trends is an important statement by one branch of Government.

I left school at 18. I took three A levels but as I was moving house and starting a new job, I have never discovered if I passed or what grades I got. I am now 35 and have worked all the time since except for one month when the company I worked for went bust.

I started as a retail assistant in a photo shop. I then got a job in London working for a photographer for two years before returning North to work in a photographic studio in Leeds. After five years, the studio went bust and I started my own business, which become so successful I have saturated my particular market.

While I was working in London, I applied to the BBC for the position of trainee cameraman. Out of hundreds of applicants I survived three interviews and got down to the final dozen. I remember that there were about as many jobs as final interviewees. I was offered a place by letter shortly afterwards.

Within days I received a phone call to double check my physics O level result. I had a C grade but BBC Recruitment insisted on a B grade. They had initially misread the CV and thought I had a B. So no job. They did say I could reapply the following year with a B grade in O level physics and I would be offered a job.

Of course, I followed other avenues.

Faxfn has been sent an interesting article War on the SAT in The American Prospect.

This reports a different approach for selecting students for university courses in the US, where school grades and SAT tests are used routinely. SAT tests (Scholastic Aptitude Tests) are successors to intelligence tests, which were designed to be a measure of aptitude not of learning or cultural background. SAT tests have been recently discussed as solution to the UK's selection problem now that A-levels alone are not regarded as a gold standard.

But University of California President Richard Atkinson feels that the current SAT tests add little information to what other tests and grades show about a student's academic capabilities. "If you know the definition of those words," he said, "the reasoning is trivial."

At the same time, it discriminates against poor and minority students, and distracts attention from the core academic subjects that high-school students should focus on. "If you know the definition of those words," he said, "the reasoning is trivial."

In a piece for the Sacramento Bee, UC takes a look at SAT I's worth which can be found on the University of California office of the President website here. Richard Atkinson says

California has made significant strides in the last few years by establishing clear guidelines for K-12 school curriculums, setting high academic standards and employing standardized tests to assess student achievements. Yet for all these reforms, the admissions process at the University of California and other universities across the nation still puts great emphasis on the SAT I, the Standardized Assessment Test, aligned neither to standards nor school curriculums.

Simple fairness tells me this is wrong. We are, after all, a society built on twin notions: first, that actual achievement should be what matters most; and second, that people should be judged on the basis of what they have made of the opportunities available to them. Therefore, it seems only right that college-bound students should be judged on what they have accomplished during four years of high school, not on the basis of a single standardized exam designed to test undefined notions of "aptitude." For those reasons, I am recommending to the faculty that the SAT I no longer be required for students applying to UC.

But he approaches the postcode issue in a different way to Chris Woodhead and Alan Smithers.

But changing standardized test requirements is only a first step. We should also adopt a more comprehensive, "holistic" admissions process that takes a range of factors into consideration, from the quality of a student's high school to the opportunities available to that student. A young person who has made exceptional progress in challenging circumstances needs to be given special attention.

The American Prospect article reports a radical new idea that the top few percent of pupils from each high school should be admitted to university.

Atkinson says that UC's policy of admitting the top 4 percent of the seniors from each high school, the first of his admissions reforms to go into effect, has been a resounding success. Last year it prompted an 8 percent jump in the applicant pool, of which a significant percentage are black or Hispanic, many from high schools that had rarely sent students to UC before. In its first year, roughly 80 percent of the graduates in the top 4 percent of their high schools applied.