Tuesday 31 December 2019

Introduction



What is the blog about? Today we begin to understand what we mean by machine intelligences and artificial intelligence. We can see that they are already beginning to improve human life. I want to suggest that our enthusiasm for the cultures of machine and artificial intelligence risks creating a particular way of thinking and and decision-making that diminishes what it means to be human.

The first post in this blog gives a justification for characterising this way of thinking and decision-making as Artificial Stupidity. The next posts trace its origins and evolution, and will examine how we came to accept it. Later posts will use the testimonies of people involved in education, the health services and policing to explore how the blind application of Artificial Stupidity works to the detriment of society. I hope that this rather methodical approach will suggest ways of structuring partnerships of human intelligence and machine algorithms to work together for the benefit of all.

When I was asked for whom was I writing this blog, my first response was, “For a general audience.” But of course, this answer dodges the question. In reality, we are each of us specialists in our own varied fields. But wherever or however we work in this society, quite reasonably we tend to be asked to account for what we do and to hold others to account.

I am writing this to start a dialogue with that facet of any of us who has ever been asked to account for their work or who has asked others to do so.

You can read the posts in any order, of course. But I'm writing them to flow in a sequence:

New Public Management and Education

Setting targets in the NHS which ignore the fundamental context  in which hospitals and administrators work – the care of health – has catastrophic, lethal consequences. Setting targets for the Police to make them more accountable and rewarding those who hit the targets set and punishing those who fail, inevitably results in people playing the system until no one really knows what the true crime figures are.

In British education today we have a system that combines the worst of both these worlds. Politicians and education policy makers do not ask and cannot say what education is for, and with their rewards and punishments they encourage a gaming of the system which is arguably even more corrupting than the gaming that happens in policing. Worse, credible evidence suggests that this system is damaging the very children’s education it claims to improve. Report after report exposes this and nothing is done.


The background

Prime Minister Jim Callaghan lays a foundation stone of new residential block Biko House at Ruskin College, Oxford
Jim Callaghan lays the foundation stone of new residential block Biko House at Ruskin College, Oxford
It has been nearly 45 years since any serving cabinet minister, let alone Prime Minister, has asked what our children’s education is for.

The last was Jim Callaghan, then prime minister, who in his 1976 speech at Ruskin College Oxford began a debate on education in Britain that lasted for years. He suggested that:
“The goals of our education … are to equip children to the best of their ability for a lively, constructive, place in society, and also to fit them to do a job of work. Not one or the other but both.” [1]
At the time of the speech, the condition of education – at least in state schools –  was monitored by Her Majesty’s Inspectorate (the HMIs). They inspected schools nationally and the Chief Inspector consolidated and summarised the information they gathered for the Secretary of State for Education. In parallel to HMIs, Local Education Authorities sent teams in to schools in their area, inspecting and more especially, advising.

HMI’s were careful to differentiate particular needs of schools during their inspections; schools understood that inspections were an opportunity to have a dialogue aimed at raising standards, and ministers accepted – even when they did not enjoy – the strict independence of HMIs from the whims and vagaries of governments. The result was that HMIs were generally respected by schools, teachers and parents.

That is not to say it was a perfect system. It suffered from inconsistent standards across the country and many people were concerned about the independence of inspectors, of local chief education officers and councillors.

Up until the 1970s in general education, as somewhat earlier in medical education, people largely accepted that the experts – teachers, Local Education Authorities and civil servants – knew best and were working together for society and for the good of children. But as had happened in medical education, and as was happening in the police, the political climate was changing and the old assumptions and authorities were being questioned, not just by the young, but by the new breed of politicians as well.

Professor Terence Kealey,  vice-chancellor of the University of Buckingham, and a former adviser to Baroness Thatcher claimed in 1980 that it was her reforms that led to more transparency and accountability within the sector, while her push to liberalise rules on fees also had an immense impact.
“Before Mrs Thatcher, universities were very similar to public utilities – run for the benefit of staff with government money. … She was determined to introduce a much higher level of accountability for public funding and greater accountability for students as customers”. [2]
Margaret Thatcher had been Edward Heath’s Secretary of State for Education. Peter Wilby of the Guardian writes that it was there that she developed
“an abiding hatred of its culture: of ‘self-righteously socialist’ civil servants,  …  of ‘trendy’ teachers  (and) of local authorities she couldn't control.” 
He goes on to write,
“As an inexperienced and diplomatically inept minister in the early 1970s, Thatcher clashed with what was later called “the education establishment”. It patronised her as an ignorant outsider, blundering into areas that she was intellectually unfitted to understand. Her revenge, taken after she reached Downing Street, transformed education at every level.” [3]
Her revenge culminated in the 1988 Education Reform act, through which she took powers and independence away from the LEAs and schools, and concentrated them in the Secretary of State for Education. He (there was always the assumption that it would be a male) would decide the National Curriculum (teachers had virtually no say in its design or construction), he would decide the ‘programmes of study’ to be taught at each key stage; he would define the attainment targets which children would be expected to have and be assessed on at the end of each key stage. [4]

The aim was to have a centralised system, with schools subject to market forces, competing for customers (it was never clear whether these customers were the children or the parents or both) and depending for their success on centrally published test and exam results. For Head Teachers and their teams, good results would mean greater job security, more money and less interference; poor results could threaten your job, starve the school of funds and mean more frequent inspections.

Again, as with the police, in the name of efficiency and accountability, we see the power to decide policies moving up the chain and the responsibility for successfully carrying out the policies moving down.

It was left to Thatcher’s successor John Major to complete the transformation. When he took over government in 1990, his new Education Secretary Kenneth Clarke removed the inspectorate’s independence as well: saying that the government was going to:
“take the mystery out of education by providing the real choice which flows from … independent inspection” [5].  
Every one of the 24,000 schools in England would be inspected at least once every four years. This was beyond the capabilities of the existing inspectorate which only had 500 officers. But rather than enlarging it, Clarke replaced it with an entirely new organisation, Ofsted, under his direct control. He gave Ofsted just nine months to produce an inspection framework, hire 7,500 new inspectors, train them, set up an infrastructure and hit the ground running by visiting at least 6,000 schools a year.

That they achieved this in such a short timescale was in its limited way a triumph of organisation. But the cost was appalling.

Remember the ostensible reasons the government gave for replacing the old system: Thatcher and then Major claimed that their reforms would give value for money by imposing Performance Management systems, improve the quality of education and, by letting parents choose their children’s school by comparing like with like, create ‘customer choice’ and ‘customer satisfaction’.

What Ofsted did was a parody of business efficiency: The only way they could hit their target was to simplify the inspection process. In the name of efficiency and levelling the playing field, what was being inspected and how, were standardised to a degree that would have been unthinkable before, and for very good reasons. Where earlier, HMI reports had been responsive to the unique context of each school, its history, locale, catchment area and so forth, now each report had to shoehorn the school into one of seven simplistic categories from ‘Excellent’ to ‘Very poor’, and all reports had to have identical section headings.

Once again, in order to make the act of measuring simpler to manage, the subject being measured was distorted. To compare like with like meant standardising the measure. To improve the quality of education by using the same data, meant using the measure as a test, so that in the end it was the system effectively designing the curriculum, in order to make it easier to examine.


What has been the result?

On the one hand, Ofsted’s own reports claim that improvements have been widespread [6]. However, independent surveys tell a different story: in 1999, only 35% of schools said that the benefits of the Ofsted system outweighed the harm they caused [8]. Later in 2008 when teachers were asked if Ofsted had improved their teaching, only 5% said it had. If you compare exam results when Ofsted began, with exam results in 2003, you see a small but significant deterioration except in the very best schools. [9], [10]   

The Academies Commission  finds “considerable evidence” that the current accountability framework “inhibits change and innovation”, so that the regime is failing to achieve its stated goal of improving education.” [11] 


The impact on children


On the ground the situation is much worse. Demos said in 2013 that the “target-driven accountability in the English education system is … toxic to the life chances of children and young people.” [12]

Schools are abandoning the idea that a child’s journey through school should be designed as far as possible to met her or his needs. We have evidence that schools have responded to the pressure of league tables by
“Dissuading students from qualifications that would have value for them as individuals and persuading them instead to take easier qualifications that will contribute to the organisation’s ranking.” [13]   
For example, is not unknown for a school to tell parents to be told that their children could not take Art and Textiles as two subjects for GCSE as they were the same qualification. But the same school would let them take Art and Photography as two separate subjects. * 

When questioned, it turns out that while the children could get two GCSEs for their studies in either case, the school would be credited with two GCSEs only if the children took Art and Photography. Even if the children passed Art and Textiles exams, the school would only be credited with one GCSE  because of the complex and arbitrary rules of  ‘discounting’ that the department of education has deemed necessary [14]. 

In other words, in at least some schools, the design of a child’s education is governed by what will benefit the school, not the child.

We have evidence that schools are narrowing the curriculum because they are

“put under pressure to achieve a particular set of (moving) targets, (so) they can find it hard to hold on to the other things as well” [15] .  
Since the culture of new management came into education, children have far fewer opportunities to become involved in those aspects of creativity and sport that  employers say are going to be essential for their future well-being.

A House of Commons Select Committee reported that they,
“Received substantial evidence that teaching to the test, to an extent which narrows the curriculum and puts sustained learning at risk, is widespread. …test results are pursued at the expense of a rounded education for children. … We believe that true personalised learning is incompatible with a high-stakes single-level test which focuses on academic learning and does not assess a range of other skills which children might possess.” [16]
In 1999 57% of teachers said that changes resulting from Ofsted inspections resulted in the quality of education either deteriorating or not changing at all. 66% said that the educational standards achieved by the pupils had either deteriorated or not changed at all. [17]

And it seems that it is those who most need support who suffer most: The 2011 Demos and Private Equity Foundation report on the relationships between education and employment prospects particularly highlighted the way that the
“Value put on academic performance tended to create situations where those who had the strongest need to develop character skills – to become ‘more resilient, better at self-direction and possessing higher levels of application’ – were least likely to be given opportunities to do so.” [18]  

The impact on teachers

But Performance Management is not just toxic for children. Mary Bousted, the general secretary of the Association of Teachers and Lecturers said at the ATL 2015 annual conference, that the profession had become  “incompatible with normal life … with teachers exhausted, stressed and burnt out”. She went on,
“In 2011, the latest year for which figures are available, just 62% of newly qualified teachers who gained Qualified Teacher Status that year were still in teaching service a year later.” [19]
And the reason she gives is that
“…newly qualified teachers see very early on just what teaching has become and decide that they do not want to be a part of it.”
Ofsted inspections were the main culprit:
“Uncertainty was induced, with teachers experiencing confusion, anxiety, professional inadequacy of the marginalisation of positive emotions. They also suffered an assault on the professional selves, closely associated among primary teachers with their  professional roles.” [20] 
Far too often the inspectors themselves show a contempt for teaching and learning. One example demonstrates this nicely: the Senior Vice President of the Teacher and Lecturers’ Union is herself a practicing teacher. During on Ofsted inspection, she was told that the lesson she had taught was ‘Good’ rather than ‘Outstanding’. When she asked the inspector what she could do to make it ‘Outstanding’, he replied, “That’s for me to know and you to find out” [19].

As one Head said,
“You are made to feel totally incompetent. You are made to feel that everything you’ve done for the last 20 years is absolutely useless. [21]
In 1999, the National Foundation for Educational Research found that  24% of heads and 38% of teachers took time off for health reasons after Ofsted inspections.  Scanlon reports that at least one Head of Department
“simply walked out in the middle of the inspection itself and never came back. In the most extreme cases, heads or teachers were off work for months due to stress-related illness.” [17]
But perhaps just as damaging is the way Performance Management has pushed teachers to adopting corrupt practices in a way that is remarkably similar to the corruption that has been so endemic in the Police.

Cheating at SATs is so widespread, that when teacher after teacher described the methods used in the Guardian in 2002, no one was very surprised. [22]

In 2009, 70 schools had SATS results annulled or changed because of cheating. [23] In 2011, there were 292 reported cases and 370 the following year. But to get a sense of how prevalent the cheating is that does not get picked up, one has to go on to parents’ internet websites, where mother after mother worries about the stories of teachers ‘helping’ their children and agonise over whether or not to object.

In another report in 2013, one school one teacher is quoted as saying,
 “Nobody had bothered to get these kids to complete coursework - just give them a C,” “Then the exam board asked for samples of pupils' work to moderate the marking. The deputy head then asked teachers to ‘help’ the students complete 18 months of coursework in a fraction of the time” [24]
“At present honest teachers have only two choices: either ignore the injustice and continue to be punished for results which are lower than neighbouring schools which cheat, or blow the whistle on friends and colleagues - an extremely difficult thing to do. [25]

The impact on Schools

It is not just teachers who are being corrupted. Whole schools are affected.
“Statistics published in December by the exams regulator, Ofqual, show a 61% rise in the number of schools and colleges found guilty of malpractice last year, with 217 penalty notices issued, one for every 30 schools or colleges in the UK.” [25]
The reasons this happens are quite clear:
“The incentives for schools to push the boundaries to raise results seem very strong, and yet the disincentives, in terms of penalties and publicity, seem weak. …This can penalise schools that have acted properly and then find themselves competing, through league tables, against others that may not have.”[25]
Concerns have also been raised about Ofsted’s judgements on schools with the poorest and most challenging intakes. The old inspectorate took a school’s context into account, for example recognising  ‘schools in very poor areas . . . which could not . . . match the achievements of the higher categories but . . . did splendid social work’.  [26]   But Ofsted was afraid that this ‘might embed low expectations in some schools and let down the most needy, so they scrapped it. The result was ‘a 1995 study which reported that 90% of schools in the two highest social contexts were judged favourably by Ofsted, compared with only 10% in the two lowest social contexts.  [27]


The impact on policymakers

As Mary Boustead said,
“The findings of the (Public Accounts Committee (PAC), February 2015, report on School Oversight and Intervention) are devastating. In essence, our education system is being run on a wing and a prayer – and if something goes badly wrong, the Government relies upon someone being brave enough to speak out. Who knows what else is going on under the radar. … The PAC finds that action to prevent decline or continuing poor performance in schools ‘is rarely achieved.’ ”[19]
The assumption is that the best way to improve standards in education, is to make judgements about the effectiveness of school leaders and teachers,  in terms of specific criteria.  The people who originally designed Ofsted’s methodology said that a school’s performance against these criteria would be an indicator of how the it was doing across the board. This was based on an assumption that the test would act in the same way as statistical sampling, from which you could extrapolate how the rest of the school was doing. But as we know now,  it did not work out that way.

But what is astonishing, particularly in an educational context, is that no one was willing to accept that things were not working out as expected and no one was willing to do the rigorous, scientific, educational process, of understanding in what way it was not performing as expected, and for what reasons, in order to design a better process. In order to learn from experience, as every teacher knows, you have to accept and analyse your mistakes. That is what we ask of children. We should expect it of our teachers and of the education policy makers as well.

Instead we have an arrogant and irrational system that differs from the one it replaced only in that it claims to be rational and evidence based. How false this claim is, was revealed in 2008.

Just before he became Prime Minister, Tony Blair revealed that he and his wife had chosen not to send their sons to the school in their local catchment area, Islington Green, but instead to send them to the more prestigious Oratory School in Knightsbridge. This was in spite of Islington Green School being rather good – if less fashionable than the Oratory School.  The Times Educational Supplement said of Islington Green,
“Ken Riddell’s design and technology department had been described as a centre of excellence in the previous inspection report, and had improved since. It was just one of a number of robust departments, with a mix of experience and new enthusiasm, drawing in not only local families but parents from neighbouring Hackney. Morale was good, GCSE results were respectable: 38 per cent were getting five A*-Cs.” [28]  
There were rumblings in the press about the Blair's decision. It so happened that on the day Blair became Prime Minister in 1997, the school was told that it had failed its week-long Ofsted inspection. The staff were stunned. Ken Riddell told the TES,
“We didn't believe it, because we knew it wasn't true. We thought it was a mistake that would be sorted out.” [idem]
The School was put in Special Measures, and its reputation was ruined.

Eight years later, in 2005, following a request under the Freedom of Information Act, the TES learned that the original inspection had not failed the school at all. But Chris Woodhead, the then Chief Inspector of Ofsted – who had not visited the school –  chose to disagree with his own inspectors’ findings in a way that supported the Blair family’s decision. He sent in another team to back him up. But they too concluded – unanimously – that it was not failing. Woodhead overruled them as well. When questioned later, he said only that the decision had been within his powers. As the School’s headmaster said, the pupils believed for seven years they had been to a rubbish school and that their teachers were incompetent.
“Placing the school in special measures had been catastrophic for staff and pupils. Many colleagues who worked at Islington Green have suffered in their careers because of this unjust decision. However, the greatest victims have been the children of Islington” [29]
“The failure had devastating consequences for the school, which became notorious when pupil violence and behaviour soared out of control. There was a mass exodus of teaching staff, including eight heads of department and deputies, and a slump in GCSE results which dragged it into the bottom 2% of all schools nationally.” [30]
The school stayed in special measures until Woodhead resigned in 2000. The Blair’s children went to the Oratory school and no one needed to ask why their parents had not sent them to school in Islington. **

The regime that had been designed to replace supposedly inconsistent and arbitrary inspections with evidence-informed policy, had started to even more inconsistent and arbitrary inspections leading to policy-based evidence.

Worse, the data Ofsted has been collecting is almost certainly unreliable. The House of Commons Select Committee on Children, Schools and Families in their Third Report of Session 2007–08, said,
“The evidence we have received strongly favours the view that national tests do not serve all of the purposes for which they are, in fact used. The fact that the results of these tests are used for so many purposes, with high-stakes attached to the outcomes, creates tensions in the system leading to undesirable consequences, including distortion of the education experience of many children. In addition, the data derived from the testing system do not necessarily provide an accurate or complete picture of the performance of schools and teachers, yet they are relied upon by the Government, the QCA and Ofsted to make important decisions affecting the education system in general and individual schools, teachers and pupils in particular.” [31]
And the very act of collecting this unreliable data is done at the expense of the education of the very children the testing is intended to benefit. The Committee report continues with a section headed, “The consequences of high-stakes uses of testing”:
“We consider that the measurement of standards across the full curriculum is virtually impossible under the current testing regime.”

New Public Management created a system of accountability in our education system which is “toxic to the life chances of children and young people”, which has demonstrably demoralised teachers and schools, which has in well-documented cases encouraged dishonest practices and yet which had failed to produce any accurate measurement of standards. 

The most astonishing is that in the twelve years since the Select Committee Report was published was published, nothing substantial has changed. The masters of education will not recognise the evidence and refuse to learn.


[Next:  From evidence-base policy to irrational dogma ]



*    Personal communication from a parent and former head teacher
**   It is interesting to note that in 1979 in spite of the fact that Islington Green School was still technically failing, Roger Water sought out their school choir to sing the chorus on the Pink Floyd recording of “Another Brick in the Wall”. The words they sang are as follows:
We don't need no education
We don't need no thought control
No dark sarcasm in the classroom
Teachers leave them kids alone
Hey! Teachers! Leave them kids alone
All in all it's just another brick in the wall
All in all you're just another brick in the wall



[1]   Callaghan, James   A rational debate based on the facts  in Education in England October 18 1976http://www.educationengland.org.uk/documents/speeches/1976ruskin.html  accessed March 23 2015 

[2]   Grove, Jack  Thatcher had ‘immense impact’ on higher education  at https://www.timeshighereducation.com/news/thatcher-had-immense-impact-on-higher-education/2003059.article#node-comments    Times Higher Education   April 8 2013    Accessed 1 December 2019


[3]   Wilby, Peter   Margaret Thatcher's education legacy is still with us  the guardian.com  15 April 2013   http://www.theguardian.com/education/2013/apr/15/margaret-thatcher-education-legacy-gove   accessed 24 March 2015


[4]   Gillard, Derek   Education in England: a brief history  in  educationengland.org.uk  January 2011  at http://www.educationengland.org.uk/history/  accessed 23 March 2015


[5]   Wilcox, B; Gray, J   Inspecting Schools  Open University Press, Buckingham  1996


[6]   Ofsted. (1993-2011). Annual Reports of HMCIs of Schools.Ofsted.


[7]   Matthews, P., & Sammons, P. (2011). Improvement through inspection: an evaluation of the impact of Ofsted's work.University of London. Institute of Education. Ofsted.

[8]   Thomas, G  Standards and school inspection: the rhetoric and the reality in “An Inspector Calls: Ofsted and its effects on schools standards” Cullingford, C (ed) Kogan Page, London 1999 


[9]   Rosenthal, Leslie  “Do school inspections improve school quality?” in  Economics of Education Review  Elsevier Vol 23  2003 pp 143-151


[10]   Shaw, I; Newton, D P; Atkin, M; Darnell, R   Do OFSTED Inspections of Secondary Schools Make a Difference to GCSE Results? in British Educational Research Journal  2003  Wiley-Blackwell Vol 29 issue 1 pp 63-75


[11]   Gilbert, C., Husbands, C., & Wigdortz, B. (2013). Unleashing Greatness: Getting the best from an academised system. The Report of the Academies Commission.London: Pearson RSA.


[12]   Park, J. (2013). Detoxifying School Accountability.London: Demos


[13]   Wolf, A. (2011). Review of Vocational Education: The Wolf report.London: Department of Education

[14 ] 
As an example of the 'complex and arbitrary rules of discounting’ this is the table of
AS Art and Design discount codes published by the Department for Education for Summer 2016 and 2017. No reason is given for the distinction made for art and photography,

From https://qualifications.pearson.com/en/subjects/art-and-design.updates.html?article=%2Fcontent%2Fdemo%2Fen%2Fnews-policy%2Fsubject-updates%2Fart-and-design%2Fart-and-design-discount-codes  accessed 7/12/20


[15]   Park 2013

[16]   House of Commons Children, Schools and Families Committee: Testing and Assessment Third Report.(2008, May 7). Retrieved March 23, 2015 from www.educationengland.org: http://www.educationengland.org.uk/documents/pdfs/2008-testing-and-assessment.pdf


[17]   Scanlon, M. (1999). The Impact of Ofsted Inspections.Slough: National Foundation for Educational Research.


[18]   Birdwell, J., Grist, M., & Margo, J. (2011). The Forgotten Half: A Demos and Private Equity Foundation report on school to work transitions.London: Demos.


[19]   Bousted, M. (2015). Mary Bousted Speech to ATL conference .Retrieved April 1, 2015 from www.ATL.org: https://www.atl.org.uk/policy-and-campaigns/conference/2015/mary-bousted-speech.asp

[20]   Jeffrey, B., & Woods, P. (1996). Feeling deprofessionalised: the social construction of emotions during an OFSTED inspection . Cambridge Journal of Education, 26(3), 325-343.

[21]   Primary teacher, L. n. . The many ways of cheating at SATs. The Guardian, Society Public voices: public values  (2002, June 4

[22]   Greenwood, P. (2002, June 2). Children and teachers testify to the cheating.Retrieved April 1, 2015 from www.theguardian.com: http://www.theguardian.com/education/2002/jun/05/schools.news1

[23]   Swaine, J. (2009, July 28). Pupils disqualified from Sats after teachers cheated.Retrieved April 1, 2015 from www.telegraph.co.uk: http://www.telegraph.co.uk/education/educationnews/5918401/Pupils-disqualified-from-Sats-after-teachers-cheated.html

[24]   Burns, J. (2013, September 4). Schools' tricks to inflate science grades revealed.Retrieved March 26, 2015 from BBC Education : http://www.bbc.co.uk/news/education-23947042

[25]   Mansell, W. (2015, January 13). School whistleblowers call for naming and shaming.Retrieved March 26, 2015 from theguardian.com: http://www.theguardian.com/education/2015/jan/13/school-coursework-cheat-whistleblowers

[26]   Gray, J. (2004). Some frames of reference and traditions of interpretation: some issues in the identification of under-achieving schools. British Journal of Educational Studies, 52(3), 293-309.

[27] Matthews, P., & Smith, G. (1995). Ofsted: inspecting schools and improvement through inspection, Cambridge Journal of Education. Cambridge Journal of Education, 25(1), 23-34.

[28]   Wallace, W. (2008, May 11).  Who failed Islington Green? Retrieved May 2, 2015 from TES: https://www.tes.co.uk/article.aspx?storycode=2095239

[29]   Shaw, M. (2008, May 11). Inspection verdict questioned.Retrieved April 1, 2015 from TES: https://www.tes.co.uk/article.aspx?storycode=2072805


[30]   Smithers, R. (2005, Febrauary 4). Woodhead overrode inspectors to fail improving school.Retrieved April 1, 2015 from The Guardian: http://www.theguardian.com/uk/2005/feb/04/politics.freedomofinformation

[31]   House of Commons Children, Schools and Families Committee: Testing and Assessment Third Report.(2008, May 7). Retrieved March 23, 2015 from www.educationengland.org: http://www.educationengland.org.uk/documents/pdfs/2008-testing-and-assessment.pdf

Saturday 28 December 2019

Measure for Measure: New Public Management and Policing


Image of a ruler and a ball of string


Measure for Measure: Tom Peters and Goodhart’s Law

One of Herbert Simon’s students at Stanford Business School was a Vietnam Vet called Tom Peters. While completing his PhD, Peters joined McKinsey and Company as a management consultant where in 1982 he wrote a book exploring the art and science of management. Unusually for business books of that time, he used the evidence from several successful companies for case studies. The book was called In Search of Excellence and was a sensation, becoming the best selling business book of all time. Its influence on New Public Management cannot be over emphasised.

To get a flavour of the doctrine Peters was promoting, it is useful to re-read a 1986 column from his website, written after he had left McKinsey to found his own consulting company. In it, Peters wrote as a true disciple of Simon,
“I think the soundest management advice I’ve heard is the old saw; ‘What gets measured gets done.’ My own organization applies this dictum rigorously. … We quantify wherever possible. … We also urge every organizational unit in every function to develop key quality measures. Progress should be posted on charts in every work space, and a quantitative goal report should be the first item of business at every staff meeting, regardless of topic.”
Reaching these goals is to be rewarded:
“There should be at least ten celebratory acts each month, no matter how small – for example, coffee and danish for a project team that completed its work on schedule.”
and he ends with this exhortation to his manager readers:
“Why don’t you plan to quantify ten key areas in the next 30 days?  Even the process of quantification can be quantified.
Peters is performing a logical slight-of-hand here.  He moves from the unarguable common-sense use of measurements as diagnostic tool to shine a light on what is going on, to the more questionable use of measurement for target setting, complete with a reward system for hitting the target (and implied punishment for failing to do so).

He uses his enthusiasm to mask this lack of logic.  (And perhaps his enthusiasm allowed him to be somewhat cavalier with the evidence he was using: nearly twenty years after the publication of In Search of Excellence, when Tom Peters had become a revered management guru beyond all criticism, he was interviewed by the magazine Fast Company. In the interview Peters says, “For what it’s worth, okay, I confess: We faked the data. … You can always worry about proving the facts later.” )[1] *

Now when Archie Cochrane was measuring the relative health of the patients under his care on differing diets, it was purely for diagnostic purposes, to show up any regularities that would indicate how effective treatments worked under differing circumstances. When, later in his career he worked with miners in south Wales, he used statistical measurements to diagnose different causes of pulmonary diseases. He would never have confused the collection of data with the setting of targets. Evidence based medicine was used purely as a diagnostic tool.

Why is this point worth labouring?

In 1975, 
 in an essay on the problems of monetary management, the British economist Charles Goodhart wrote that,
“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”[2]
Now known as Goodhart's law, this has been translated into plain English as:
“When a measure becomes a target, it ceases to be a good measure.”
In other words, when a measurement is used as a target rather than as a diagnostic tool, sooner or later it will become so untrustworthy as to stop being useful as a measure.

But Goodhart's law does not fully help us see the absurdity of using a measure as a target.


Football gives us a wonderful example. It used to be a simple spectator sport with stars and character whom the spectators got to know over time. But in the modern game, mostly watched on television,  dedicated HD cameras and complex tracking systems* allow most aspects of a Premier League soccer player's performance during a game to be measured and recorded.

For example, we know that during the 2015-2016 season, Kevin de Bruyne of Manchester City averaged 0.28 goals per match, made an average of 45.08 passes per match and made 28 tackles of which 84% were successful.  However in the 2018-2019 season, he averaged only 0.11 goals per match, made an average of 38.16 passes per match and made 22 tackles of which 68% were successful.   [3]

You can see that for Mr De Bruyne, his manager or his agent these statistics could be useful as measures. But would it not be absurd to use them as targets? Can you imagine Pep Guardiola his manger saying at the start of a new season, "Kevin. welcome back. Now, for the new season, we'd like you to raise the number of successful tackles you average per game from 68%b to 72%, or else we'll fine you.  And we'll pay you a bonus if you pass more than 45 balls per match.” 

When we look at the use of targets in the management of policing in the UK., it becomes blindingly clear how destructive and how absurd these policies are

Portraits of James Goodhart and Tom Peters
James Goodhart and Tom Peters



Centralizing accountability in the Police force

Badge of British Chief Constable
The badge of  Chief Constable

Until 1964 in England and Wales, it was local government who monitored the police in each borough. For all the system’s many faults and opportunities for cronyism, at least there was a sense that policing was done by local consent. But corruption scandals involving the Brighton and Nottingham watch committees in the late 1950s prompted the setting up of a Royal commission on the Police and, in 1964, the passing of the Police 1964 Act.


This got rid of the locally elected watch committees and reduced the number of autonomous police forces from 158 to just 49, creating a hierarchy that centralised accountability, taking it away from local, public scrutiny. This

“…moved the system of British police organization further from institutional accountability to the public, and towards a more centralized system…and this continued to be a source of on-going controversy.” [4]  
According to the Scarman enquiry, the Brixton riots were a direct result of police acting without any consultation with the community or the home-beat officers. Scarman recommended changes in training and law enforcement to take local contexts into account. However the Home Office did very little in this direction. As Barry Loveday writes, 
“In place of major reform, police services were instead made subject to an increasing range of performance measures.” [5] 
Rather than make policing more context aware, the Home Office chose to centralise control and devolve responsibility. When in 2001 David Blunket, Blair’s Home Secretary, said that before crime could be tackled effectively, it needed to be measured, it sounded so reasonable. But as the British Medical Association said, referring to evidence-based medicine, so much depends on what is being measured by whom and how that data will be used.

In the case of crime, the Home Office decided that the ‘what’ was to be crime level reduction and detection rates, and the ‘by whom’ was to be the police themselves. The Home Office would then use measurements to create the ‘performance indicators’ – targets.

The catastrophes in the NHS showed how dangerous it can be to turn measurements into targets. But where the NHS showed how it is possible to hit the targets and still deliver abysmal – even lethal – levels of service, the Police showed how easy it was for them to hit and even exceed targets by manipulating the figures to say whatever they wanted them to say.


The Police were – and still are – under considerable pressure to hit the targets set. Not just for Chief Constables, but all the way down the ranks, where every officer’s career, promotion prospects, and sometimes job security, depended and still depends on how successful they are at reducing the number of crimes recorded and increasing the detection rates. But since the recording of this data is left to the police themselves, and as the pressure to meet the targets increases, it is hardly surprising that so many cases have emerged of officers from the top down recording the evidence in such a way that the figures show what the politicians want to see. So prevalent has this become over the last twenty years, that the practice has acquired its own name – ‘Gaming’. But this ‘gaming’ is not fun and it is certainly not harmless. If we look more closely at how gaming takes place, we will see the extent and degree of destruction it causes.



Tampering with recorded crime levels: Cuffing

The detection rate is calculated as a percentage of recorded crimes. This means that if, for example, a police force records 1,000 burglaries and solves 500 of them, their detection rate is 50%. If the following year they solve the same number, but have only recorded 900 burglaries, the detection rate goes up to 55%.

So if a police force or department is worried about hitting its detection rate targets, it is very tempting to increase the ratio by simply reducing the number of crimes they record. How would they do this? The process is called cuffing, a term borrowed from conjurors hiding cards up their sleeve.


A trivial example of ‘cuffing’ was reported by the Daily Mail, citing a memo from a police inspector in Norwich, which announced that one of the targets for 2008 was to “Keep recorded crime down to 1,500” and that to achieve this, “

Rank-and-file officers in the Norfolk force have been told that incidents such as car vandalism should not be classed as an offence when there is ‘no idea how it happened’ ” [6]
But cuffing can involve far more serious crimes. If the police have reasonable grounds for disbelieving a woman who says she has been raped, they will not record it as a crime. It is ‘no crimed’

Now it is a fact that investigating a rape allegation can be time consuming. From 2006 to 2011 recorded crime in Kent fell steadily, and theKent Constabulary won plaudits for their achievement. On the face of it, they were doing better than many other forces.


In 2011 police forces in Gloucestershire, Humberside, and Devon and Cornwall ‘no crimed’ just 2%, 5% and 10% respectively of reported rapes. It turns out that in the same year in Kent, the police decided that as many as 30% of reported rapes were not crimes, and did not need investigating.


Is it really likely that women in Kent are fifteen times more likely to lie about being raped than women in Gloucestershire? If it is not, then we must accept that a great many victims of rape and their families were dreadfully let down by the police in Kent. The cost to society of rapists being allowed to go free in the name of hitting targets is appalling – as even the police admit. A 1998 Audit Report by Her Majesty’s Inspector of Constabulary on the West Midlands Police said that

“The failure to record crimes prevents offences being investigated and offenders brought to justice. In a number of cases this has allowed criminals free rein to commit serious crimes, including murder”. [7
When Her Majesty’s Inspectorate of Constabulary cracked down on ‘cuffing’ in the West Midlands in 1998/9, they found that some 16% of crimes were being recorded in unofficial crime registers, data that was not taken into account in official statistics. The following year, with ‘cuffing’ discouraged, the number of recorded crimes increased by 16%. [8]


False detection: TIC

Cuffing is not the only way the crime statistics are ‘gamed’ to make a police force’s performance look better. There is an administrative procedure whereby the police can invite an offender to admit to additional offences and to ask for them to be ‘Taken Into Consideration’. This process is called a TIC. The TIC was introduced for two good reasons: it reduces the bureaucratic burden on the courts, and it allows a prisoner to make a fresh start after serving the sentence for the original crime, without fear of arrest for a past offences.

But let us imagine a case where a police force has a backlog of unsolved crimes and a detection rate that is falling below the targets set. They have an offender on a quite serious charge, who would like it to be changed to a lesser one. In such a situation, it is not unknown for the police and the offender to come to an unofficial agreement: if the offender admits to having committed a number of previously unsolved crimes, the police will reduce the charge. Sometimes, the offender is not so easily persuaded. Over the last few years evidence has emerged of police offering drink, drugs or sex in return for these fictional confessions, including in Cleveland in 2002, offering drugs and alcohol; the West Midlands Police in 2003 offering drugs; the Greater Manchester Police, Stockport in 2004 offering access to relatives; the Bedfordshire Police in 2006 offering access to sexual encounters; in Merseyside in 2007 access to alcohol, and the same in South Wales in 2012. [9]


It is hard to say how many TICs are the result of corrupt practices, but when in 1995, for example, the Chief Constable of Greater Manchester Police clamped down on one kind of TIC, detection rates for car crime and burglary fell by two thirds. [10]. It would be reasonable to infer that some two thirds of some crimes were ‘solved’ thanks to TICs .


This was still going on nearly 20 years later: in November 2012 five officers from an offender management unit in Kent were arrested following allegations by another officer about the abuse of TIC procedures.


It also seems that there are no incentives to stop : in June 2013 Her Majesty’s Inspectorate of Constabulary published the results of its investigation into TIC procedures in Kent, but said that an internal investigation carried out by the force failed to uncover any criminal offences.


Crime has become what is reported as crime. Falsifying the reports was not reported, ergo it was not a crime, and the criminals were free to continue.



 Neglect of duty: Skewing

To the extent that the Police are charged with reducing crime, we might have expected them to take action. But the inducements to take action selectively are great. In 1999 Her Majesty’s Inspectorate of Constabulary reported that,
“There was evidence in one force that a divisional commander refused to allow his detectives to put more than minimal resources into a serious sexual crime investigation, preferring instead they concentrate their efforts on less serious crime such as car theft. This occurred because whether they solved a rape or the theft of a car radio, the division would only be credited with one detection.” [11] 
When gang warfare and related drug and gun crime became a major problem in the West Midlands in 2012, police were deployed to areas with crimes that were easier to solve. For example, in Derbyshire in 2012,
“Tarique Gaffur, a former Assistant Commissioner of the Metropolitan Police Service, suggested that a strategy to respond to the threat of gangs by concentrating police resources on the deprived areas of the capital had not been implemented as ‘it would have taken officers away from the politically favoured policy of neighbourhood policing’.” [12]
The kind of policing that the politicians prefer is one issue; the callous refusal to act on the difference between major and minor crimes is another:
“A police force failed to investigate properly the violent robbery of a show jumper which led to her murder because its officers were busy inquiring into stolen chickens.” [13]
This was not a one-off: Kirk Reid, a south London chef raped and sexually assaulted more than 71 women over eight years. He had been identified as a suspect for a series of sex attacks in 2004 and crossed the police radar at least 12 times, but no one pursued inquiries into him. He went on to attack at least 20 women. As Patrick says, “The investigation into the way the Met dealt with the handling of the serial rapist Kirk Reid also indicated that other performance-related priorities had hampered the case.” [14] The Independent Police Complaints Commission report on the case contained testimony from senior members of the specialised sex crime investigative Sapphire unit in Southwark which described a department in crisis because management - who were concerned with hitting national targets - considered car crime a higher priority than rape and sex offences.

A delegate at the delegate at the 2007 Police Federation of England and Wales Annual Conference who was an operational officer in the Metropolitan police, told Dr Roger Patrick the retired Chief Inspector of the West Midlands Police,

“Every borough is playing the game; those that are not are seen as under-performing. Policing has completely lost its way. We only investigate crimes that matter in terms of performance data.”[15]
The catastrophic effects of the culture of New Public Management had been pointed out well before: In October 1993, the then Deputy Principal of the London Business School, Andrew Likierman wrote warning of potential problems. In 1994, the Oxford handbook of Criminology said that,
“Criminal statistics had to be analysed as the product not of a neutral fact-collecting process, but of a… process which is geared first and foremost to…police aims and needs.” [16] 
 In 1999 Her Majesty’s Inspector of Constabulary’s reported on police integrity, describing various types of ‘gaming’ it had uncovered. In 2003, the Chief Constable of Thames Valley Police warned the Public Administration Select Committee that police gaming had become ‘administrative corruption’ .

Roger Patrick the retired Chief Inspector of the West Midlands Police, was asked in 2014 what the true level of crime is. His answer was that we simply do not know. He gave two reasons. The first is that

“The police data is being manipulated to such an extent that it amounts to little more than sophistry. [17] 
The second is that the Crime Survey for England and Wales (CSEW) only covers a limited range of offences such as burglary and vehicle crime and is reluctant to include such white collar crime as credit card and internet fraud.

When finally the Public Administration Select Committee published its report on crime statistics in April 2014, tellingly it had the title, Caught red-handed: Why we can’t rely on police recorded crime statistics. In it, they said,

“The attitudes and behaviours which lead to the misrecording of crime have become ingrained…leading to the subordination of data integrity to target-chasing.”[18]
and later
“The Home Office, which claims credit for abolishing national numerical targets, should make clear in its guidance to PCCs that they should not set performance targets based on police-recorded crime data as this tends to distort recording practices and to create perverse incentives to misrecord crime. The evidence for this is incontrovertible. In the meantime, we deprecate such target setting in the strongest possible terms. Police-recorded crime data should not be used as the basis for personal performance appraisal or for making decisions about remuneration or promotion. We regard such practice as a flawed leadership model, contrary to the policing Code of Ethics.” [19] .
But as happened over and over again, this advice is being ignored: In October 2014, Mark Simmons, Deputy Assistant Commissioner, Territorial Policing proudly told Boris Johnson the Mayor of London,
“At the start of the year we set Lambeth a 7% reduction in burglary as their staging post…. Merton, for example, had a 5% reduction target. For Richmond we set a 9% reduction target on burglary.” [20] 

Of course the Police have to be accountable and their performance assessed. But the introduction of New Public Management theory to the management of the Police has resulted in a situation where no accurate performance assessment is possible, where no one knows any longer how much crime is being committed, where criminals are bribed to confess to crimes they have not committed, where resources are deployed to solve easily detected minor offences at the expense of serious crimes, and where rapists and murderers are allowed to continue to rape and kill.


We have to ask, why it is that all this has been reported and acknowledged at the highest levels, and yet virtually nothing has been done, indeed is being done, to make change happen.


Again, it must be argued that this behaviour, flying in the face of all logic and evidence, ignoring the bigger picture, is absolutely consistent with what we expect of cults.


What is artificially stupid about this behaviour is measuring performance out of the broad context in which it occurs . I'll look at the significance of this in a later post.  Meanwhile, I am reminded of this old joke. 
One night my friend sees a drunkard on his knees under a streetlamp feeling the ground with his hands. “What are you doing?” asks my friend. “I ’ve dropped my car keys and I can’t find them.”  “Where did you drop them?” asks my friend. “Over there.” says the drunkard pointing away into the darkness.  “Well, why are you looking for them here?” “Because the light is better here.” 



[Go to next chapter   NPM and Education]


* Peters would later deny that he said he had faked the data, although he admitted to approving the text of the interview in which the phrase appeared.


**  See The Technology Behind the World Cup's Advanced Analytics published by Vice magazine  in June 2014.  The technology has advanced significantly since then.  
Much of the information in this post is derived from A Tangled Web: Why you can’t believe crime statistics’ by Dr Rodger Patrick, formerly Chief Inspector of West Midlands Police and published by CIVITAS in December 2014. His PhD was originally going to be an assessment of the impact of Performance Management on police effectiveness. However, when it came apparent that the data was virtually meaningless because it had been so distorted by gaming-type practices, the focus of the research then shifted to the identification, categorization and measurement of ‘gaming’ practices and the implications for the governance and regulation of the police.



[1] Peters, Tom J “Tom Peters’s true confessions” in Fat Company at https://www.fastcompany.com/44077/tom-peterss-true-confessions November 30 2001 Accessed December 28 2019

[2] Goodhart, Charles “Problems of Monetary Management: The U.K. Experience” in “Inflation, Depression, and Economic Policy in the West.” Courakis, Anthony S. (ed) Barnes and Noble Books, Totowa NJ USA 1981



[3]   Football Association Premier League Ltd,  https://www.premierleague.com/players/4288/player/stats?co=-1&se=79   https://www.premierleague.com/   Accessed 12 February 2020

[4] Williams, C. A. (2007). Rotten Boroughs: the crisis of urban policing and the decline of miunicipal independence 1914-64. In J. Moore, & J. Smith (Eds.), Corruption in Urban Politics and Society, Britain 1780-1950.UK: Ashgate.

[5] Loveday, B. (2006). Policing performance: The Impact of Performance Measures and Targets on Police Forces in England and Wales. International Journal of Police Science & Management, 8(4), 282-293.


[6] Levy, Andrew. “How police plan to cut crime: They won't count vandalism.” 6 May 2008.Mail Online. 15 March 2015 <http://www.dailymail.co.uk/news/article-564109/How-police-plan-cut-crime-They-wont-count-vandalism.html>.


[7] (Her Majesty’s Inspector of Constabulary 1998) (Patrick, A Tangled Web: Why you can’t believe crime statistics 2014)

[8] Her Majesty’s Inspector of Constabulary. (1998). Audit Report on the West Midlands Police.West Midlands Police. HMIC

[9] Hansard. (2002, March 6). Publications and Records.Retrieved May 22, 2015 from http://www.publications.parliament.uk/pa/cm200102/cmhansrd/vo020306/debtext/20306-32.htm

[10] Her Majesty's Inspectorate of Constabulary. (1997). Greater Manchester Police : 1997/98 primary inspection report. London: HMIC.

[11] Her Majesty's Inspectorate of Constabulary. (1999). Police integrity : England, Wales and Northern Ireland ; securing and maintaining public confidence.The Home Office Directorate.
[12] Leppard, D. (2011, August 14). “Met scrapped plan to swamp rioters” . The Sunday Times.

[13] Foggo, D. (2006, December 10). “Police too busy chasing chickens to stop murder”. The Sunday Times

[14] Patrick, R. (2012, June). Home Affairs Written evidence submitted by Dr Rodger Patrick [IPCC 03].Retrieved April 5, 2015 from www.parliament.uk: http://www.publications.parliament.uk/pa/cm201213/cmselect/cmhaff/494/494vw03.htm

[15] Patrick, Roger. A Tangled Web: Why you can’t believe crime statistics. London: CIVITAS, 2014


[16] Maguire, M. “Crime Statistics, Patterns, and Trends: Changing Perceptions and their Implications.” in Maguire, M, R Morgan and R Reiner. The Oxford Handbook of Criminology. Oxford: The Clarendon Press, 1994.


[17] (Patrick, R  “Home Affairs Written evidence submitted by Dr Rodger Patrick [IPCC 03].” June 2012. www.parliament.uk. 5 April 2015 <http://www.publications.parliament.uk/pa/cm201213/cmselect/cmhaff/494/494vw03.htm>

[18] Public Administration Select Committee. “Caught red-handed: Why we can’t rely on police recorded crime statistics.” 09 April 2014. www.parlianament.uk. 15 April 2015 <http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/news/crime-stats-substantive/>

[19]   idem

[20] Mayor's Office for Policing And Crime (MOPAC) Challenge Quarterly Performance Review October 21 2014 at https://www.london.gov.uk/sites/default/files/14-10-21_mopac_challenge_-_performance_vwi_transcript_formatted_2.pdf accessed December 28 2019