When a project has failed, why continue investing in it? This is an important question for large organisations such as the NHS, which has a history of investing “vast amounts of taxpayer’s money into unrealistic and ultimately unsuccessful projects,” says a new report.

The National IT Programme for the NHS, Individualised Patient Choices and Mental Health Services Reform are three costly examples of the NHS failing to quit while it was ahead, say study authors Marianna Fotaki, professor of business ethics at the University of Warwick, and Paula Hyde, professor of organisation studies at Durham University, reporting in the journal Human Relations.

Examining the reasons behind escalating commitment in the face of failure, they say that organisations such as the NHS develop blind spots due to “a perfect storm of unworkable policies and defensive behaviour.”

Unconscious social demands – such as the expectation that a well-run health service can prevent disease or death – often underpin unrealistic policies, and the very fact that society expects organisations to address such large and intractable problems means that failure is inevitable, say the authors. But the policies themselves demand that organisations will remain committed.

Certain groups within organisations, such as clinicians or patients, are often acutely aware of problems, but their voices can be cut off through “splitting” and blame. “Splitting” is a defence which helps people cope with doubts, conflicting feelings or anxiety; they separate negative and positive feelings, no longer recognising an object as a whole but as separate parts, idealising the “good” part and seeing the “bad” part as carrying the potential to cause harm.

Splitting in the system, mostly between policy and its implementation, enables idealisation of the task, which then becomes an aspirational “grand project,” causing organisations to abandon the very task they have been created to fulfil, say the authors.

The National IT Programme for the NHS, the world’s largest civil IT project with unprecedented investment beginning in 1999, was finally abandoned in 2011 after the National Audit Office cast serious doubts about the wisdom of ploughing further money into it. Here, the necessary step towards creation of blind spots was the push to invest in the scheme without consulting or collaborating with clinical staff or learning from past failures, say the authors. Clinicians who expressed concerns about the Programme were criticised for failing to embrace much-needed change and were only included in the process once it hit difficulties.

An important implication of the study concerns the inability of power-holders and/or policymakers to recognise the origins of blind spots in such overly-ambitious policies, as well as their own emotional investment in these policies, says Prof Fotaki.

“There is enormous pressure to demonstrate success, often fed by public scrutiny. Such intensification of commitment to a chosen course of action, driven by a desire to avoid humiliation associated with failure, may lead to greater and greater material losses,” she warns.