How We Risk Learning the Wrong Lessons from the Horizon IT Scandal

Since the start of 2024; the Post Office Horizon IT Scandal has reached a new level of public interest in the UK. In the scandal, faulty accounting software has been blamed for multiple suicides and what has been described as “the most widespread miscarriage of justice in UK history”, with those wrongly imprisoned including a pregnant woman.

Since late last year, I have worked on addressing this scandal from multiple fronts. For example; in November 2023 research I led found that 75% of software engineers in the UK faced retaliation the last time they reported wrongdoing and also shed new light on gagging clauses used by the Post Office in the wake of the Horizon IT scandal amongst a myriad of other findings.

In this article, I want to explore how the British Computer Society, which presents itself as the voice of software engineers in addressing the scandal, may well be teaching the wrong lessons to society.

Regulatory Failings

In January 2024, I led a public interest investigation which found that the British Computer Society (BCS) failed to act when one of its members used their regulated status to convince courts to engage in miscarriages of justice; despite the status provided by the BCS was the only computer qualifications the individual presented to the court and the BCS being required by the Engineering Council UK to uphold the conduct of its members.

Following the BCS being asked for comment, the BCS were disingenuous with the media by claiming they would take action after the long-drawn-out legal processes were completed when the evidence I’d uncovered indicated that the key individual's BCS membership has likely already long lapsed through non-renewal. The individual concerned (Gareth Jenkins) is currently a person of interest to the Metropolitan Police in their investigations into the scandal and had requested criminal immunity so that his upcoming evidence to the public inquiry would not be used against him.

This information was obtained after I gained sight of part of a Post Office witness statement marked “confidential” and was corroborated by a report to Post Office Ltd marked “legally privileged and confidential” by a lawyer, Brian Altman KC:

Following a Freedom of Information Act request I made recently, Post Office Limited confirmed they held a copy of a witness statement from the individual concerned - further corroborating the information:

The BCS is far from the only regulator involved in this matter. Seldom mentioned in the media is the fact that the Post Office is regulated by the Financial Conduct Authority (FCA), who operate some bureaucratic rules to attempt to regulate the UK’s financial sector.

A recent investigation by Reuters which I contributed to found that the FCA had been assessing Freedom of Information Act requests differently when they came from journalists. It has also been reported the FCA "dismissed complaints from a whistleblower and allegedly left them open to a barrage of retaliation from their former employer after officials wrongly interpreted the law".

For the Post Office to bring their own prosecutions, they engaged the services of the highly-regulated legal field, where professionals are regulated by bodies like the Solicitors Regulation Authority and Bar Standards Board.

In all these instances, from the British Computer Society to the legal profession, the regulatory bodies failed. The victims of the Horizon IT scandal’s first taste of justice was when a journalist from ComputerWeekly began to cover their story as Alan Bates led a campaign to get justice for his colleagues.

Calls for Regulation by the BCS

Without shame; the British Computer Society is now calling for AI "to be regulated to avoid its own Post Office Horizon Scandal" by requiring practitioners to be licensed - seemingly an opportunistic attempt to capitalise on a scandal, when they regulated the professional qualifications used to convince courts to engage in this miscarriage of justice. It is additionally ironic that whilst the BCS refuse to commence disciplinary action against their members until the public inquiry is complete, they seem perfectly content to begin providing recommendations to society.

However, at its core, the calls for regulation fundamentally misunderstand how such disasters are prevented. This can be seen by the sheer weight of regulation the Post Office was under in these cases, yet miscarriages continued to occur.

Dr Ron Westrum wrote in the British Medical Journal’s Quality & Safety publication in 2004 a paper entitled “The Three Typologies of Organisational Culture”. The following table demonstrates these three organisational topologies, describing how different organisations process information:

Generative cultures are “psychologically safe” - they focus on outcomes and people are free to raise the alarm when things go wrong, rather than being shot. By contrast, pathological organisations are those where failure leads to scapegoating and messengers are shot. However - bureaucratic organisations are hardly desirable either. Messengers are neglected and rules take priority over addressing the causes of failure. For poor leaders, bureaucratic management is the easiest - instead of changing culture they pull the lever of more rules rather than addressing the issues.

From my experience of looking at the Horizon IT Scandal and other catastrophic software failures - many of the organisations or regulators involved either were pathological organisations or bureaucratic. However, a generative culture would have allowed one of the insiders who raised concerns about the Horizon IT problems (like David McDonnel did) to have their voices listened to and their concerns investigated.

Regulation of software engineers may help gatekeep some from the software engineering profession, but to address the real issues of the Horizon IT Scandal - we need to develop more generative culture organisations.

Legislation must be part of the answer in strengthening protection for software engineer whistleblowers who raise the alarm to serious issues (as protections in the UK must be strengthened) and some form of regulation may be part of the answer to addressing some of the issues, however, we should not pretend that rule-oriented cultures will suddenly lead to the psychological safety needed to stop these scandals - history, indeed in this very case, has proven that it does not.

Update: 21st February 2024 - Response from the BCS

On Monday, 19th February 2024 - the Director of Communications at the British Computer Society contacted me about this article. They wished to raise two points.

Firstly, the BCS said “it's important to note that BCS is not a regulatory body” and the “BCS does not have regulatory authority”. However, these claims appear to be contradicted by the fact that the UK Government lists "Chartered Engineer" on the Regulated Professions Register. Additionally, the Engineering Council UK lists a number of activities which are restricted by laws and regulations from being carried out without Chartered Engineer status. As a licensed Professional Engineering Institution of the Engineering Council UK, the BCS can award such status (as Mr Jenkins presented to court) and should uphold the conduct of members through its code of conduct.

I asked the BCS how they reconcile their statement that the "BCS does not have regulatory authority" with the fact that the BCS has the power to issue regulatory status - but have not yet received a response.

Secondly, the BCS stated: “Following the completion of the inquiry and any other relevant legal proceedings, any confirmed BCS members who are found in breach of the code of conduct may be subject to disciplinary actions. However, it's essential to understand that these actions would be within the framework of BCS's internal policies and procedures, rather than regulatory enforcement.” I found it curious that the BCS refer to “confirmed BCS members” as the use of the phrase “confirmed” could imply there was a question as to whether Mr Jenkins was actually a BCS member.

However, what the BCS may not have been aware of was that I was able to verify that Mr Jenkins did have a BCS membership. This is because the BCS register of Chartered IT Professionals does not appear to remove members when their membership lapses - accordingly Gareth Jenkins is still listed as a Chartered IT Professional there, registered via the BCS:

Mr Jenkins does not however (still) appear on the Engineering Council UK’s register of Chartered Engineers - which led me to understand his membership had lapsed.

I shared this with the BCS and asked them the questions: “Is it therefore the case that the way the BCS is trying to manage this from a PR angle to raise doubt around Mr Jenkins' membership status - but when the inquiry reports to take the position that no members of the BCS were involved (despite him being a former member, and the BCS not taking any action at the time)? What is the approach taken for former members?” However, I have not yet received a reply.

I also asked the BCS about their rationality for waiting until after any legal proceedings are complete before taking any regulatory action - but again have not yet received a reply: ‘When the BCS states it will wait for "the completion of the inquiry and any other relevant legal proceedings" - what is the basis for the BCS doing this? Could you share the policy basis? Does the BCS make any exemptions for instances where there is a risk to the public or a risk of serious and imminent harm?’

It is also interesting to note that evidence exists that someone may have written to the BCS to raise their attention to this matter in previous years to ask them to take regulatory action.