'The public sector is on its knees - of course AI sounds appealing'

“We know that AI will also be a source of disruption to the way we live and work, and a multiplier of risk. We’ve all heard the warnings – which we must heed. I know how worried people are. So we have to rise to this moment,” Prime Minister Andy Burnham told the UN General Assembly in New York last month.

In his first major international trip, which included a meeting with President Donald Trump during which artificial intelligence was not discussed, Burnham set out how his government wanted to act as the “honest broker” as the world deals with the opportunities and dangers of AI. The UK will take on the G20 presidency next year, with a summit to be held in Manchester in November. AI, Burnham said, would be “at the heart” of it.

“We will bring the right people together on the key issues: delivering on the potential of AI for good, and dealing with the risks,” he continued. “And we will work towards agreeing a single set of global principles and standards to ensure the transparency and access we need to increase our preparedness – and to ensure that AI development is safe.”

The speech came after weeks of mixed headlines on AI and what it might mean for the future.

Jacob Coxon, a 27-year-old ex-Anthropic researcher, had taken to Twitter at the start of the month to warn the tech “could kill us all by the end of the decade”. Tech bosses – including Elon Musk, OpenAI’s Sam Altman and Anthropic’s own Dario Amodei – called for the pace of development to be slowed.

People, not AI systems, remain responsible for decisions

Others dismissed the warning. Nvidia’s Jensen Huang said it was “irresponsible” to set out “doomsday narratives” which were “not grounded in science”. Trump described the fears as a “hoax” and appeared to rule out regulation, writing on TruthSocial that his government would not “in any way hinder or stifle the growth of this incredible industry”.

Yet for all the furore, artificial intelligence is already being rolled out by companies and governments across the world. The Scottish Government is no exception.

Scotland’s AI strategy was published earlier this year. It committed to ensuring the country “remains in the vanguard of this new technology in the years ahead”, while also acknowledging the “wider risks and uncertainties that come with the rapid evolution of AI”.

AI featured heavily in the new civil service strategy, Serving Scotland, published last month. “AI and automation is already removing repetitive, manual work from our processes, freeing skilled people to focus on what needs human judgement. But we are not yet using this to its full potential, to make the best use of the resources we have,” it said. It went on to promise the development of AI agents, as well as “learning and support” so staff could use them “responsibly and effectively”.

AI could be implemented across public services in order to reduce that administrative burden

But fresh guidance from the government’s digital directorate was also published in September, warning that “people, not AI systems, remain responsible for decisions” and the tech should not be used “where decisions cannot be explained or properly reviewed”.

It’s a careful balancing act for decision makers, who want to be able to reap the rewards of new technology but are also aware of the potential risks. Those risks include but are not limited to cybersecurity, data privacy, job losses, environmental issues and bias. While the regulation of AI is a reserved area, that does not mean the Scottish Government does not have a key role, particularly when it comes to how the tech is deployed across public services.

StormID, an AI solutions firm that already works with a number of public bodies, last year produced a paper estimating the technology could save millions of hours in staff time if rolled across public services. The paper, titled ‘Automate tasks, not jobs’, estimated that a low-adoption scenario would see 16.6 million staff hours released, while a high-adoption scenario could give 62.1m staff hours back per year by 2030.

Company founder Paul McGinness says many of the efficiencies to be made are focused on lifting the “administrative burden” currently placed on staff. In practice, he says, that means reducing the time healthcare staff spend on preparing clinical documentation, teaching staff spend on lesson planning, or staff in the justice sector spend on analysing case documents.

“A lot of that yields quite a significant administrative burden for people who are very well trained, and they didn’t necessarily train to become people to do the admin work,” McGinness says. “Our proposition is that AI could be implemented across public services in order to reduce that administrative burden, release capacity back into the system and give public sector professionals time back to do the work that only they can do.”

But, he says, that will only work if services are redesigned and staff are brought along with the adoption of any new technologies. “AI on its own doesn’t achieve anything. You’ve got to redesign services; you’ve got to change how people work and change people’s working practices in order to accommodate and make the best use of AI. Without that, AI will not have the effect,” he says.

The research paper outlined five common threads running through the work of 50 public services which could benefit from automation: case management; application processing; triage; scheduling; and knowledge-intensive documentation. It recommends the Scottish Government build shared and reuseable AI components that could be used across services.

Most organisations don’t have an AI problem, they have a process problem, a data problem or a capacity problem

And McGinness says there are already some good examples of this. “The Scottish Government has done some really good work already around building shareable, reusable components that can be integrated into different public services. An example of that is the identity service that they’ve built, which is now used across a number of different public services, including the new NHS app. Another example is their payment service, which is in use across a few different areas, including social security. One that we’ve helped them with is around their forms engine, which is now used in planning and in the pensions agency.

“These kinds of reusable components, they’ve got really good history of building them and then they build them once and then they can be used in multiple services. Our view is that [government should] continue that strategy and invest centrally in building shareable AI components across those five patterns that we identified. You do the work once around policy, around guardrails, around safety and so on, and then other public sector bodies can benefit from it by integrating it into their services.”

What AI isn’t there to do, he adds, is to replace the roll of human decision making. This means taking a “risk-based approach” to deployment, working out what parts of services can and should be automated, and understanding “where do we need to keep a human in the loop”. This, he adds, will also help to build trust in AI. “Then over time, as you build more confidence in the decisions or the outputs the AI is providing, you can move that dial further and say, it can automate more,” he says.

While he acknowledges some of the concerns, he argues that good service design and quality assurance – the same as using any new technology – will mitigate risks. He also argues that too often, sceptics of AI are “moving too quickly” to talk about the dangers of AI making decisions in sensitive areas, like clinical decisions or in the justice system. “That’s so far away and there’s so much other stuff to deal with before we get to any of those decisions,” he says.

No other technology or product gets awarded that freedom

Paul Duffy, chief executive of another AI firm Zudu, agrees. “I’d be cautious anywhere a decision has serious consequences for an individual and can’t easily be explained or challenged. There are some sectors at more risk than others. Benefits eligibility, criminal justice, medical diagnosis without clinical oversight, hiring decisions – these are areas where AI can support people, but it shouldn’t be making the final call on its own.”

He goes on to argue that accountability “can’t be outsourced to a vendor or an algorithm”. “It starts with treating AI like any other system that touches sensitive information, not as something magical that sits outside normal governance. The basics still apply: know where your data goes, who can access it, and what the tool is and isn’t allowed to do.”

But more important than thinking about how to ensure AI is deployed safely across organisations is whether it will solve the issue in the first place. “The problem is that the conversation usually starts with the technology rather than the problem,” Duffy says. “Most organisations don’t have an AI problem, they have a process problem, a data problem or a capacity problem, more often than not one that has existed for a long time, and AI is sometimes the right answer to it.”

He agrees with McGinness that the gains to be won from AI are often “unglamorous” – like reducing admin or the more efficient sharing of information. “What we see is an execution gap. Plenty of organisations have bought licences or have run a pilot, and very few have turned that into measurable change in how they work. Closing that gap means diagnosing the problem properly first, being honest about whether AI is the right tool, and measuring the outcome in terms people outside the IT department care about – time saved, money saved, better service.”

Steph Wright, co-founder of Our AI Collective, is more sceptical about its benefits, though she agrees with Duffy that too often “we’re starting with the technology and going, ‘what problem could it possibly solve?’” rather than identifying the problem first. Her simple test is asking organisations using or thinking of using AI: why?

“They need to be able to answer and say, we have this problem to solve and we considered these potential solutions and AI is the best possible solution for us. We’ve done the due diligence to ensure that it protects our staff, it protects our clients, it protects our users if we use it in this way, and these are the clearly defined benefits,” she says.

This, she says, is not often the case. “The amount of companies I’ve spoken to that rolled out Copilot, rolled out ChatGPT, or rolled out Claude, and then thought, ‘maybe we should have a policy in place’. No other technology or product gets awarded that freedom.”

The public sector is on its knees. There’s no money, it’s stretched, it’s under-resourced... Of course AI sounds so appealing

She argues the “hype” around AI has meant too many companies and governments are worried about falling behind, creating a rush to adopt it. “It’s all about ‘we need to build, we need to innovate, we need to progress’. But I always like to remind everyone that digital isn’t always better. Innovation isn’t always progress.”

She continues: “You’re just being told, economic growth – for whose benefit? Scotland has so many problems with its education, its healthcare, child poverty, violence against women and girls. They’re not going to be fixed by AI or technology as a whole.”

Meanwhile, the negative impacts of AI’s use are already being felt around the world, she says. And the recent warnings from tech companies about a “speculative, hypothetical future threat” is “complete and utter distraction” from the “real proper harms that AI technologies are doing right now”. She points the role of AI chatbots being cited in court cases relating to self-harm, psychosis and suicide, or the involvement of AI firms in military operations, such as airstrikes in Iran being carried out with the assistance of AI. In a Scottish context, she highlights the environmental impacts of the several proposed hyperscale data centres.

That’s not to say she doesn’t think it can be useful. One area AI is especially good at it sifting through large amounts of data and finding patterns – a use that could clearly support the work of public bodies which hold a lot of data. But she adds that there is a danger of outsourcing decisions which should ultimately be made by humans.

“We are increasingly delegating and absolving ourselves of responsibility to the illusion of efficiency and productivity. The public sector is on its knees. There’s no money, it’s stretched, it’s under-resourced, and it’s only going to get worse. Of course AI sounds so appealing,” she says.

But she argues it’s important to go back to the basics. On the claim the AI can be used to automate bureaucratic tasks, for example, she argues the real question is whether these tasks are necessary in the first place. “If it can just be automated and got rid of, then why is it there?”

Her advice to government would be to “properly examine what the problems are and  look at how they can be solved before leaping in with a technology solution”. “I’m not anti-technology,” Wright adds. “I used to be a total tech-optimist. I used to think technology ultimately can be beneficial for society as a whole, but it needs to be used intentionally, deliberately, in clearly defined ways with wider regard to its impacts on everyone else.”

Ultimately, decision makers need to ask whether it aligns with people’s values, she says. “I always say let’s apply a lens of care, instead of a lens of profit – which is the primary lens through which all these conversations are framed. But the government is not responsible for making profit for the private sector. The government is there to create a better country for the people it serves. They are public servants. They serve the public. So what is the benefit for the public? And has that been clearly defined?”