The AI industry needs to ‘professionalise’ in order to provide confidence that the rapidly-evolving technology is safe for public consumption, according to Britain’s chartered institute for IT.
Similar to highly-regulated industries where safety is mission-critical – such as medicine and aviation – the developers of AI products and services need to be guided by clear ethical standards and safeguards.
Emma Logan, the deputy president of the member organisation British Computer Society, told the annual ScotSoft conference in Edinburgh today that we are now reaching a point where assurance is required.
She said: “Professional judgment matters within a wider framework of rules, standards, and accountability. The same principle runs through other professions. We require doctors to spend years training and to work within clear professional regulatory standards, because bad medical judgment can ruin lives.
“Lawyers, architects, engineers, all have qualifications, codes, oversight, and responsibilities which go beyond simply being technically capable. These structures emerge because society learns, sometimes painfully, what happens when expertise is untested or accountability is absent.”
Logan referred to a number of recent instances of rogue ‘frontier AI’ agents taking over systems. The Hugging Face incident in July saw 1,200 OpenAI agents start to communicate autonomously, before exploiting loopholes in their training infrastructure and building a secret “message board” within a shared repository to collaborate.
Roughly 700 of the agents joined forces to target Hugging Face, the world’s most popular open-source platform and community hub for artificial intelligence and machine learning.
The incident – described by industry insiders as a ‘wake-up call’ – is just one of an increasing number. Today, the Australian Government revealed that OpenAI agents had similarly hacked into the country’s national healthcare data wesbite. Prime Minister Anthony Albanese complained directly to OpenAI boss Sam Altman, he revealed at the United Nations in New York, with one of the chief criticisms being the delay in being notified about the incident. The hack occurred in June, which OpenAI said it only discovered in August.
At today’s event in Edinburgh, Logan pressed the case for greater urgency in tackling the fast-evolving industry. She said that the BCS is leading the government’s ‘AI assurance stakeholder consortium’, bringing together government and industry, standards bodies, and practitioners to ‘help build some of the foundations for this emerging profession’.
She also shared some statistics from a newly-released AI skills and adoption survey carried out by the BCS. The survey found that 35 per cent of BCS members rated AI as the ‘single biggest skills gap’, with seven in 10 putting it in their top three.
She said: “So, for all the understandable excitement about AI, the people closest to its implementation are telling us something quite simple: organisations still do not have enough people who know how to use it well, and that gap is not only about knowing how to build a model.
“The skills our members talked about ranged from agentic AI development and process automation, right through to something much more basic and, I think, much more important – the judgment to question what an AI system produces, rather than simply take it on trust.”
The call for formal standards from the BCS is vital, Logan said, because the public need assurance that what they are using is safe.
She said: “As the systems on which society depends become more complex and consequential, we build standards, professional expectations, regulation where it is needed, and independent scrutiny around the people responsible for them, I think the assurance of AI systems is now at a very similar point.
“Of course, it’s still an emerging field. People use the term in slightly different ways, but boundaries are not always neat, and the career routes are still developing. But at exactly the same time, organisations are becoming desperate for assurance. Boards want to know whether the AI they are buying is safe. Governments want confidence in systems being used for public services. Investors want to understand risk. Businesses want to know whether suppliers’ claims can actually be trusted. Members of the public, quite reasonably, want to know whether somebody has properly scrutinised systems that may affect their jobs, their finances, their healthcare.”
She said the survey further revealed that members themselves – who range from students to c-suite executives in the tech industry – were crying out for some form of ethical training for AI. Four out of five members said they would want some form of recognised ethical training or certification, and not merely ‘how to use CoPilot’.
She said: “Professionalism begins when expectations become real, when maintaining your competence is part of your job, and when ethical behavior is not optional, when you’re expected to exercise judgment rather than simply follow instructions, and when there are moments in which you’re required to challenge something, even if doing so is awkward.”
As part of the consortium, the BCS intends to play a role in helping to build some of the foundations of what she described as an ’emerging profession’.
Scotland’s new AI minister opened the ScotSoft event at the Edinburgh International Conference Centre with a call to support the country’s nascent AI economy. Alyn Smith said he wants Scotland to become the ‘natural place for businesses to set up and grow’.
He said: “I want to see Scotland as a supportive, natural environment from which you can set up your companies based on your research, collaborate with our brilliant universities and colleges to make sure that we’ve got the skills to make sure that we’re fit for the future.
“And I’ll be honest with you, we’ve got a way to travel. Scotland is not at the forefront of this…it’s not as aligned as it needs to be. So we’ve got the talent, we’ve got the skills, we’ve got the location, we’ve got all that everything we need. What we need more than that is to make sure everybody is joined up and collaborating across the silos, across the boundaries, and making sure that we’re focused on the outcomes, which is healthy, profitable companies employing good people in good jobs.”