Future public inquiries such as the recent Alex Salmond affair at Holyrood could be jeopardised by poor information management practices, a government internal review has found.

The way that the Scottish Government stores information across departments is ‘inefficient’, difficult to access and use, leading to a widespread use of ‘alternative’ ways of documenting information among staff.

Furthermore there is a ‘culture of apathy and negativity’ towards a legacy electronic records and document management system that some staff haven’t had training on since its adoption 15 years ago.

In responding to official inquiries, dedicated personnel have been drafted in to provide ‘greater surety’ that the results would ‘complete and accurate’, according to A Review of Information Management in the Scottish Government, published last month.

As such official government responses to high-profile inquiries including into the Scottish Government’s handling of harassment complaints, the Sheku Bayou inquiry and the Scottish child abuse inquiry were found to be ‘generally reactive and tactical in nature’ and with teams working to ‘overcome the complexity and inefficiencies inherent in the current information environment’.

With a Covid-19 inquiry described as ‘inevitable’, the audit found that only 28 per cent of government information resides within searchable systems which could pose ‘significant information risks’ around legal compliance – in line with the Public Records Act and in responding to Freedom of Information requests – as well as performance, security and ‘sustaining a non-corporate culture’.

The review highlighted four previous government initiatives to try and sort out long-standing concerns around the way information is stored and managed – two at UK level and two business cases launched by the Scottish Government in 2017 and 2019. The latest findings “remain applicable”, the report says, describing the collective picture as a ‘tipping point’ for the organisation to ‘re-evaluate and reset its approach to information management’.

Scottish Conservative Chief Whip Stephen Kerr said: “This report raises serious questions for the SNP Government. We see all too often that they try and use every means possible to avoid scrutiny and now it appears they can’t even get their own house in order when it comes to storing data.

“With technology continuing to develop at a rapid pace, it is imperative that the SNP Government urgently act on the findings of this report. If they produced an improvement plan four years ago, then why are serious problems still being highlighted?

“We have seen far too many failures in recent months from SNP Ministers when it comes to transparency, especially in relation to the Salmond inquiry. This report should be a wake-up call to ensure data systems are running as effectively and openly as possible.”

Scottish Liberal Democrat MSP Alex Cole-Hamilton, who sat on the Salmond inquiry, added: “The Scottish Government’s record keeping frequently left the Salmond inquiry committee cursing. Documents were provided late or only appeared after multiple requests.

“But this problem is wider than just one inquiry. The Scottish Government needs to overhaul its information management if trust in the system is to be rebuilt.”

According to the review most of the information the government holds on its SCOTS network cannot be readily accessed or searched. Of the five systems – including two drives, a public folder and email – it is only the electronic records and documents management (eRDM) system that is designed to be a corporate information system. All the others, the report notes, are ‘repositories with largely unstructured and unaudited information which is not subject to sufficient business governance especially from a corporate perspective’.

An analysis from the iTECS (Information and Technology Services) division found that there were – as of January this year – 155 million government emails, 38 million items of the eRDM system and a combined 95 million items stored on the two drives and public folder. In addition to the sheer volume of information, the audit noted that there was ‘likely to be significant duplication of information across all of these repositories’.

The sheer volume of unstructured information lying across a
multitude of disparate repositories and sources amounts was described as ‘a corporate risk in its own right’

Some work has been carried out to improve the way information is stored and retrieved including a recently implemented eDiscovery solution. The software has improved the capability to reliably search unstructured content and provide greater assurance about the outputs. Significant recent investment to modernise key corporate systems and services – including an automation-based workflow engine – have also led to improvements. And the review notes the progress in digital transformation brought about by the Covid-19 pandemic, which led to the rapid deployment of Microsoft Teams to 16,000 staff.

But it cautions: “Information and records management should not be afterthoughts in the delivery of new technology. The development of clear information management guidance, policy and governance should be planned in from the outset as core dependencies and workstreams in any projects delivering new or upgraded digital information systems. In essence the SG should do all it can to avoid creating a ‘digital heap of tomorrow’ containing poorly structured, poorly managed and difficult to find information which is created and managed in a non-corporate manner.”

In total, eight recommendations were laid out in the report – including improving corporate culture, developing a whole of government information management strategy and phasing out the use of network drives as information repositories – which the government has pledged to adopt in full.

A Scottish Government spokesperson said: “The need to store a huge increase of information electronically in recent years has brought benefits and significant challenges to government organisations in the UK and globally, including the Scottish Government.

“We have made significant investments in information governance in response to Cabinet Office information governance reviews in 2015 and 2017. As a result, we improved processes for interrogation of corporate information, including enhanced document management practices and improved digital storage and tools.

“A programme of work to implement the eight recommendations of the Review has already started and will continue to deliver improvements before completion in 2022.”