Why the policy profession must embrace AI
Michael Padfield explains the benefits of a range of tools that will drive better strategy, greater democracy and improved delivery
Ever wondered if there’s a better way of distilling key information than ploughing through hundreds of lengthy documents? Ever groaned at the prospect of wading through mountains of consultation responses? Ever dreamt of being able to detect and track early delivery problems with a new policy, (almost) at the touch of a button?
The good news is that thoughts like these are no longer fantasies for policy professionals working in the UK Civil Service. New artificial intelligence tools will transform the policy profession for the better, enhancing decision-making and cutting out tedious, administrative tasks.
Some policymakers may be nervous about this future, wary of the black box of AI or sceptical of the Government’s ability to do this when many services remain paper-based or work off old IT. Such concerns are legitimate and need to be addressed, but they should not hold the Government back from exploring the potential of AI to make tens of thousands of jobs both easier and more efficient.
The future, many of us are convinced, is a policy profession empowered by AI, not replaced by it. As AI tools come to automate routine research, drafting and administrative tasks, the policy professional will have more time to cultivate crucial but distinctly human skills, such as relationship building, critical thinking and most importantly, good decision-making.
As Sir Geoffrey Vos, Master of the Rolls and Head of Civil Justice, told the Manchester Law Society in March 2024, embracing AI is not just a practical imperative. ‘We owe a duty to those we serve” he said, “to make constructive use of whatever technology is available if it helps to provide a better, quicker and more cost-effective service to the public.”
In this article I will explore AI tools that the Incubator for Artificial Intelligence (i.AI), an innovative delivery team in the Department for Science, Innovation and Technology, is building for civil servants across the three pillars of the policy profession: strategy, democracy and delivery. Some, as indicated, are still in the early testing stage of development but all, ultimately, will improve how policy is made, how policy is translated into legislation, and how policy can better the lives of citizens.
Strategy
Strategy, defined as ‘generating, assessing and applying evidence and analysis to understand the context…’, is the first pillar of the Policy Profession Standards framework.
Happily, the evidence and analysis function within government has improved significantly over the years. In paper-based bureaucracies of the past it was not easy to share knowledge across an organisation, draw relevant insights or overcome the power of selected officials who may have acted as unelected gatekeepers.
Thankfully, the digital age has democratised access to information. And yet it remains difficult to make sense of it all. A policy professional will spend many hours wading through documents to pull out the key lessons. Summarising information is exactly what AI is good at, which is why i.AI built Redbox.
Redbox
Redbox is a generative AI tool built by i.AI to provide civil servants with secure access to large language models. As of January 2024, Redbox is being trialled by over 1500 officials in the Cabinet Office. Currently available across the Cabinet Office.
Civil servants are already using Redbox to ‘chat’ with government documents, restructure information and generate first drafts of briefings, providing both productivity benefits and better policy outcomes. A current No10 Innovation Fellow working at the heart of government said the tool has ‘revolutionised’ how they carry out research, synthesising 50 documents in a matter of seconds into an insightful, reliable, bullet-point summary with headings. This is a task which would normally take a full day’s work.
‘Participation and engagement’ are also central to the strategy pillar of the Policy Profession Standards. However, while consultation should be a critical part of the policymaking process it is often seen to be a low value activity; analysis of consultation responses, in particular, can be costly, slow, and unscientific. With pattern identification another of AI’s core strengths, it was natural that the i.AI team should build Consult.
Consult
Consult is a Large Language Model (LLM) tool built by i.AI to draw out the key themes from consultation responses, currently in Alpha testing. Currently available for selected pilots.
With Consult, the policy professional will be able to draw quick and comprehensive insights from thousands of submissions, in a fraction of the normal time. The policymaker, armed with a stronger evidence base for change, will be more able to identify improvements to a policy. Early tests are very encouraging. An evaluation using real consultation data found that an analysis which was set to take a team of 22 officials 20 weeks could be done to the same level with Consult 1000 times quicker and 400 times cheaper.
Now imagine this applied to the 700-800 consultations the Government runs every year. And going further, think of how such technology could reshape the way the Government engages with the public on critical issues. Tools like Consult open up the possibility of a more dynamic, participatory democracy. The Government could test ideas more quickly and more often, iterate to make better policy, engage the public more broadly, and accelerate the pace of change. Consider its potential to speed up planning approvals, or better understand public opinion around sensitive issues like assisted dying. There is so much to play for beyond the existing, rigid consultation process.
Democracy
Legislation can seem impenetrable to non-lawyers, but it need not be so. In fact, it is crucial that policymakers understand the legislative context in which they operate. Hence, the democracy pillar of the policy profession, defined as ‘understanding and supporting good governance’, which includes ‘working with Ministers’ and ‘Parliament and Law’.
Knowledge of the statute book gives a policymaker clarity over the options for intervention, allowing for better decisions over whether and how to legislate. The Government can pride itself on its current search tool, legislation.gov.uk, which contains all Acts of Parliament since 1267, but i.AI has now built a new AI-powered search engine, Lex, which will radically enhance research of this legislative information.
Lex
Lex is an AI-powered search engine built by i.AI in collaboration with the Ministry of Justice, to accelerate the process of searching and drafting legislation. It is currently in Alpha trial with various legislation teams in government.
With Lex, the policy professional will be able to ask questions of the statute book and generate snapshots of relevant legislation across a particular policy area. Empowering the policy profession in this way will reduce its dependence on departmental lawyers, a precious resource. Beyond this immediate use case, consider how AI navigation of the statute book could open up further possibilities, such as identifying precisely where old legislation must be amended when new bills come into effect, flagging gaps in the statute book, allowing small and medium-sized enterprises to check their requirements with different regulations, or even introducing machine-readable law to guarantee compliance in automated systems.
As well as legislation, Parliament itself can feel like a mystery to the policymaker. Staying on top of debates, votes and hearings is a challenge, carrying the risk that teams may introduce legislation without fully factoring in parliamentary concerns, leading to amendments or the possibility of legislation not being enacted at all.
Hansard, the official report of all parliamentary debates dating back over 200 years, is another dataset which should be a source of national pride. It is another perfect foundation for AI analysis, which is why the Incubator built Parlex.
Parlex
Parlex, a sister tool to Lex, is a Hansard-based AI tool which brings the opinions of parliamentarians into the hands of the policymaker. Parlex provides an intelligent search over Hansard, giving officials insight into the key debates around policy areas and the views of specific MPs and Peers.
Using Parlex, the policy professional can better prepare policy and better brief Ministers. It allows more time to engage with relevant parliamentarians rather than just striving to identify who they are.
An official within the Parliamentary Business and Legislation secretariat, given the task of steering the Government’s legislative programme, has been using Parlex to summarise crucial day-long bill debates chronicled by Hansard in just seconds. In that official’s view, Parlex is a ‘game-changer’ in bringing parliamentary debates into the corridors of Whitehall.
Ultimately Parlex will empower the House of Commons. This AI tool will allow policymakers to consider more meaningfully the contributions of members, it will provide a better understanding of parliamentary debates if made available to the public, and it will lead to MPs being assessed less on whip-enforced voting records and more on the thoughtfulness of their contributions.
Delivery
The AI-empowered policy professional should not stop at strategy and democracy; the policy they develop and negotiate through the legislature also has to be delivered. Too often, delivery is detached from the remit of the policy professional, with more energy spent on preparing announcements than on ensuring a policy’s impact. Policymakers have clear ministerial oversight throughout policy formulation and that policy’s passage through Parliament, but sometimes its delivery takes place in the shadows.
There is, therefore, a role for AI in making delivery more visible to the policy professional, allowing for earlier, more targeted intervention when things veer off track, and ultimately for better policy design in the first place. This is why i.AI built Scout.
Scout
Scout is an AI tool designed to improve major government project delivery by automatically analysing thousands of documents to help detect early ‘teething’ problems, enabling interventions to keep critical projects on track. It has been co-developed with the Infrastructure and Projects Authority and the code is available on Github.
Tracking and detection tools like Scout will give the policymaker, and therefore Ministers, a much stronger grip on project delivery. It is impossible to understate the impact of this; major projects, from railways to national IT databases, represent over £805 billion in whole-life spending. Through these efforts, delivery will not only be visible to the policymaker but also to the public, securing better outcomes with increased transparency and accountability.
Conclusion
This is a new era for the policy profession, and the citizen is counting on the Civil Service to embrace it. Just as we will have AI-empowered policy professionals, we will also have AI-empowered citizens, more able to express themselves effectively to government, more able to scrutinise the impact of policies and more demanding of the policy professional as a result.
Far from diminishing the hand of the policy profession, the tools outlined in this article will strengthen it. As a human expert, the policy professional will shape and finish AI outputs, remaining a key decision-maker accountable to Ministers and the public. Most importantly, time will be freed up to deliver those very human elements of policymaking, such as critical thinking, communication and relationship building, which are often the difference between success and failure.
With tools like Redbox, Parlex, Consult and Scout, there are a multitude of reasons to be optimistic. We can empower Parliament, engage citizens, form better policy and deliver for Ministers, all through embracing technology. This is the future.
Michael Padfield is Engagement and Strategy Manager, i.AI – Incubator for Artificial Intelligence – at the Department for Science, Innovation and Technology (DSIT).
Find the latest from i.AI at: https://ai.gov.uk/
