Does the Computer say yes to Levelling Up? Will ChatGPT Replace the Policy Wonk?

Published: Posted on

Rebecca Riley goes head to head with a chatbot to see if it can answer any of the big questions about regional inequality in the UK.

ChatGPT is the latest and potentially impressive artificially intelligent chatbot yet. It was released at the beginning of December, and in just five days hit a million users. I narrowly managed to set up an account before I refreshed the page, and their servers were at capacity again, you can try here.

I thought I would see if they could answer some of the questions that plague our team at City-REDI and many academics and policymakers over many years. Who knows, maybe computers may have the answer? Should Whitehall civil servants be quaking in their boots in fear of being replaced by AI and could local authorities use this in the future to find responses to their policy challenges?

I initially went big…

chat AI response to How do you narrow the gap in growth between London and the rest of the UK? response: It is important to address the factors that are contributing to the gap in growth between London and the rest of the UK. This could include investing in infrastructure and education in other parts of the country, as well as promoting economic development outside of London. Additionally, policies that support the growth of businesses in other regions could help to narrow the gap. It will likely require a combination of efforts to address the underlying causes of the growth gap and support the development of other parts of the country.

This response came back within seconds, and generally ok for a high-level summary of potential areas to look at for solutions. It highlights the redistribution of investment in infrastructure and education and supporting businesses to grow. It also points to the need for a combination of efforts to tackle underlying causes of growth gaps. So far ok, not a wild answer but not helpful for policymakers in terms of developing an innovative approach or suggesting solutions.

So I opted to probe the issue of what are the causes?

question - What are the underlying causes of the UK growth gap? answer - I'm sorry, but I don't have information about the specific underlying causes of the growth gap between London and the rest of the UK. As a language model, I don't have access to current news or other information that would be necessary to answer this question accurately. My knowledge is limited to general information that was available at the time of my training.

We then hit the detail wall. This AI is a language model, it’s exploring what people are talking about and publishing (although we don’t know the details of the algorithm) it looks for whether there are similar questions and answers, in its pool of content. It’s likely that it can’t access academic publications as usually they are behind paywalls, any future AI will only be able to access open publications. But the explanation is interesting for policymakers, the chatbot doesn’t have access to the current news and information that would be necessary to answer the question. The causes of the growth gap are complex, changing, and specific to place that needs a lot of knowledge and information to determine and interpret. Different places have a different mix of issues relative to London, but this can only be understood by those that know the place (and the true picture of London) for comparison.

So, I tried more specifically and looked at Birmingham.

question - What are the causes of Birmingham's growth gap with the UK? answer - As a language model, I don't have information about the specific causes of any growth gap in Birmingham or any other specific location. I can only provide general information about the potential factors that could contribute to a growth gap. These could include a lack of investment in education and infrastructure, a lack of support for businesses in the area, or other economic and social factors. Without access to current information, it would be difficult for me to provide a more detailed answer.

The chatbot can only provide general information again and says this. What is interesting about this is it’s also an accurate reflection of national policy, just like the chatbot national policymakers can only respond to that which is visible and known to them. That the bias that will filter into the AI programmes will be the same bias that exists in human understanding, limited by our knowledge, and biased by what we store, believe and remember in our own data pool. Place and people bias will still creep in, and our perceptions of the strengths and weaknesses of places will shape our policy responses. When it comes to levelling up this is key ‘without current information, it would be difficult for me to provide a more detailed answer’.

So, I tried a slightly different question.

question What policies would help Birmingham grow? answer - Without access to current information about Birmingham's economy and specific policies that have been implemented, it is difficult for me to provide a detailed answer to this question. In general, however, policies that support the growth of businesses and promote economic development can help a city like Birmingham grow. This could include investing in education and training programs to develop a skilled workforce, investing in infrastructure to support businesses, and providing support and incentives for businesses to locate and grow in the area. It will likely require a combination of efforts to address the specific challenges and opportunities facing Birmingham and support its growth.

Again, without specific information on Birmingham the bot can’t answer, but can go back to generalities and back to the broad themes in my initial question and we have gone full circle.

Policy development relies on current, robust information, that is geographically and thematically relevant. For both humans and AI, this data must exist, be available for analysing and this analysis must be free from bias. So, the challenges people face in developing policy are the same for AI. Where a people-driven process will always have an advantage is in the tacit knowledge of felt experiences and what will make a real difference to people’s lives.

AI however will make accessing large amounts of data easier in future, but it will take the skills, knowledge, and capacity of people to make it relevant and responsive in real-time to an emotional and fundamental need. Current data and evaluation of impacts are vital to assessing the place and people effects of policy and building better connections and collaborations between the national and local is key to ensuring national policy is responsive to those local needs. Without these real co-created processes a greater understanding of the reality of levelling up won’t be recognised in national policy.

My final question was to go to the specific policy area, and I chose one where there is a lot of policy activity and skills.

question - what are the solutions to filling skills gaps? answer - An error occurred. If this issue persists please contact us through our help center at

At this point, I think the chatbot gave up, or the server was overloaded. Just like the policymakers, the questions became too difficult, or it was having too many questions fired at it to cope. So even computers get overwhelmed by the complexity and the demands of finding policy solutions.

This blog was written by Rebecca Riley, Associate Professor, City-REDI/WMREDI, University of Birmingham.

The views expressed in this analysis post are those of the authors and not necessarily those of City-REDI or the University of Birmingham.

Sign up for our mailing list

Leave a Reply

Your email address will not be published. Required fields are marked *