Rebecca Riley goes head to head with a chatbot to see if it can answer any of the big questions about regional inequality in the UK.
ChatGPT is the latest and potentially impressive artificially intelligent chatbot yet. It was released at the beginning of December, and in just five days hit a million users. I narrowly managed to set up an account before I refreshed the page, and their servers were at capacity again, you can try here.
I thought I would see if they could answer some of the questions that plague our team at City-REDI and many academics and policymakers over many years. Who knows, maybe computers may have the answer? Should Whitehall civil servants be quaking in their boots in fear of being replaced by AI and could local authorities use this in the future to find responses to their policy challenges?
I initially went big…
This response came back within seconds, and generally ok for a high-level summary of potential areas to look at for solutions. It highlights the redistribution of investment in infrastructure and education and supporting businesses to grow. It also points to the need for a combination of efforts to tackle underlying causes of growth gaps. So far ok, not a wild answer but not helpful for policymakers in terms of developing an innovative approach or suggesting solutions.
So I opted to probe the issue of what are the causes?
We then hit the detail wall. This AI is a language model, it’s exploring what people are talking about and publishing (although we don’t know the details of the algorithm) it looks for whether there are similar questions and answers, in its pool of content. It’s likely that it can’t access academic publications as usually they are behind paywalls, any future AI will only be able to access open publications. But the explanation is interesting for policymakers, the chatbot doesn’t have access to the current news and information that would be necessary to answer the question. The causes of the growth gap are complex, changing, and specific to place that needs a lot of knowledge and information to determine and interpret. Different places have a different mix of issues relative to London, but this can only be understood by those that know the place (and the true picture of London) for comparison.
So, I tried more specifically and looked at Birmingham.
The chatbot can only provide general information again and says this. What is interesting about this is it’s also an accurate reflection of national policy, just like the chatbot national policymakers can only respond to that which is visible and known to them. That the bias that will filter into the AI programmes will be the same bias that exists in human understanding, limited by our knowledge, and biased by what we store, believe and remember in our own data pool. Place and people bias will still creep in, and our perceptions of the strengths and weaknesses of places will shape our policy responses. When it comes to levelling up this is key ‘without current information, it would be difficult for me to provide a more detailed answer’.
So, I tried a slightly different question.
Again, without specific information on Birmingham the bot can’t answer, but can go back to generalities and back to the broad themes in my initial question and we have gone full circle.
Policy development relies on current, robust information, that is geographically and thematically relevant. For both humans and AI, this data must exist, be available for analysing and this analysis must be free from bias. So, the challenges people face in developing policy are the same for AI. Where a people-driven process will always have an advantage is in the tacit knowledge of felt experiences and what will make a real difference to people’s lives.
AI however will make accessing large amounts of data easier in future, but it will take the skills, knowledge, and capacity of people to make it relevant and responsive in real-time to an emotional and fundamental need. Current data and evaluation of impacts are vital to assessing the place and people effects of policy and building better connections and collaborations between the national and local is key to ensuring national policy is responsive to those local needs. Without these real co-created processes a greater understanding of the reality of levelling up won’t be recognised in national policy.
My final question was to go to the specific policy area, and I chose one where there is a lot of policy activity and skills.
At this point, I think the chatbot gave up, or the server was overloaded. Just like the policymakers, the questions became too difficult, or it was having too many questions fired at it to cope. So even computers get overwhelmed by the complexity and the demands of finding policy solutions.
This blog was written by Rebecca Riley, Associate Professor, City-REDI/WMREDI, University of Birmingham.
Disclaimer:
The views expressed in this analysis post are those of the authors and not necessarily those of City-REDI or the University of Birmingham.