Donald Houston discusses his recent secondment at the ONS. What did he learn whilst he was there and what did the ONS gain from having an academic on board?
I am a labour market researcher and a long-time user of ONS data. I have always been impressed at the quality and precision of the labelling of ONS data. I’d even describe myself as an ONS ‘groupie’. I know, I know – I don’t get out much.
After working with ONS’s Coronavirus Infection Survey (CIS) on the impact of Covid-19 on the size of the workforce, now seemed like a good time to extend this work and take it closer to government. ONS’s Partnership Development Programme offered an opportunity to work at ONS on a secondment basis with the Health Analysis & Pandemic Insights (HAPI) unit.
My experience at ONS was an overwhelmingly positive one and I learned a lot. This is not entirely surprising as the ONS was recently ranked as one of the UK’s best employers. This was borne out by the positive, professional and supportive attitudes of all those I encountered, not to mention some very technically able and dedicated public servants. The analysis I did while at ONS brought me into contact with officials in government and an ONS article reporting the findings gave exposure to a different audience compared to traditional academic publications.
A learning experience
So what specifically did I learn from my time at ONS? First and foremost was the value of robust quality assurance procedures. Every piece of analysis run by ONS must be replicated and checked. Replication is done by a different analyst using different software. Each and every statistic reported in an ONS article is checked and validated. A system is in place to ensure these quality assurance procedures have been followed before publication will be approved.
Related to this, ONS aims to have a ‘single version of the truth’. On one hand, this is an anathema to most academics (particularly in the social sciences and humanities) who – quite rightly – see ‘knowledge’ as subjectively constructed and contestable. On the other hand, at least in the realm of national statistics used in high-level political debates, aiming for a single truth can help eliminate less robust ‘truths’, through identifying the most robust data available to measure something. Although a ‘single version of the truth’ is a technical concept in information management systems to ensure consistency and an understandable one for a national statistics agency to adopt, it does to some extent draw on the founding principles in the scientific method of replicability and falsifiability.
To provide an example of these processes in action, in my work at ONS an initial result differed from a recently published ONS statistic that had been produced using a different method but similar data. This prompted further analysis in order to replicate the earlier finding using the different method with the same data – initial attempts failed, which in turn revealed a statistical technique was not calibrated in quite the way that had been assumed. In the end, different mathematical procedures applied to the same data did in fact produce identical results. This process of replicability – such an important one in scientific method – had revealed an erroneous assumption in how a statistical technique was calibrated. Typical academic practices of largely sole-working followed by peer-review of outputs probably would not have flushed out this error (albeit it was quite a minor one). One of my ONS collaborators with some pride – but not a hint of schadenfreude – told me she’d read a study that found 10% of published academic research papers to contain errors in data coding or labelling. ONS one, academia nil.
Academic in the midst
So what did ONS gain from having an academic in their midst? Feedback from ONS colleagues indicates three broad benefits. One is having space and time to apply a robust statistical technique (decomposition analysis) to be able to disentangle the twin effects of population ageing and rising ill-health in shrinking the size of the UK workforce since the pandemic. Another is bringing theory (labour demand and supply) to aid the interpretation of findings. Finally, lateral thinking and a breadth of perspective in being able to see connections between different datasets and societal issues.
Now the funded secondment is over, I am pleased to say that I continue collaborative research as an ONS research associate, integrating with research in my ‘day job’ at City-REDI, University of Birmingham. This brings mutual benefits and efficiencies in terms of access to data, expertise and coordination of efforts. Siloed researchers in government departments, statistics agencies, universities, private companies, think tanks and other organisations are not conducive to maximising the quality or relevance to society and government of effort from the research community. Moving forward, I hope my time at ONS has made me more willing and able to traverse these boundaries – as well as making me more inclined to run checks on data and methods.
I would like to thank Emma Rourke, Julie Stanborough and Vahé Nafilyan at ONS for making this possible. I would also like to thank Jane Evans, Tim Gibbs, Clare Grant, Matt Lelii and Hugh Stickland for their contributions, support and engagement with the work.
This blog was written by Donald Houston, Professor of Regional Economic Development, City-REDI / WMREDI, University of Birmingham.
Disclaimer:
The views expressed in this analysis post are those of the authors and not necessarily those of City-REDI / WMREDI or the University of Birmingham.