Energy Data Landscape – Challenges and Opportunities

Published: Posted on

Edward Barbour and Grant Wilson, Associate Professors in Chemical Engineering, explored detailed domestic heat pump demand data and became interested in their operational costs under different energy tariffs. They quickly discovered that tariff data was difficult to obtain and compare across different suppliers.

From left to right : Ed, Adrian, Jean and Grant

With help from Mingxue (Jean) Du, a Research Software Engineer, they identified several key issues:

  • Inconsistent formats that do not meet FAIR data principles
  • Limited access: Most suppliers require postcode, and consumption estimates before showing prices
    PDF-based data: Difficult to extract and analyse due to format variability
  • API access: Only Octopus Energy offers a usable API
  • Many suppliers provide a snapshot of prices rather than longitudinal dataset
  • Although minor, Inconsistencies between different suppliers: some include VAT in their pricing data while others don’t; the same region is known by different names to different suppliers

These challenges highlight a highly fragmented data landscape, making longitudinal research and policy analysis uncertain. Jean developed a proof-of-concept visualisation tool (shown below) to show tariff trends over time.

The team suggests that Ofgem consider standards for energy suppliers to make tariff data adhere to FAIR principles:

  • Findable: researchers and computers can find the data
  • Accessible: data and/or metadata also needs to be accessed by both researchers and computers
  • Interoperable: data exist in file formats that are not dependent on proprietary or obsolete software
  • Reusable: data should be ready for research and processing

The next step is to analyse the available data and present additional findings to Ofgem and to advocate for policy changes that improve data transparency and usability for the wider research community.

We are so pleased to learn about how the project team makes use of what is on offer from Advanced Research Computing, particularly to hear of how they have made use of the RSE group, BEAR compute and storage, – if you have any examples of how it has helped your research then do get in contact with us at bearinfo@contacts.bham.ac.uk.

We are always looking for good examples of use of High Performance Computing to nominate for HPC Wire Awards – see our recent winner for more details.