Over the past few months, BlueBEAR users may have experienced periods of reduced service availability, planned maintenance, and interruptions to normal operations. We recognise that these disruptions can be frustrating, particularly when research projects depend on reliable access to High Performance Computing (HPC) resources. This blog post explains the factors behind recent outages and outlines the steps we have taken to improve service resilience.
- Protecting the service against emerging security threats
- Challenges during extreme weather conditions
Protecting the service against emerging security threats
One of the primary causes of disruption has been the need to address several significant security vulnerabilities affecting HPC systems worldwide, including key components of the BlueBEAR infrastructure.
As cybersecurity threats continue to evolve, some vulnerabilities require immediate intervention to protect institutional systems and research data. In several cases, this has meant carrying out emergency maintenance and applying software, firmware, and operating system updates outside of the normal maintenance schedule.
While these activities can temporarily reduce service availability, they are essential for maintaining the security and integrity of the environment that supports research across the University.
Challenges during extreme weather conditions
Alongside security-related maintenance, recent periods of unusually high temperatures have affected the data centre facilities that host BlueBEAR. This has been an unusually hot year, and the MET Office states that “By 15 July, 2026 has already recorded more days above 30°C than the whole of 1976”, ref Met Office.
HPC systems generate significant heat and depend on specialised cooling infrastructure to operate safely. During periods of exceptionally warm weather, these cooling systems come under increased pressure, as has been seen at several other institutions in the last few months, and in some cases have seriously affected the availability of their services. In the last few months, precautionary measures have been required to protect the Research Data Store from overheating, resulting in temporary reductions in available compute. Protecting hardware from thermal damage is essential to maintaining its long-term reliability and avoiding more significant outages. We have taken the precaution of adding temporary air conditioning to enable us to keep services running over the coming weeks and believe services should be stable for the rest of the summer.
In the medium-term, we are working with Estates to build a specialist air-cooled space to host these heat-sensitive air-cooled storage systems, and thus reduce the risk of similar heat-related outages.
A research-focused service
BlueBEAR is designed primarily to support computationally-intensive and data-intensive research activities. Its architecture, operational model, and maintenance schedules are optimised to provide researchers with access to advanced computing resources that would not typically be available in standard teaching environments.
As a research platform, BlueBEAR prioritises flexibility, capability, and security. This means that occasional maintenance periods are necessary to ensure the service continues to meet the changing needs of the University’s research community.
Looking ahead
We appreciate the patience and understanding shown by our community during recent disruptions. Maintaining a secure, reliable, and high-performance research computing environment requires continual investment, maintenance, and improvement.
This summer we have several large-scale projects already in progress – to fully replicate all Research Data Store data across two sites, giving us the ability to maintain services while one datacentre is down for maintenance; upgrading and overhauling the backup and archive service, securing our researchers’ data daily and for years to come; and installing and commissioning Baskerville NCR, maintaining University of Birmingham’s position at the forefront of cutting-edge research capability.
The Advanced Research Computing team remains committed to supporting researchers across the University and will continue to communicate openly about planned maintenance, service issues, and infrastructure improvements via the relevant service mailing lists and the IT Services System Status page. Further general information on BEAR services maintenance can also be found on our documentation web pages on maintenance and downtime
Thank you for your continued support and understanding.
Aslam Ghumra on behalf of the ARC Team