Created by several members of the Research Software Group (RSG)
In April the RSG organised its first ever team hackathon. For two weeks, all team members had time set aside from their regular project work to join in the various sessions offered. We were encouraged to sign up for anything that was interesting to us or that we wanted to know more about, as well as to sessions where we had some knowledge or experience to share. As a growing team, it was a great opportunity for us to work with members of the team we don’t regularly collaborate with, to learn from each other and to work on challenges together. All of the hackathon sessions were organised and led by team members and covered a wide range of topics and activities. Some of the sessions were aimed at improving our own internal workflows while others focused on the services we offer to other staff and students at the University.
Docker and containers
Two of the sessions focused on our own internal tools and workflows centered around Docker and other container solutions. In both sessions there was a mixture of experience, with more experienced Docker users sharing their knowledge with the rest of the group. The mix really worked well, with interesting discussions on security, optimisation and other topics that were not on the agenda. There was another session where we worked together to introduce Docker to our template Django project that underpins most of our web development projects. Later in the week we had a session which included demos of how containers can be used throughout the development lifecycle. We started with how Dev Containers can facilitate reproducible development environments. We then moved on to a discussion of containers in Continuous Integration pipelines and a demonstration of producing documentation as part of the build process.
Testing accessibility and code quality
In one of the sessions we evaluated our workflow for checking the quality and consistency of our Python code and tested some new linting tools and configurations that we can apply to all of our Python projects. We also looked at the automated accessibility testing workflow and discovered a way to run these tests more efficiently. In another session we sketched out a plan for a new researcher-facing website for BEAR project administration and tested out some of the ideas together. We now have a framework in place to develop that project further.
Data Science
A further session concentrated on productivity tools for data science. In this session we discussed common pain points in data science projects such as data versioning, managing large numbers of models and handling complicated workflows. We crowdsourced a list of helpful tools to address each challenge area such as DVC for data versioning, Kedro for model management and spent some time setting up and evaluating each tool in pairs. On the second day, we used example data science projects from Carpentries-style courses to set up demo project repositories and assess how easily the tools could be integrated into a ‘real’ project.
Improving our documentation
Some of the longer sessions of the hackathon involved work to improve the resources we provide to researchers around the University. This involved a comprehensive review of the documentation of our Tier 2 HPC system Baskerville and our Tier 3 cluster, BlueBEAR. Both the content of the documentation and the ease of navigation and search was addressed. As a team with a wide variety of HPC and Baskerville experience we relied on each other’s differing points of view to ensure the clarity of the documentation. A day was also spent developing examples that address different aspects of using the system and enable users to write their own code based on them. We also investigated options for adding video content to the documentation. All of the changes deemed necessary have been turned into GitLab issues and work has already started on making the improvements.
BEAR Challenge
In our final session we worked on our challenges for this year’s BEAR Challenge event. The session started with an overview of last year’s BEAR Challenge. Each group then presented their current work on the challenges for this year’s event and highlighted the areas where they would welcome input from others. Different groups tackled different aspects of the BEAR Challenge for example the presentation format of the different challenges and the auto-marking system. In the afternoon we worked on one of the specific challenges and discussed the complexity required in the problem solving to allow for a meaningful scoring process and identifying the type of approach we will use for the marking of the challenge.
Summary and on to next year!
The hackathon was a great success for our team. Feedback at the team meeting afterwards was very positive. We enjoyed it and learned a lot. Although it was noted that sometimes it can be uncomfortable to be working on things that you do not know much (or indeed anything!) about, there is nothing wrong with being in that position and there is always someone else on the team who can help or you can all figure it out together. For the four newest members of the team (who spent their first two weeks at the hackathon) it was a great way to integrate quickly into the team. We are aiming to make this an annual event and the 2025 hackathon is already in our calendar.