1x PhD Studentship - Automatically Detecting, Reproducing and Fixing Flaky Software Tests

3.5 year PhD studentship

The cost of software failures is a huge burden to the worldwide economy that was estimated to be at least £1.3 trillion in 2017. Consequently, software testing, a vital defence against failures, contributes to a large proportion of software development effort and cost.

Flaky tests are a particular strain on resources allocated to software development because they intermittently pass and fail without changes to tests or project code, with often maddening, non-obvious causes. Flaky tests are tests that can fail when code is working, and pass when it isn't. Because developers can no longer trust the results of their tests, they are unable to gain confidence that software is working correctly, potentially exposing end-users to the consequences of software failures. Flaky tests are a common occurrence in industry, significantly disrupting software development — even for companies with the greatest resources to tackle them, including Microsoft, Facebook, and Google.

This PhD aims to develop automated approaches to mitigate and alleviate the problem of flaky tests in everyday development processes, including, for example: (a) detecting potential flaky tests ahead of time; (b) discerning when a test is flaky and when there is a genuine test failure; (c) helping developers reproduce flaky tests so they can debug them; and (d) automatically repairing flaky tests so that they are no longer flaky.

Supervisor Bio

Phil McMinn is a Professor of Software Engineering whose specialism is in software testing (philmcminn.com). He has been awarded research funding from the EPSRC, HEFCE, and from industry including Meta (Facebook). He has supervised nine PhD students to completion (philmcminn.com/past-phd), and currently manages a team of four PhD students and a Post-Doctoral researcher (philmcminn.com/#team).

About the School/Research Group

You will be a part of the vibrant software testing research group. In the Department of Computer Science at Sheffield, 99% of our research was rated in the highest two categories in the REF 2021, meaning it was classed as world-leading or internationally excellent.

Candidate Requirements

• A 1st or 2:1 (or equivalent) in a computing degree.
• Good programming skills (e.g., Python, Java).
• If English is not your first language, you must have an IELTS score of 6.5 overall, with no less than 6.0 in each component.

How to apply

To apply for a PhD studentship, applications must be made directly to the University of Sheffield using the Postgraduate Online Application Form. Make sure you name Phil McMinn as your proposed supervisor.

Information on what documents are required and a link to the application form can be found here - www.sheffield.ac.uk/postgraduate/phd/apply/applying

Your application needs to include a research proposal that outlines your reasons for applying, explains details of your skills and experience that are relevant to the position, and details any initial ideas you have about the project.

In the first instance, all potential candidates are advised to contact and talk to Phil McMinn (p.mcminn@sheffield.ac.uk).

Funding details

The PhD studentship will cover standard UK home tuition fees and provide a tax-free stipend at the standard UK Research Council rate (currently £19,237 for the 2024/25 academic year) for 3.5 years. Overseas students are welcome to apply but will only be eligible for a tuition fee only studentship. Alternatively, overseas students can self-fund the difference between the UK and overseas tuition fees. International fees for Computer Science can be found here: www.sheffield.ac.uk/new-students/tuition-fees/fees-lookup

Advert information