PhD Studentship - Medical Vision-Language Models

Funding: School of Computer Science Studentship consisting of the award of fees, together with a tax-free maintenance grant of £19,237 per year for 3.5 years.

Lead Supervisor’s full name & email address

Dr. Duygu Sarikaya: d.sarikaya@leeds.ac.uk

Co-supervisor’s full name & email address

Dr. Nishant Ravikumar: n.ravikumar@leeds.ac.uk

Project summary

Vision language models (VLM) can process both visual information and natural language and learn associations between visual information and corresponding text descriptions. With their ability to extract semantics and insights from multi-modal data, they have shown impressive capabilities in tasks such as image captioning, visual question-answering, and text-to-image search. However, such models have seen limited adoption within real-world healthcare applications. Visual Question Answering (VQA) is a task that involves understanding and answering questions about images. It combines both computer vision, which interprets the content of images, and natural language processing, which deals with understanding and generating human language. The answers to these questions require an understanding of the image, the language, and domain-specific knowledge.

Medical visual question answering models can assist clinicians in clinical decision-making and increase efficiency in the clinical workflow. They can be used to develop text-to-image search engines that allow users to query an image and its visual contents through natural language, making it possible to find medical images that fit specific criteria for research, discovery, or educational purposes.

Please state your entry requirements plus any necessary or desired background

A first class or an upper second class British Bachelors Honours degree (or equivalent) in an appropriate discipline.

Advert information