PhD Studentship: Decision-Making under Uncertainty: The Priority Elicitation Problem

The University of Exeter’s Department of Computer Science is inviting applications for a PhD studentship funded by Faculty of Environmental Science and Economy  to commence on  September 2025 or as soon as possible thereafter.  For eligible students, the studentship will cover Home tuition fees plus an annual tax-free stipend of at least £19,237 for 3.5 years full-time, or pro rata for part-time study.  The student would be based in the Department of Computer Science in the Faculty of Environment, Science and Economy at the Streatham Campus in Exeter.

Computing systems increasingly operate in dynamic and uncertain environments [1]. A key application of such systems is in the Internet of Things (IoT), where networked sensors and actuators enable real-time adaptation to environmental changes. Consider a self-adaptive IoT network such as a smart home that autonomously manages energy consumption while balancing multiple, often conflicting requirements, such as comfort, cost efficiency, and sustainability. Effective decision-making in such a system requires continuously evaluating trade-offs under different environmental conditions, such as changing weather patterns, fluctuating energy costs, or variations in occupants’ schedules.

Typically, the initial priority settings for requirements are made by domain experts during the design phase based on expected system behaviour. However, unforeseen situations may arise that challenge these initial assignments, potentially compromising the system’s ability to adapt effectively. For example, during a sudden heatwave, a smart home system must decide whether to prioritise maintaining a comfortable indoor temperature or minimising electricity costs. While the initial priority settings may emphasize energy efficiency or cost reduction, extreme environmental conditions may require a change in priorities to ensure occupant’s comfort.

The research project aims to explore techniques that could facilitate experts in the elicitation of priorities. One possible direction could be to use the technique of Inverse Reinforcement Learning (IRL) [2], [3]. IRL is an AI-based technique that supports imitation of the preferred system behaviour by using its behavioural history. It helps in the inference of the reward values by taking the observed history of policies as input. As the priorities are represented as utilities based on the multiple rewards [4].[5]; techniques like IRL could infer these reward values and support the experts with the elicitation process.

References

[1]          L. Garcia, H. Samin, and N. Bencomo, ‘Decision Making for Self-Adaptation Based on Partially Observable Satisfaction of Non-Functional Requirements’, ACM Trans Auton Adapt Syst, vol. 19, no. 2, p. 11:1-11:44, Apr. 2024, doi: 10.1145/3643889.

[2]          S. Arora and P. Doshi, ‘A survey of inverse reinforcement learning: Challenges, methods and progress’, Artif. Intell., vol. 297, p. 103500, Aug. 2021, doi: 10.1016/j.artint.2021.103500.

[3]          C. A. Rothkopf and C. Dimitrakakis, ‘Preference Elicitation and Inverse Reinforcement Learning’, in Machine Learning and Knowledge Discovery in Databases, D. Gunopulos, T. Hofmann, D. Malerba, and M. Vazirgiannis, Eds., Berlin, Heidelberg: Springer, 2011, pp. 34–48. doi: 10.1007/978-3-642-23808-6_3.

[4]          C. F. Hayes et al., ‘A practical guide to multi-objective reinforcement learning and planning’, Auton. Agents Multi-Agent Syst., vol. 36, no. 1, p. 26, Apr. 2022, doi: 10.1007/s10458-022-09552-y.

[5]          H. Samin, N. Bencomo, and P. Sawyer, ‘Decision-making under uncertainty: be aware of your priorities’, Softw. Syst. Model., vol. 21, no. 6, pp. 2213–2242, Dec. 2022, doi: 10.1007/s10270-021-00956-0.

Advert information