Data Science and Analytics Thrust | Principled Exploration in Sequential Decision Making

Interactive Machine Learning (IML) possesses the unique capability to harness feedback from interactions, making it indispensable in a wide array of real-world sequential decision-making applications. However, a significant challenge, known as the "exploitation and exploration dilemma," prominently arises within sequential decision-making. In this context, learners must not only exploit current information but also explore to uncover potential knowledge for long-term gains. Despite decades of research yielding a rich landscape of algorithms, frameworks, and theories for effectively utilizing collected data to train machine learning models, a fundamental question has remained largely unaddressed: How can IML models systematically make principled exploration for long-term benefits, alongside the full exploitation of current data? In this report, I will motivate the exploration strategies by human principles in sequential decision-making and present our research efforts in developing principled exploration strategies in IML systems with theoretical performance guarantees and real-world applications.