Stochastic optimal control in high dimensional partially observable environments

Computational and Applied Mathematics Colloquium

Meeting Details

For more information about this meeting, contact Kristin Berrigan, John Harlim.

Speaker: Kostas Papakonstantinou, Penn State Civil and Engineering

Abstract: Computational decision-making able to directly offer optimal actions to decision-makers/agents is of increasing relevance in many fields of application and is also related to autonomous systems. A seamless integration of stochastic models and data with computational decision-making is discussed in this talk. As shown, challenging sequential decision problems in nonstationary dynamic environments can be efficiently formulated along the premises of stochastic control, through Markov Decision Processes (MDPs), Partially Observable Markov Decision Processes (POMDPs), and mixed approaches thereof. In systems with relatively low dimensional state and action spaces, MDPs and POMDPs can be satisfactorily solved to global optimality through appropriate dynamic programming algorithms, such as value iteration with asynchronous updates and point-based approaches for partial observability cases. However, optimal planning for large systems with multiple components is computationally hard and severely suffers from the curse of dimensionality. New developments on Deep Reinforcement Learning (DRL) methods and their capacity of addressing this problem are discussed, with emphasis on our developed DRL formulations and novel algorithmic schemes, able to solve otherwise intractable problems with immense state and action spaces, as is often the case with large, real engineering systems. DRL relations to Artificial Intelligence and Machine Learning are also explained. The talk concludes with ongoing efforts and numerous possible future research directions along these lines, as well as with several pertinent diverse applications, from centralized/decentralized infrastructure management, to emergency response of cooperating agents, to autonomous robotic navigation and wildfire prevention.


Room Reservation Information

Room Number: 114 McAllister

Date: 10/21/2019

Time: 12:20pm - 1:30pm