Title :

Optimal Control of Markov Decision Processes

Speaker :

Prof. Roger Brockett

An Wang Professor of Computer Science and Electrical Engineering, Division of Engineering and Applied Sciences

Founder of the Harvard Robotics Laboratory, Harvard University, USA

Venue :

Room 215, William M. W. Mong Engineering Building, CUHK

Date :

Dec 14, 2009, Monday
10:30 AM - 11:30 AM

Abstract :

A large number of applications of the optimal control of Markov processes are known, ranging from problems motivated by situations arising in service industries to problems involving network control. These problems are often classified as being finite time problems or infinite horizon problems. Recently we derived a differential equation for the expected value of the minimal cost associated for a wide class of such problems involving the optimal adjustment of the transition rates subject to an appropriate performance measure. Although the optimal policy is given explicitly in feedback form, depending only on the solution of a certain differential equation, it has remained unclear as to if the feedback control policy approaches a constant policy in steady state or if, perhaps, it approaches a periodic policy. In this talk we will discuss recent results in this area with emphasis on the asymptotic situation.

Biography :

Dr. Roger Brockett joined the Department of Electrical Engineering at the Massachusetts Institute of Technology in 1963 as an assistant professor and Ford Foundation Fellow, working in automatic control. In 1969 he was appointed Gordon McKay Professor of Applied Mathematics in the Division of Applied Sciences at Harvard, where he became An Wang Professor of Electrical Engineering and Computer Sciences in 1989. Dr. Brockett has held visiting positions at a number of universities, including the Tokyo Institute of Technology. Over the past 30 years, Dr. Brockett has been involved in the professional activities of IEEE, SIAM and AMS, having served on the advisory committees and editorial boards for several groups in these societies, including IEEE`s Transactions on Automatic Control and the IEEE Transactions on Robotics and Automation. He was a co-editor for Systems and Control Letters from its founding until 1980. In 1989, he received the American Automatic Control Council`s Richard E. Bellman Award, and in 1991, he received the IEEE Control Systems and Engineering Field Award. In 1996 he received SIAM`s Reid Prize for his work in differential equations and control. He was elected to the National Academy of Engineering in 1991.

    **************************************** ALL ARE WELCOME ****************************************

Enquiries: Ms. Winnie Wong or Prof. Wen J. Li, Department of Mechanical and Automation Engineering, CUHK at 2609 8337, or Prof. Wong Wing Shing, Department of Information Engineering, The Chinese University of Hong Kong at 3163 4202. *MAE Series (2009-10)