Analysis and Control of Markov Chains
Spring 2008
|
EE 292E Analysis and Control of Markov Chains Spring 2008 |
| Course Information |
Prerequisites: EE 263 and EE 278, or equivalent background in probability, linear algebra, and dynamic systems. Experience with Matlab.
| Instructors |
| Grading |
Course grades will be based 15% on homework assignments, 35% on a midterm exam, and 50% on a final exam.
Students are encouraged to work on homework problems in groups but must write up their own solutions. When writing up solutions, students should write the names of people with whom they discussed the assignment.
| Course Readings |
The primary textbook will be Dynamic Programming and Optimal Control, Vol. 1 (3rd edition), by Dimitri Bertsekas. The instructor will also provide some supplementary notes over the course of the quarter.
| Announcements |
| Tentative Lectures |
| Date | Topic | Read Sections | Notes |
|---|---|---|---|
| 1 April | Deterministic Systems | 2.1 | Note |
| 3 April | Viterbi Decoding, Caching | 2.2 | Note |
| 8 April | Caching | 4.1 | Note |
| 10 April | Linear-Quadratic Control, Stochastic Systems | 4.2 | Note |
| 15 April | Dynamic Pricing | Note | |
| 17 April | Inventory Management | Note | |
| 22 April | Portfolio Management | 4.3 | Note |
| 24 April | Optimal Stopping, Scheduling | 4.4, 4.5 | Note |
| 29 April | Imperfect State Information | 5.1 | Note |
| 1 May | Separation Principle | 5.2 | Note |
| 6 May | Belief States | 5.4 | Note |
| 8 May | Infinite Horizon Problems | Note | |
| 13 May | Algorithms for Infinite Horizon Problems | Note | |
| 15 May | Algorithms for Infinite Horizon Problems | Note | |
| 20 May | Algorithms for Infinite Horizon Problems | Note 1 Note 2 | |
| 22 May | Queueing and Uniformization | Note | |
| 27 May | Multi-Armed Bandits | Note 1 Note 2 | |
| 29 May | Approximate Dynamic Programming | Note | |
| 3 June | Review |
| Homework |
| Exams |
Students can choose one of two time slots to take the five-hour, open-book, open-notes take-home final:
June 4 from 12PM to 5PM, and June 5 from 12PM to 5PM.
Final 2005-2006: Exam Solution
Final 2007-2008: Exam Solution