Handouts

• The following book reviews the basics of probability and random variables: Schaum's Outline of Probability, Random Variables, and Random Processes. Alternatively, the following high school textbook is the excellent introduction to real life probability concepts "Data Management 12, Student Edition" by Wayne Erdman, Maria Rosa Cruiscuolo, Roland Meisel, David Petro, Jacob Speijer, Wendy Telford, McGrawHill Education, ISBN 9781259256363. In particular, read Chapter 1 "Introduction to Probability". Do all exercises from sections 1.4 and 1.5 in Chapter 1 to make sure you understand well mutually exclusive events, independent/dependent events and conditional probability.
• Basic facts from probability (a PDF file).
• Probabilities and Expectations, a handout prepared by Ashique Mahmood, Sep 9, 2015.
• More information about Normal (Gaussian) distribution: from MathWorld.
• If you would like to read more about expected value and variance of discrete random variables, then this 6th chapter of a textbook written by Charles Grinstead and J. Laurie Snell can be useful. A complete textbook is freely available for download thanks to the American Mathematical Society.
• Selecting an outcome according to a given probability distribution: this handout includes a program in C that will help you to complete Assignment 1. The following handout is a Python program that simulates a fair coin and a fair die. It is written by Raiyan Rahman.
• A solution of the gambler's problem using the policy iteration algorithm: Example 4.2 from the text-book (page 82 in the 2nd edition). Acknowledgement: This C program was written by Wee Pham.
• A solution of the Blackjack game using the on-policy Monte Carlo algorithm: This C program was written by Wee Pham. Warning: there are a few minor errors in this program, but nevertheless it demonstrates well the method of Monte-Carlo on-policy control using epsilon-greedy policies.

• Reinforcement learning, a brief review paper written by Peter Dayan & Watkins, CJCH (2001). Published in: Encyclopedia of Cognitive Science London, England: MacMillan Press.

• Reinforcement Learning: A Survey, by Kaelbling, L.P., Littman, M.L., and Moore, A.W. Journal of Artificial Intelligence Research (1996), Volume 4, pages 237-285.

• Reinforcement Learning by Peter Dayan. A detailed overview of the basic theory underlying reinforcement learning, and its links with neuroscience, psychology, statistics and engineering. Published in CR Gallistel (editor), Steven's Handbook of Experimental Psychology, New York, NY: Wiley.

• OR-Gym: A Reinforcement Learning Library for Operations Research Problems, written by Christian D. Hubbs, Hector D. Perez, Owais Sarwar, Nikolaos V. Sahinidis, Ignacio E. Grossmann, John M. Wassick, October 20, 2020. Available at arxiv.org as 2008.06319