Optimization References

My list of reference materials containing for mathematical optimisation, based on Quora.

Lecture notes

Highly recommended: video lectures by Prof. S. Boyd at Stanford, this is a rare case where watching live lectures is better than reading a book.

  • EE263: Introduction to Linear Dynamical Systems (video): http://www.stanford.edu/~boyd/ee263/videos.html

  • EE363: Linear Dynamical Systems: http://www.stanford.edu/class/ee363/

  • EE364a: Convex Optimization I (video): http://www.stanford.edu/class/ee364a/videos.html

  • EE364b: Convex Optimization II (video): http://www.stanford.edu/class/ee364b/videos.html

  • 6.253: Convex analysis and optimization: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-253-convex-analysis-and-optimization-spring-2010/lecture-notes/

  • Optimization courses at MIT: http://optimization.mit.edu/classes.php

  • Optimisation Course in CMU 10-725 Optimization Fall 2012

Books

  • S. Bubeck, “Convex Optimization: Algorithms and Complexity”, arXiv:1405.4980, 2015

  • F. Clarke, “Functional Analysis, Calculus of Variations and Optimal Control”, Springer, 2013

  • Liberzon, D., “Calculus of Variations and Optimal Control Theory - A Concise Introduction”, Princeton University Press, 2012

  • S. Boyd and L. Vandenberghe, “Convex Optimization”, Cambridge University Press, 2004

  • G. Calafiore and L. El Ghaoui, “Optimization Models”, Cambridge University Press, 2014

  • R. T. Rockarfellar and R. J. B. Wets, “Variational Analysis”, Springer, 1998

  • D. G. Luenberger and Y. Ye, “Linear and Nonlinear Programming”, 4th ed., Springer, 2016

  • J. Frédéric Bonnans, J. Charles Gilbert, C. Lemaréchal and C. A. Sagastizábal, “Numerical Optimization”, 2nd ed., Springer, 2006

  • Papadimitriou & Steiglitz, Combinatorial Optimization: Algorithms and Complexity: http://www.amazon.com/Combinatorial-Optimization-Algorithms-Christos-Papadimitriou/dp/0486402584

  • Lawson & Hanson, Solving Least Squares Problems: http://books.google.com/books/about/Solving_Least_Squares_Problems.html?id=ROw4hU85nz8C

  • Bellman, Dynamic Programming: http://www.amazon.com/Dynamic-Programming-Richard-Bellman/dp/0486428095/

  • Bellman, Applied Dynamic Programming: http://www.amazon.com/Applied-Dynamic-Programming-Richard-Bellman/dp/B0000CLNVK

  • Bellman, Adaptive Control Processes: http://www.amazon.com/Adaptive-Control-Processes-Bellman/dp/0691079013

  • Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning: http://www.amazon.com/Genetic-Algorithms-Optimization-Machine-Learning/dp/0201157675

  • Gill, Murray, Wright, Practical Optimization: http://www.amazon.com/Practical-Optimization-Philip-Gill/dp/0122839528
  • Ben-Tal & Nemirovsky, Lectures on Modern Convex Optimization: http://www.amazon.com/Lectures-Modern-Convex-Optimization-Applications/dp/0898714915

  • Bertsekas, Introduction to Linear Optimization: http://www.amazon.com/Introduction-Linear-Optimization-Scientific-Computation/dp/1886529191

  • Bertsekas, Convex Analysis and Optimization: http://www.amazon.com/Convex-Analysis-Optimization-Dimitri-Bertsekas/dp/1886529450

  • Bertsekas, Nonlinear programming: http://www.amazon.com/Nonlinear-Programming-Dimitri-P-Bertsekas/dp/1886529000/

  • Bertsekas, Dynamic Programming and Optimal Control: http://www.amazon.com/Dynamic-Programming-Optimal-Control-Vol/dp/1886529086

  • Rockafellar, Convex Analysis: http://www.amazon.com/Analysis-Princeton-Landmarks-Mathematics-Physics/dp/0691015864/

  • Nesterov, Introductory Lectures on Convex Optimization: A Basic Course: http://www.amazon.com/Introductory-Lectures-Convex-Optimization-Applied/dp/1402075537

  • Ruszczynski, Nonlinear Optimization: http://www.amazon.com/Nonlinear-Optimization-Andrzej-Ruszczynski/dp/0691119155/

  • Fletcher, Practical Methods of Optimization: http://www.amazon.com/Practical-Methods-Optimization-R-Fletcher/dp/0471494631

  • Nocedal & Wright, Numerical Optimization: http://www.amazon.com/Numerical-Optimization-Operations-Financial-Engineering/dp/0387303030/ Press et al.

  • Numerical Recipes: http://www.amazon.com/Numerical-Recipes-3rd-Scientific-Computing/dp/0521880688

  • Dennis & Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations: http://www.amazon.com/Numerical-Unconstrained-Optimization-Nonlinear-Mathematics/dp/0898713641

  • Cornuejols & Tütüncü, Optimization Methods in Finance: http://www.amazon.com/Optimization-Methods-Finance-Mathematics-Risk/dp/0521861705/

  • Stengel, Optimal Control and Estimation: http://www.amazon.com/Optimal-Control-Estimation-Advanced-Mathematics/dp/0486682005/

  • Kirk, Optimal Control Theory: http://www.amazon.com/Optimal-Control-Theory-Donald-Kirk/dp/0486434842/

  • Spall, Introduction to Stochastic Search and Optimization: http://www.amazon.com/Introduction-Stochastic-Search-Optimization-James/dp/0471330523/

  • Lasdon, Optimization Theory for Large Systems: http://www.amazon.com/Optimization-Theory-Large-Systems-Lasdon/dp/0486419991

  • Deb & Kalyanmoy, Multi-Objective Optimization Using Evolutionary Algorithms: http://www.amazon.com/Multi-Objective-Optimization-Using-Evolutionary-Algorithms/dp/047187339X

  • Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning: http://www.amazon.com/Genetic-Algorithms-Optimization-Machine-Learning/dp/0201157675/

  • Minoux, Mathematical Programming: http://www.amazon.com/Mathematical-Programming-Wiley-Interscience-mathematics-optimization/dp/0471901709

  • Camacho & Alba: Model Predictive Control: http://www.amazon.com/Predictive-Control-Advanced-Textbooks-Processing/dp/1852336943

  • Hillier, Introduction to Operations Research: http://www.amazon.com/Introduction-Operations-Research-Student-Access/dp/0077298349/

  • Puterman, Markov Decision Processes: http://www.amazon.com/Markov-Decision-Processes-Programming-Probability/dp/0471727822

  • Powell, Approximate Dynamic Programming: http://www.amazon.com/Approximate-Dynamic-Programming-Dimensionality-Probability/dp/0470171553/

Other

  • Grešovnik, Optimization Links: http://www2.arnes.si/~ljc3m2/igor/links.html

  • 8 Arsham, Intro to Modeling and Optimization: http://home.ubalt.edu/ntsbarsh/opre640a/partviii.htm

  • Matlab Optimization Toolbox resources: http://www.mathworks.com/help/toolbox/optim/

  • Bennett et al., The Interplay of Optimization and Machine Learning Research: http://jmlr.csail.mit.edu/papers/volume7/MLOPT-intro06a/MLOPT-intro06a.pdf

  • Evolutionary algorithms chapter in Jason Brownlee’s book : http://www.cleveralgorithms.com/nature-inspired/evolution.html

  • Brent, Algorithms for Minimization without Derivatives: http://maths-people.anu.edu.au/~brent/pub/pub011.html