Practical Optimization Methods

This introductory textbook presents optimization theory and computational algorithms useful in practice. The approach is practical and intuitive, rather than emphasizing mathematical rigor. Computationally oriented books in this area generally present algorithms alone, and expect readers to perform computations by hand. Some books are written in traditional computer languages, such as Basic, Fortran or Pascal. The programs in this text help with computations. This book is the first text to use Mathematica to develop thorough understanding optimization algorithms, fully exploitingMathematica's symbolic, numerical and graphic capabilities.

This introductory textbook adopts a practical and intuitive approach, rather than emphasizing mathematical rigor. Computationally oriented books in this area generally present algorithms alone, and expect readers to perform computations by hand, and are often written in traditional computer languages, such as Basic, Fortran or Pascal. This book, on the other hand, is the first text to use Mathematica to develop a thorough understanding of optimization algorithms, fully exploiting Mathematica's symbolic, numerical and graphic capabilities.

This book is suitable for practicing scientists and engineers especially those using mathematica in their work.

1 Optimization Problem Formulation.- 1.1 Optimization Problem Formulation.- 1.2 The Standard Form of an Optimization Problem.- 1.3 Solution of Optimization Problems.- 1.4 Time Value of Money.- 1.5 Concluding Remarks.- 1.6 Problems.- 2 Graphical Optimization.- 2.1 Procedure for Graphical Optimization.- 2.2 GraphicalSolution function.- 2.3 Graphical Optimization Examples.- 2.4 Problems.- 3 Mathematical Preliminaries.- 3.1 Vectors and Matrices.- 3.2 Approximation Using the Taylor Series.- 3.3 Solution of Nonlinear Equations.- 3.4 Quadratic Forms.- 3.5 Convex Functions and Convex Optimization Problems.- 3.6 Problems.- 4 Optimality Conditions.- 4.1 Optimality Conditions for Unconstrained Problems.- 4.2 The Additive Property of Constraints.- 4.3 Karush-Kuhn-Tucker (KT) Conditions.- 4.4 Geometric Interpretation of KT Conditions.- 4.5 Sensitivity Analysis.- 4.6 Optimality Conditions for Convex Problems.- 4.7 Second-Order Sufficient Conditions.- 4.8 Lagrangian Duality.- 4.9 Problems.- 5 Unconstrained Problems.- 5.1 Descent direction.- 5.2 Line Search Techniques-Step Length Calculations.- 5.3 Unconstrained Minimization Techniques.- 5.4 Concluding Remarks.- 5.5 Problems.- 6 Linear Programming.- 6.1 The Standard LP Problem.- 6.2 Solving a Linear System of Equations.- 6.3 Basic Solutions of an LP Problem.- 6.4 The Simplex Method.- 6.5 Unusual Situations Arising During the Simplex Solution.- 6.6 Post-Optimality Analysis.- 6.7 The Revised Simplex Method.- 6.8 Sensitivity Analysis Using the Revised Simplex Method.- 6.9 Concluding Remarks.- 6.10 Problems.- 7 Interior Point Methods.- 7.1 Optimality Conditions for Standard LP.- 7.2 The Primal Affine Scaling Method.- 7.3 The Primal-Dual Interior Point Method.- 7.4 Concluding Remarks.- 7.5 Appendix-Null and Range Spaces.- 7.6 Problems.- 8 Quadratic Programming.- 8.1 KT Conditions for Standard QP.- 8.2 The Primal Affine Scaling Method for Convex QP.- 8.3 The Primal-Dual Method for Convex QP.- 8.4 Active Set Method.- 8.5 Active Set Method for the Dual QP Problem.- 8.6 Appendix-Derivation of the Descent Direction Formula for the PAS Method.- 8.7 Problems.- 9 Constrained Nonlinear Problems.- 9.1 Normalization.- 9.2 Penalty Methods.- 9.3 Linearization of a Nonlinear Problem.- 9.4 Sequential Linear Programming-SLP.- 9.5 Basic Sequential Quadratic Programming-SQP.- 9.6 Refined SQP Methods.- 9.7 Problems.- A.1 Basic Manipulations in Mathematica.- A.2 Lists and Matrices.- A.3 Solving Equations.- A.7 Online Help.