book

Engineering Optimization

Name: Engineering Optimization
Author: R. Russell Rhinehart
ISBN: 9781118936337

by R. Russell Rhinehart

May 2018

Intermediate to advanced

776 pages

25h 42m

English

Wiley

Read now

Unlock full access

Cover
Title Page
Preface
IntroductionKey PointsBook AspirationsOrganizationRationale for the BookTarget AudiencePresentation Style
Acknowledgments
Nomenclature
About the Companion Website
Section 1: Introductory Concepts
1 Optimization
1.1 Optimization and Terminology1.2 Optimization Concepts and Definitions1.3 Examples1.4 Terminology Continued1.5 Optimization Procedure1.6 Issues That Shape Optimization Procedures1.7 Opposing Trends1.8 Uncertainty1.9 Over‐ and Under‐specification in Linear Equations1.10 Over‐ and Under‐specification in Optimization1.11 Test Functions1.12 Significant Dates in Optimization1.13 Iterative Procedures1.14 Takeaway1.15 Exercises
2 Optimization Application Diversity and Complexity
2.1 Optimization2.2 Nonlinearity2.3 Min, Max, Min–Max, Max–Min, …2.4 Integers and Other Discretization2.5 Conditionals and Discontinuities: Cliffs Ridges/Valleys2.6 Procedures, Not Equations2.7 Static and Dynamic Models2.8 Path Integrals2.9 Economic Optimization and Other Nonadditive Cost Functions2.10 Reliability2.11 Regression2.12 Deterministic and Stochastic2.13 Experimental w.r.t. Modeled OF2.14 Single and Multiple Optima2.15 Saddle Points2.16 Inflections2.17 Continuum and Discontinuous DVs2.18 Continuum and Discontinuous Models2.19 Constraints and Penalty Functions2.20 Ranks and Categorization: Discontinuous OFs2.21 Underspecified OFs2.22 Takeaway2.23 Exercises
3 Validation
3.1 Introduction3.2 Validation3.3 Advice on Becoming Proficient3.4 Takeaway3.5 Exercises

Section 2: Univariate Search Techniques
4 Univariate (Single DV) Search Techniques
4.1 Univariate (Single DV)4.2 Analytical Method of Optimization4.3 Numerical Iterative Procedures4.4 Direct Search Approaches4.5 Perspectives on Univariate Search Methods4.6 Evaluating Optimizers4.7 Summary of Techniques4.8 Takeaway4.9 Exercises
5 Path Analysis
5.1 Introduction5.2 Path Examples5.3 Perspective About Variables5.4 Path Distance Integral5.5 Accumulation along a Path5.6 Slope along a Path5.7 Parametric Path Notation5.8 Takeaway5.9 Exercises
6 Stopping and Convergence Criteria: 1‐D Applications
6.1 Stopping versus Convergence Criteria6.2 Determining Convergence6.3 Combinations of Convergence Criteria6.4 Choosing Convergence Threshold Values6.5 Precision6.6 Other Convergence Criteria6.7 Stopping Criteria to End a Futile Search6.8 Choices!6.9 Takeaway6.10 Exercises
Section 3: Multivariate Search Techniques
7 Multidimension Application Introduction and the Gradient
7.1 Introduction7.2 Illustration of Surface and Terms7.3 Some Surface Analysis7.4 Parametric Notation7.5 Extension to Higher Dimension7.6 Takeaway7.7 Exercises
8 Elementary Gradient‐Based Optimizers
8.1 Introduction8.2 Cauchy’s Sequential Line Search8.3 Incremental Steepest Descent8.4 Takeaway8.5 Exercises
9 Second‐Order Model‐Based Optimizers
9.1 Introduction9.2 Successive Quadratic9.3 Newton–Raphson9.4 Perspective on CSLS, ISD, SQ, and NR9.5 Choosing Step Size for Numerical Estimate of Derivatives9.6 Takeaway9.7 Exercises
10 Gradient‐Based Optimizer Solutions
10.1 Introduction10.2 Levenberg–Marquardt (LM)10.3 Scaled Variables10.4 Conjugate Gradient (CG)10.5 Broyden–Fletcher–Goldfarb–Shanno (BFGS)10.6 Generalized Reduced Gradient (GRG)10.7 Takeaway10.8 Exercises
11 Direct Search Techniques
11.1 Introduction11.2 Cyclic Heuristic Direct (CHD) Search11.3 Hooke–Jeeves (HJ)11.4 Compare and Contrast CHD and HJ Features: A Summary11.5 Nelder–Mead (NM) Simplex: Spendley, Hext, and Himsworth11.6 Multiplayer Direct Search Algorithms11.7 Leapfrogging11.8 Particle Swarm Optimization11.9 Complex Method (CM)11.10 A Brief Comparison11.11 Takeaway11.12 Exercises
12 Linear Programming
12.1 Introduction12.2 Visual Representation and Concepts12.3 Basic LP Procedure12.4 Canonical LP Statement12.5 LP Algorithm12.6 Simplex Tableau12.7 Takeaway12.8 Exercises
13 Dynamic Programming
13.1 Introduction13.2 Conditions13.3 DP Concept13.4 Some Calculation Tips13.5 Takeaway13.6 Exercises
14 Genetic Algorithms and Evolutionary Computation
14.1 Introduction14.2 GA Procedures14.3 Fitness of Selection14.4 Takeaway14.5 Exercises
15 Intuitive Optimization
15.1 Introduction15.2 Levels15.3 Takeaway15.4 Exercises
16 Surface Analysis II
16.1 Introduction16.2 Maximize Is Equivalent to Minimize the Negative16.3 Scaling by a Positive Number Does Not Change DV16.4 Scaled and Translated OFs Do Not Change DV16.5 Monotonic Function Transformation Does Not Change DV16.6 Impact on Search Path or NOFE16.7 Inequality Constraints16.8 Transforming DVs16.9 Takeaway16.10 Exercises
17 Convergence Criteria 2
17.1 Introduction17.2 Defining an Iteration17.3 Criteria for Single TS Deterministic Procedures17.4 Criteria for Multiplayer Deterministic Procedures17.5 Stochastic Applications17.6 Miscellaneous Observations17.7 Takeaway17.8 Exercises
18 Enhancements to Optimizers
18.1 Introduction18.2 Criteria for Replicate Trials18.3 Quasi‐Newton18.4 Coarse–Fine Sequence18.5 Number of Players18.6 Search Range Adjustment18.7 Adjustment of Optimizer Coefficient Values or Options in Process18.8 Initialization Range18.9 OF and DV Transformations18.10 Takeaway18.11 Exercises
Section 4: Developing Your Application Statements
19 Scaled Variables and Dimensional Consistency
19.1 Introduction19.2 A Scaled Variable Approach19.3 Sampling of Issues with Primitive Variables19.4 Linear Scaling Options19.5 Nonlinear Scaling19.6 Takeaway19.7 Exercises
20 Economic Optimization
20.1 Introduction20.2 Annual Cash Flow20.3 Including Risk as an Annual Expense20.4 Capital20.5 Combining Capital and Nominal Annual Cash Flow20.6 Combining Time Value and Schedule of Capital and Annual Cash Flow20.7 Present Value20.8 Including Uncertainty20.9 Takeaway20.10 Exercises
21 Multiple OF and Constraint Applications
21.1 Introduction21.2 Solution 1: Additive Combinations of the Functions21.3 Solution 2: Nonadditive OF Combinations21.4 Solution 3: Pareto Optimal21.5 Takeaway21.6 Exercises
22 Constraints
22.1 Introduction22.2 Equality Constraints22.3 Inequality Constraints22.4 Constraints: Pass/Fail Categories22.5 Hard Constraints Can Block Progress22.6 Advice22.7 Constraint‐Equivalent Features22.8 Takeaway22.9 Exercises
23 Multiple Optima
23.1 Introduction23.2 Solution: Multiple Starts23.3 Other Options23.4 Takeaway23.5 Exercises
24 Stochastic Objective Functions
24.1 Introduction24.2 Method Summary for Optimizing Stochastic Functions24.3 What Value to Report?24.4 Application Examples24.5 Takeaway24.6 Exercises
25 Effects of Uncertainty
25.1 Introduction25.2 Sources of Error and Uncertainty25.3 Significant Digits25.4 Estimating Uncertainty on Values25.5 Propagating Uncertainty on DV Values25.6 Implicit Relations25.7 Estimating Uncertainty in DV and OF25.8 Takeaway25.9 Exercises
26 Optimization of Probable Outcomes and Distribution Characteristics
26.1 Introduction26.2 The Concept of Modeling Uncertainty26.3 Stochastic Approach26.4 Takeaway26.5 Exercises
27 Discrete and Integer Variables
27.1 Introduction27.2 Optimization Solutions27.3 Convergence27.4 Takeaway27.5 Exercises
28 Class Variables
28.1 Introduction28.2 The Random Keys Method: Sequence28.3 The Random Keys Method: Dichotomous Variables28.4 Comments28.5 Takeaway28.6 Exercises
29 Regression
29.1 Introduction29.2 Perspective29.3 Least Squares Regression: Traditional View on Linear Model Parameters29.4 Models Nonlinear in DV29.5 Maximum Likelihood29.6 Convergence Criterion29.7 Model Order or Complexity29.8 Bootstrapping to Reveal Model Uncertainty29.9 Perspective29.10 Takeaway29.11 Exercises
Section 5: Perspective on Many Topics
30 Perspective
30.1 Introduction30.2 Classifications30.3 Elements Associated with Optimization30.4 Root Finding Is Not Optimization30.5 Desired Engineering Attributes30.6 Overview of Optimizers and Attributes30.7 Choices30.8 Variable Classifications30.9 Constraints30.10 Takeaway30.11 Exercises
31 Response Surface Aberrations
31.1 Introduction31.2 Cliffs (Vertical Walls)31.3 Sharp Valleys (or Ridges)31.4 Striations31.5 Level Spots (Functions 1, 27, 73, 84)31.6 Hard‐to‐Find Optimum31.7 Infeasible Calculations31.8 Uniform Minimum31.9 Noise: Stochastic Response31.10 Multiple Optima31.11 Takeaway31.12 Exercises
32 Identifying the Models, OF, DV, Convergence Criteria, and Constraints
32.1 Introduction32.2 Evaluate the Results32.3 Takeaway32.4 Exercises
33 Evaluating Optimizers
33.1 Introduction33.2 Challenges to Optimizers33.3 Stakeholders33.4 Metrics of Optimizer Performance33.5 Designing an Experimental Test33.6 Takeaway33.7 Exercises
34 Troubleshooting Optimizers
34.1 Introduction34.2 DV Values Do Not Change34.3 Multiple DV Values for the Same OF Value34.4 EXE Error34.5 Extreme Values34.6 DV Is Dependent on Convergence Threshold34.7 OF Is Irreproducible34.8 Concern over Results34.9 CDF Features34.10 Parameter Correlation34.11 Multiple Equivalent Solutions34.12 Takeaway34.13 Exercises
Section 6: Analysis of Leapfrogging Optimization
35 Analysis of Leapfrogging
35.1 Introduction35.2 Balance in an Optimizer35.3 Number of Initializations to be Confident That the Best Will Draw All Others to the Global Optimum35.4 Leap‐To Window Amplification Analysis35.5 Analysis of α and M to Prevent Convergence on the Side of a Hill35.6 Analysis of α and M to Minimize NOFE35.7 Probability Distribution of Leap‐Overs35.8 Takeaway35.9 Exercises
Section 7: Case Studies
36 Case Study 1
36.1 Process and Analysis36.2 Exercises
37 Case Study 2
37.1 The Process and Analysis37.2 Exercises
38 Case Study 3
38.1 The Process and Analysis38.2 Exercises
39 Case Study 4
39.1 The Process and Analysis39.2 Pre‐Assignment Note39.3 Exercises
40 Case Study 5
40.1 The Process and Analysis40.2 Exercises
41 Case Study 6
41.1 Description and Analysis41.2 Exercises
42 Case Study 7
42.1 Concepts and Analysis42.2 Exercises
43 Case Study 8
43.1 Description and Analysis43.2 Exercises
44 Case Study 9
44.1 Description and Analysis44.2 Exercises
45 Case Study 10
45.1 Description and Analysis45.2 Exercises
Section 8: Appendices
Appendix A: Mathematical Concepts and Procedures
A.1 Representation of RelationsA.2 Taylor Series Expansion (Single Variable)A.3 Taylor Series Expansion (Multiple Variable)A.4 Evaluating First Derivatives at x0A.5 Partial Derivatives: First OrderA.6 Partial Derivatives: Second OrderA.7 Linear Algebra NotationsA.8 Algebra and Assignment Statements
Appendix B: Root Finding
B.1 IntroductionB.2 Interval Halving: Bisection MethodB.3 Newton’s: Secant VersionB.4 Which to Choose?
Appendix C: Gaussian Elimination
C.1 Linear Equation SetsC.2 Gaussian EliminationC.3 PivotingC.4 Code in VBA
Appendix D: Steady‐State Identification in Noisy Signals
D.1 IntroductionD.2 Conceptual ModelD.3 Method EquationsD.4 Coefficient, Threshold, and Sample Frequency ValuesD.5 Type‐I ErrorD.6 Type‐II ErrorD.7 Alternate Type‐I ErrorD.8 Alternate Array MethodD.9 SSID and TSID VBA Code
Appendix E: Optimization Challenge Problems (2‐D and Single OF)
E.1 IntroductionE.2 Challenges for OptimizersE.3 Test FunctionsE.4 Other Test Function Sources
Appendix F: Brief on VBA Programming
F.1 IntroductionF.2 To StartF.3 GeneralF.4 I/O to Excel CellsF.5 Variable Types and DeclarationsF.6 OperationsF.7 LoopsF.8 ConditionalsF.9 DebuggingF.10 Run Buttons (Commands)F.11 Objects and PropertiesF.12 Keystroke MacrosF.13 External File I/OF.14 Solver Add‐InF.15 Calling Solver from VBA
Section 9: References and Index
References and Additional Resources
Books on OptimizationBooks on Probability and StatisticsBooks on SimulationSpecific TechniquesSelected Landmark PapersSelected Websites Resources
Index
End User License Agreement

Content preview from Engineering Optimization

Section 7Case Studies

Ten case studies are provided. My view is that the simple end‐of‐chapter exercises are appropriate to experiencing, and thereby understanding, isolated concepts. However, the real world of applications is not the doing of isolated exercises. These case studies are simple enough to be learning tools, but complicated enough to help a reader see how to structure and perform optimization within a situational context.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781118936337Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Engineering Optimization

by R. Russell Rhinehart

Section 7Case Studies

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.