Optimization refers to minimizing or maximizing the objective function by systematically choosing the values of optimization variables from or within an allowed set defined by the constraint functions. Many engineering problems can be effectively characterized in the form of optimization. Thus, optimization theory is a powerful tool to solve engineering problems. In order to map from engineering problem to optimization issue, objectives, constraints, and variables should be extracted from the engineering problem and expressed in a mathematical fashion. Objective can be the key performance metric we care about. In wireless communication, objective can be capacity or throughput. For the radar system, detection rate can be the design goal. For smart grid, the total cost for purchasing power should be minimized. Constraints are the physical limits of the system or the performance requirements. Variables can be the adjustable or controllable parameters in the system, for example, weights, gains, power, and so on. Besides, optimality, feasibility, and sensitivity should also be taken into account. Reasonable constraints should be set for the optimization problem, and active constraints should be given more attention.
There are many categories of optimization formats: