What is Optimization?

• Making something better.
• Increase efficiency.

Optimization problem

• A problem in which we have to find the values of inputs (also called solutions or decision variables) from all possible inputs in such a way that we get the “best” output values.
• Definition of “best”- Finding the values of inputs that result in a maximum or minimum of a function called the objective function.
• There can be multiple objective functions as well (depends on the problem).

Optimization algorithm

An algorithm used to solve an optimization problem is called an optimization algorithm.

Evolutionary Algorithms

Algorithms that simulate physical and/or biological behavior in nature to solve optimization problems.

Genetic Algorithm (GA)

• It is a subset of evolutionary algorithms that simulates/models Genetics and Evolution (biological behavior) to optimize a highly complex function.
• A highly complex function can be:
• 1. Very difficult to model mathematically.
• 2. Computationally expensive to solve. Eg. NP-hard problems.
• 3. Involves a large number of parameters.

Background of GA

• Introduced by Prof. John Holland in 1965.
• The first article on GA was published in 1975.
• GA is based on two fundamental biological processes:
• 1. Genetics (by G.J. Mendel in 1865): It is the branch of biology that deals with the study of genes, gene variation, and heredity.
• 2. Evolution (by C. Darwin in 1875): It is the process by which the population of organisms changes over generations.

Natural selection in Evolution

1. A population of individuals exists in an environment with limited resources.
2. Competition for those resources causes the selection of those fitter individuals that are better adapted to the environment.
3. These individuals act as seeds for the generation of new individuals through recombination and mutation.
4. Evolved new individuals act as initial population and Steps 1 to 3 are repeated.

Nature-GA Analogy

Structure of GA

GA vs Traditional Algorithms

Applications of GA

• 1. Acoustics
• 2. Aerospace Engineering
• 3. Financial Markets
• 4. Geophysics
• 5. Materials Engineering
• 6. Routing and Scheduling
• 7. Systems Engineering

Problems with GA

• 1. Population Selection Problem
• 2. Defining Fitness Function
• 3. Premature or rapid convergence of GA
• 4. Convergence to Local Optima

#evolutionary-algorithms #data-science #genetic-algorithm #algorithm