Hands-On Mathematics for Deep Learning

Particle swarm optimization

This algorithm uses the concept of swarm intelligence, where you have a school of fish or a flock of birds. Let's suppose they are trying to find some food. They arrive at an area and spread out a bit, starting to look for food individually. When one of them finds food, it lets the others know so that they can join in.

Each individual in the population knows its current position and velocity and only keeps track of the previous best positions it has visited. The velocity vector determines the direction of the search, and if the individual has a high velocity, then it has a more explorative character, whereas if it has a low velocity, it has a more exploitative character.

At the start of each iteration, the whole ...