4THE LOTTERY TICKET HYPOTHESIS
What is the lottery ticket hypothesis, and, if it holds true, how is it useful in practice?
The lottery ticket hypothesis is a concept in neural network training that posits that within a randomly initialized neural network, there exists a subnetwork (or “winning ticket”) that can, when trained separately, achieve the same accuracy on a test set as the full network after being trained for the same number of steps. This idea was first proposed by Jonathan Frankle and Michael Carbin in 2018.
This chapter illustrates the lottery hypothesis step by step, then goes over weight pruning, one of the key techniques to create ...
Get Machine Learning Q and AI now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
