Kaggle is a crowdsourced data analysis competition platform. Businesses bring their data problems and Kaggle’s army of armchair data scientists compete to come up with the best solution.
It’s a fascinating idea which has so far seen contestants compete to solve problems ranging from analysing medical records to predicting which patients are likely to need hospitalization, to scanning the deep cosmos for traces of dark matter.
The San Francisco-based company were founded in 2010, inspired by a competition organized by Netflix the previous year. The streaming TV and movie company had challenged the public to come up with better algorithms to predict what their customers would like to watch next, to help them improve their own recommendation engines. Netflix have since gone on to use Kaggle to organize their later competitions.
Chief scientist of Google – one of the many companies which have used Kaggle’s services – Hal Varian has described it as: “a way to organize the brainpower of the world’s most talented data scientists and make it accessible to organizations of every size”.1
Anyone can register with Kaggle and enter most of their competitions, competing for prizes of up to $100,000 at the time of writing. Previous competitions such as the Heritage Health Prize in 2012 offered prizes of up to £500,000. Certain competitions, however, are reserved for “masters”: site members who have proved their mettle in previous competitions. ...