Kaggle is a very unique company. The firm has collected a global network of brilliant minds, who compete in contests set up by companies looking to find new correlations in their business data. For example, a car company might want to find patterns in their customers, potentially flagging the least trustworthy.
Here’s how it works: Two sets of data are provided, one that has the listed outcome (the loan defaulted, or what have you), and one that does not. The competing analysts construct an algorithm using the data that includes results, and then apply it to the second set of data that lacks the result.
The bets algorithm wins, in terms of predicting the correct result. Of course, both sets of data are historical data, so the company running the contest knows the real results. Prize money is distributed so that, instead of winner takes all, many top answer receive various amounts of prize cash. This keeps people fighting to hone their algorithm, and improve their score, and thus move up the list of top results.
Today Kaggle has announced a new product, Prospect, that is opening a new avenue for the company. With Prospect, companies will provide massive data sets and allow for analysts to comb it for the best way to proceed. Whatever is the most promising direction is decided by all the data scientists, by voting. Then, the company can run a regular Kaggle competition with that vein.
From Kaggels’ press release:
Kaggle Prospect extends Kaggle’s crowdsourcing model to a new part of the competition lifecycle by giving organizations an opportunity to share their raw data with Kaggle’s community of scientists, who will suggest how machine learning can be used to leverage that data.
Essentially, Prospect is for companies who have data, and want to improve their business, but might not know exactly how best to proceed in turing that data into decisions. Kaggle allows for competitions to be private, with only top analysts invited, for firms with more sensitive data, or more secretive policies.
What’s fun about Kaggle is that the winners of these competitions are folks around the world with a knack for problem solving, and not always degrees in mathematics. And degrees don’t matter on Kaggle; all that matters is result. Prospect is an interesting idea. If Kaggle can convince its customers to make more data available, and its motley crew of thinkers can manage the huge sets of information, it could grow the company’s business greatly.