How two startups may just have realized Google’s founders’ 14-year old dream

Charity Engine, a startup that launched at last year’s The Next Web Conference, has partnered with its first paying customer – and maybe even realized a prediction made by Larry Page and Sergey Brin back at the beginning of Google’s life.

Charity Engine allows volunteers to let third parties use spare processor capacity on their computers in return for the chance of winning a cash prize, while raising money for charities. You can read more about it in our earlier coverage, and in this video interview.

Now Datafiniti, a data-focused search engine, has harnessed Charity Engine to expand its Web crawling capacity by 50%, and increase its search speed – what appears to be the first time that a commercial webcrawler has been based on a volunteer grid of distributed computing power.

Manchester, UK-based Charity Engine’s CEO Mark McAndrew says that he believes this means that a prediction made in Page and Brin’s Stanford paper about Google in 1998 has finally come true. The paper noted:

“Of course a distributed system… will often be the most efficient and elegant technical solution for indexing, but it seems difficult to convince the world to use these systems because of the high administration costs of setting up large numbers of installations.

“Of course, it is quite likely that reducing the administration cost drastically is possible. If that happens, and everyone starts running a distributed indexing system, searching would certainly improve drastically.”

McAndrew believes that Charity Engine’s volunteer-powered network, which anyone can hire, realises that dream. The problem of getting people to contribute their processor cycles is solved by offering a cash prize and charity donations as a carrot.

While it’s still early days for the startup, which opened to the public three months ago, he says that it has 2200 machines on its grid, averaging 5000 computing cores providing 100,000 core hours per day, charged at a penny each.

With Charity Engine set to give away its second $10,000 prize to one of its participants in the next few weeks, it may well be worthwhile adding your computer to its grid, helping out Datafiniti – and a bunch of charities – along the way.

➤ Charity Engine