Google’s AutoML Zero lets the machines create algorithms to avoid human bias

It looks like Google’s working on some major upgrades to its autonomous machine learning development language ‘AutoML.’ According to a pre-print research paper authored by several of the big G’s AI researchers, ‘AutoML Zero’ is coming, and it’s bringing evolutionary algorithms with it.

AutoML is a tool from Google that automates the process of developing machine learning algorithms for various tasks. It’s user-friendly, fairly simple to use, and completely open-source. Best of all, Google’s always updating it.

In its current iteration, AutoML has a few drawbacks. You still have to manually create and tune several algorithms to act as building blocks for the machine to get started. This allows it to take your work and experiment with new parameters in an effort to optimize what you’ve done. Novices can get around this problem by using pre-made algorithm packages, but Google’s working to automate this part too.

Per the Google team’s pre-print paper:

It is possible today to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space.

Despite the vastness of this space, evolutionary search can still discover two-layer neural networks trained by backpropagation. These simple neural networks can then be surpassed by evolving directly on tasks of interest, e.g. CIFAR-10 variants, where modern techniques emerge in the top algorithms, such as bilinear interactions, normalized gradients, and weight averaging.

Moreover, evolution adapts algorithms to different task types: e.g., dropout-like techniques appear when little data is available.

In other words: Google’s figured out how to tap evolutionary algorithms for AutoML using nothing but basic math concepts. The developers created a learning paradigm in which the machine will spit out 100 randomly generated algorithms and then work to see which ones perform the best.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

After several generations, the algorithms become better and better until the machine finds one that performs well enough to evolve. In order to generate novel algorithms that can solve new problems, the ones that survive the evolutionary process are tested against various standard AI problems, such as computer vision.

Read: Why the quickest path to human-level AI may be letting it evolve on its own

Perhaps the most interesting byproduct of Google’s quest to completely automate the act of generating algorithms and neural networks is the removal of human bias from our AI systems. Without us there to determine what the best starting point for development is, the machines are free to find things we’d never think of.

According to the researchers, AutoML Zero already outperforms its predecessor and similar state-of-the-art machine learning-generation tools. Future research will involve setting a more narrow scope for the AI and seeing how well it performs in more specific situations using a hybrid approach that creates algorithms with a combination of ‘Zero’s’ self-discovery techniques and human-curated starter libraries.