Why IBM’s AI Fact Sheets should be the industry standard

Every once in awhile an idea comes along that’s so good it makes you wonder why it took so long for someone to think of it. IBM’s AI Fact Sheets is one of those ideas.

AI Fact Sheets are a lot like packaged food nutrition labels. They contain information about an AI model’s development, capabilities, benchmark performance, and more.

Big Blue today announced its plans to “commercialize key automated documentation capabilities from IBM Research’s AI Factsheets methodology into Watson Studio in Cloud Pak for Data throughout 2021.”

In other words: businesses and developers using Watson Studio in Cloud Pak for Data will soon have access to an automated AI Fact Sheets tool to create transparency and info reports.  The tool would generate most, if not all, of the AI Fact Sheet’s information automatically.

Per IBM:

The goal of the FactSheet project is to foster trust in AI by increasing transparency an increased understanding of how AI was created and deployed and enabling governance the ability to control how AI is created and deployed. Increased transparency provides information for AI consumers to better understand how the AI model a program component that is generated by learning patterns in training data to make predictions on new data, such as a loan application. or service an executable program, deployed behind an API, that allows it to respond to program requests from other programs or services was created. This allows a consumer of the model to determine if it is appropriate for their situation.

Quick take: We love this idea. While it’s a bit more complex than we can get into in this article (research paper here), the bottom line is that anything that standardizes transparency in machine learning models is a good thing.

This comes as part of IBM’s “AI Governance” initiative along with new AI consultation services. The company also announced several interesting new Watson capabilities.

Today we announced Watson Discovery's new Reading Comprehension feature in beta. It can identify pinpointed answers in response to natural language queries from a plethora of complex enterprise documents. https://t.co/Hunb0jPtBr #ai #ml #nlp

— Fredrik Tunvall (@tunvall) December 9, 2020

Final thoughts: We’ve heard a lot about the “black box” over the years, but the truth of the matter is that the black box isn’t hiding much. The real issue with AI at the big business scale is that there’s no standard for explaining what an AI model specifically does and how well it can do it.

Realistically, the model with the best marketing team is likely to beat out the best model if nobody can objectively explain which is best-suited for a specific application.

AI Fact Sheets is a blueprint for an industry standard to directly address these issues.