AI Now 2019 report slams government use of facial recognition, biased AI

“Community groups, workers, journalists, and researchers—not corporate AI ethics statements and policies—have been primarily responsible for pressuring tech companies and governments to set guardrails on the use of AI.” AI Now 2019 report

AI Now’s 2019 report is out, and it’s exactly as dismaying as we thought it would be. The good news is that the threat of biased AI and Orwellian surveillance systems no longer hangs over our collective heads like an artificial Sword of Damocles. The bad news: the threat’s gone because it’s become our reality. Welcome to 1984.

The annual report from AI Now is a deep-dive into the industry conducted by the AI Now Institute at New York University. Its focused on the social impact that AI use has on humans, communities, and the population at large. It sources information and analysis from experts in myriad disciplines around the world and works closely with partners throughout the IT, legal, and civil rights communities.

This year’s report begins with twelve recommendations based on the institute’s conclusions:

• Regulators should ban the use of affect recognition in important decisions that impact people’s lives and access to opportunities.
• Government and business should halt all use of facial recognition in sensitive social and political contexts until the risks are fully studied and adequate regulations are in place.
• The AI industry needs to make significant structural changes to address systemic racism, misogyny, and lack of diversity.
• AI bias research should move beyond technical fixes to address the broader politics and consequences of AI’s use.
• Governments should mandate public disclosure of the AI industry’s climate impact.
• Workers should have the right to contest exploitative and invasive AI—and unions can help.
• Tech workers should have the right to know what they are building and to contest unethical or harmful uses of their work.
• States should craft expanded biometric privacy laws that regulate both public and private actors.
• Lawmakers need to regulate the integration of public and private surveillance infrastructures.
• Algorithmic Impact Assessments must account for AI’s impact on climate, health, and geographical displacement.
• Machine learning researchers should account for potential risks and harms and better document the origins of their models and data.
• Lawmakers should require informed consent for use of any personal data in health-related AI.

The permeating theme here seems to be that corporations and governments need to stop passing the buck when it comes to social and ethical accountability. A lack of regulation and ethical oversight has lead to a near total surveillance state in the US. And the use of black box systems throughout the judiciary and financial systems has proliferated even though such AI has been proven to be inherently biased.

AI Now notes that these entities saw a significant amount of push-back from activist groups and pundits, but also points out that this has done relatively little to stem the flow of harmful AI:

Despite growing public concern and regulatory action, the roll-out of facial recognition and other risky AI technologies has barely slowed down. So-called “smart city” projects around the world are consolidating power over civic life in the hands of for-profit technology companies, putting them in charge of managing critical resources and information.

For example, Google’s Sidewalk Labs project even promoted the creation of a Google-managed citizen credit score as part of its plan for public-private partnerships like Sidewalk Toronto. And Amazon heavily marketed its Ring, an AI-enabled home-surveillance video camera. The company partnered with over 700 police departments, using police as salespeople to convince residents to buy the system. In exchange, law enforcement was granted easier access to Ring surveillance footage.

Meanwhile, companies like Amazon, Microsoft, and Google are fighting to be first in line for massive government contracts to grow the use of AI for tracking and surveillance of refugees and residents, along with the proliferation of biometric identity systems, contributing to the overall surveillance infrastructure run by private tech companies and made available to governments.

The report also gets into “affect recognition” AI, a subset of facial recognition that’s made its way into schools and businesses around the world. Companies use it during job interviews to, supposedly, tell if an applicant is being truthful and on production floors to determine who is being productive and attentive. It’s a bunch of crap though, as a recent comprehensive review of research from multiple teams concluded.

Per the AI Now 2019 report:

Critics also noted the similarities between the logic of affect recognition, in which personal worth and character are supposedly discernible from physical characteristics, and discredited race science and physiognomy, which was used to claim that biological differences justified social inequality. Yet in spite of this, AI-enabled affect recognition continues to be deployed at scale across environments from classrooms to job interviews, informing sensitive determinations about who is “productive” or who is a “good worker,” often without people’s knowledge.

At this point, it seems any company that develops or deploys AI technology that can be used to discriminate – especially black box technology that claims to understand what a person is thinking or feeling – is willfully investing in discrimination. We’re long past the time that corporations and governments can feign ignorance on the matter.

This is especially true when it comes to surveillance. In the US, like China, we’re now under constant public and private surveillance. Cameras record our every move in public at work, in our schools, and in our own neighborhoods. And, worst of all, not only did the government use our tax dollars to pay for all of it, millions of us unwittingly purchased, mounted, and maintained the surveillance gear ourselves. AI Now wrote:

Amazon exemplified this new wave of commercial surveillance tech with Ring, a smart-security-device company acquired by Amazon in 2018. The central product is its video doorbell, which allows Ring users to see, talk to, and record those who come to their doorsteps. This is paired with a neighborhood watch app called “Neighbors,” which allows users to post instances of crime or safety issues in their community and comment with additional information, including photos and videos

A series of reports reveals that Amazon had negotiated Ring video-sharing partnerships with more than 700 police departments across the US. Partnerships give police a direct portal through which to request videos from Ring users in the event of a nearby crime investigation.

Not only is Amazon encouraging police departments to use and market Ring products by providing discounts, but it also coaches police on how to successfully request surveillance footage from Neighbors through their special portal. As Chris Gilliard, a professor who studies digital redlining and discriminatory practices, comments: “Amazon is essentially coaching police on . . . how to do their jobs, and . . . how to promote Ring products.

The big concern here is that the entrenchment of these surveillance systems could become so deep that the law enforcement community would treat their extrication the same as if we were trying to disarm them.

Here’s why: In the US, cops are supposed to get a warrant to invade our privacy if they suspect criminal activity. But they don’t need one to use Amazon’s Neighbors app or Palantir’s horrifying LEO app.  With these, Police can essentially perform digital stop-and-frisks on any person they come into contact with using AI-powered tools.

AI Now warns that these problems — biased AI, discriminatory facial recognition systems, and AI-powered surveillance — cannot be solved by patching systems or tweaking algorithms. We can’t “version 2.0” our way out of this mess.

In the US, we’ll continue our descent into this Orwellian nightmare as long as we continue to vote for politicians that support the surveillance state, discriminatory black-box AI systems, and the Wild West atmosphere that big tech exists in today.

Amazon and Palantir shouldn’t have the ultimate decision over how much privacy we’re entitled to.

If you’d like to read the full 60-page report, it’s available online here.