We’ve surrounded the Earth with surveillance satellites, but who is that good for?

“Earthrise, that iconic photograph snapped from the Apollo 8 space capsule 50 years ago on a Christmas-eve orbit around the Moon, forced a self-absorbed species to reflect on its fragility.

“Prior to that image, people had a perception of the planet being essentially infinite in its capacity to take all the damage we dish out,” recalls John Amos. A geoscientist who worked for years helping oil companies scout prospects from space, Amos was among a generation inspired by the “overview effect” to shift into activism. He left industry to launch an environmental-surveillance nonprofit called SkyTruth.

“Seeing that vanishingly small green-and-blue dot surrounded by the absolute, almost horrifying blackness of open space—that made a lot of people think we need to be wielding our awesome power more wisely,” he says. Along

with aerial views of an enormous oil spill off the coast of Santa Barbara the following month, ‟Earthrise” helped spur the creation of Earth Day and the major environmental-protection laws of the 1970s. Such is the power of a transformative shift in perspective enabled by a major leap in engineering.

Half a century later, the arcs of fast-rising technologies—commercial rockets, cubesats, and drones, plus cloud computing, machine learning, and the Internet of Things—are now intersecting in ways that will again indelibly alter how we see the world and our place in it.

It is no longer just advanced militaries and rich corporations who can keep tabs on what people are up to half a world away. Watchdogs such as Global Forest Watch, Global Fishing Watch, and SkyTruth are combing through satellite photos and radar scans to alert authorities to illegal clear-cutting, rogue fishing, mountaintop removals, and other environmental misbehavior.

Researchers at the Cornell Lab of Ornithology have exploited Amazon cloud servers to assemble millions of amateur birdwatcher reports into exquisite animated maps that plot the changing abundance of 122 bird species throughout North America. Ranchers are stapling health-monitoring microchips to their livestock. Beekeepers are sticking wireless sensors into their hives.

Farmers are planting high-tech electronics into the soil along with their crops. Autonomous sailboats now ply the Southern Ocean to look for krill swarms, and AI-enhanced drones patrol elephant habitats in southern Africa to spot poachers on the prowl.

“We are on the curling edge of the wave,” Amos says. The power of the ‟Earthrise” image came, in part, from the humbling invisibility of human existence in it. “This new instrumentation is more about seeing what people are up to in the environment,” he says. “The promise of this convergence in technology lies in imposing radical transparency on corporate activity and supply chains everywhere.” As SkyTruth’s motto goes, “If you can see it, you can change it.”

Or so one would hope. One lesson from past revolutions in social media and other technologies is that “Power always learns, and powerful tools always fall into its hands,” Zeynep Tufekci of the Berkman Klein Center for Internet and Society wrote recently in MIT Technology Review. In less than a decade, starry-eyed promises inspired by Internet-organized protests in Tahrir Square gave way to the grim realities of data trafficking, cyberbullying, election meddling, and surveillance states.

Radical transparency in our online lives has frayed, not tightened, the fabric of societies. And now, in the unrelenting march of technological monitoring, things are getting real.

As sprawling webs of sensors, cameras, and satellites give some (but not others) a near-real-time view from everywhere, certain parallels to recent history are unmistakable—and unnerving. Data-gathering startups are popping up all over, but they are locked in cutthroat races to monetize what they tout as “intelligence.” Tech giants are “indexing the Earth” by hoovering up satellite images the moment they are released to public access and then pouring the data into vast archives.

Though today these companies often beneficently grant free access to selected researchers and nonprofits, their long-term business strategies hinge on converting data into dollars. And those who are being observed and recorded are not the customers—they are the products. Does any of this sound familiar?

Yet history is not destiny. If we anticipate many of the ways that these new windows on the world might be misused, it should be possible to set up rules and agencies that will encourage the best uses while thwarting bad actors. Surprisingly, experts say that might not be as hard as it sounds.

The unblinking eye

Online instrumentation of the built and natural environment is extending our view of the real world in three dimensions at once: in extent, in detail, and in time. We are fast approaching a moment when, for a price, you can put eyes on any given spot on the planet on any given day, then scroll back through time to see how that place has changed.

Advances in machine-learning algorithms have made it possible to train an unblinking AI gaze on a location of interest and get automated alerts when the software spots something amiss. One can now tag, say, a whaling ship and follow it from port to port. With each passing year, this tech makes it easier to conduct oversight—in the most literal sense—of those who exploit the land and sea.

What’s more, the number of eyes in orbit is skyrocketing, driven by launch costs that have plummeted 90 percent since the Space Shuttle era. Europe’s Copernicus program has fielded six Earth-observing satellites, called Sentinels, since 2014. It plans to expand the fleet to 30 within the next decade. Last year, China added six high-resolution optical and infrared imaging satellites to its fast-growing constellation.

In the US, the most dramatic growth in environmental surveillance is happening in the private sector. San Francisco–based Planet Labs has launched 331 Earth-observing satellites since it spun up in 2013. Nearly 150 of Planet’s cubesats, each roughly the size of a loaf of bread, took to space in 2018 alone. (For comparison, 173 satellites owned by governments were operating at the start of 2016.)

The company now collects 300 million square kilometers of imagery every day. That’s nearly three-fifths the surface area of the Earth.

And at the same time as the number of images is exploding, so is the number of pixels in each square kilometer photographed. The smallest detail visible from Landsat 8, NASA’s flagship Earth-imaging spacecraft, is 15 meters—the size of a city bus.

Today Planet’s largest orbiting cameras can pick out a person sunbathing in their backyard, though not the interesting bits. Its competitor, Maxar, snaps photos from orbit at 30-centimeter resolution, sharp enough to see people and large animals on the hoof. Views from the newest Chinese satellites are thought to be sharper still.

Such surveillance works only when skies are sunny. But soon, constellations of very high-resolution satellites will use radar to peer through clouds and the dark of night as well. “That kicks open some doors,” Amos says. In particular, high-def radar can light up the vast “dark fleet” of fishing vessels that refuse to broadcast their positions via radio transponders.

Armed with new ways of seeing and with software that can merge surveillance quickly from multiple sources, watchdogs now have more opportunities to illuminate shady behavior while there is still time to do something about it. “We’re just now seeing the beginning of what is going to become possible,” he says.

In January, the Center for Strategic and International Studies reported that it had used returns from a space-based radar—along with tracking data from radio, visible, and infrared satellites—to monitor fishing activity among the disputed Spratly Islands in the South China Sea.

The study revealed that the number of boats operating surreptitiously in the area is “exponentially higher” than those broadcasting as they should. A large fraction of the vessels that look like fishing boats, the investigators concluded, are actually militia patrolling the waters on behalf of China.

A different group of researchers working with Global Fishing Watch recently combined observations from multiple satellites to uncover illicit Chinese behavior of a different kind. In a study soon to be published, “[W]e show that Chinese ships are fishing for squid in North Korean waters, almost certainly in violation of UN sanctions, while the North Koreans are fishing for squid in Russian waters,” says Paul Woods, the team’s chief technology officer.

“There’s no way at this scale that there isn’t a financial transaction between the North Koreans and businesses in China,” he suspects, despite international accords prohibiting such trade.

In addition to getting clearer views of the present, activists are now able to exploit cloud computing to replay the past from extensive image archives maintained by Google, Amazon, and various government agencies. Amazon makes all data from Landsat 8 freely available on its cloud storage service.

Websites such as EO Browser have taken advantage of the open-use policy to create services that let you zoom in to any location, enter a time range, pick the satellite sources you like, and then produce a time-lapse video of how the scene has evolved.

Though the search engine and imagery are currently free, thanks to financial backing by the European Union, they do not include the more detailed views offered by companies such as Planet and Maxar.

But commercial firms have been working with watchdogs and government agencies on projects that demonstrate the potential of this technology, which can help attract paying customers. At the Rights Lab at the University of Nottingham, for example, researchers have been scouring image series supplied by Planet and Maxar for environmental evidence of slave labor at brick kilns in India, in cotton fields in Turkmenistan, and in mangrove-displacing shrimp farms in Bangladesh.

Google has lent its Earth Engine system to Global Forest Watch, Global Fishing Watch, and a new surface-water viewer developed by the European Commission and the UN Environment Program.

The viewer, developed over three years and 10 million hours of computing time, maps disappearing streams and dwindling lakes worldwide over the past 35 years. Scroll over northern California in the viewer, and the demise of Goose Lake is unmistakable. Click on the lake, and year-by-year measurements reveal that the 2008 drought sounded its death knell.

Access to space has become cheap enough that better-funded nonprofits can now put their own birds into the sky. The Environmental Defense Fund plans to launch by 2021 a satellite it is building to measure methane leaking from oil and gas production sites worldwide, revisiting each site at least once a week.

To keep tabs on things that can’t yet be seen from space—or to follow up on interesting sightings—some activist groups are deploying drones into the skies and oceans. Last winter, autonomous sailing robots built by Saildrone set off from South America to circumnavigate Antarctica.

The months-long mission gathers data for scientists at NOAA and CSIRO on how fast the Southern Ocean is absorbing CO₂ from the atmosphere, a crucial variable in global climate models. The floating drones are also mapping the abundance of phytoplankton and krill, which form the base of the marine food web.

Also in 2018, biologists reviewing satellite images of the Danger Islands off the northern tip of the Antarctic Peninsula spotted what looked like a previously unknown megacolony of Adélie penguins. Researchers sailed to the islands and flew drones over the colony, confirming the discovery of 750,000-plus breeding pairs.

One of the biggest challenges in environmental surveillance has been digging through haystacks for elusive needles. “Society already struggles with too much data, not enough information, Amos says. “And this is certainly happening in remote sensing as well.”

After many years of false starts, artificial intelligence finally seems ready to help solve this problem. In southern Africa, for example, the Lindbergh Foundation’s Air Shepherd program had high hopes that, by flying surveillance drones equipped with infrared scanners over elephant and rhino habitats, its teams could catch poachers before the poachers caught their prey.

But the monitoring teams found it hard to stare for hours on end at shaky, grainy, night-vision video. Hunters often slipped past them.

So the group linked up with engineers at Microsoft, Carnegie Mellon, and UCLA to train an AI neural network to detect both poachers and wild animals in their recorded videos. Once they got the system working well in the lab, they tested it in the field in South Africa. It worked so well that they are now using it in national parks in Botswana and other African countries.

Global Fishing Watch has used AI to distinguish fishing vessels from cargo and naval ships. A research team at Stanford reported in April that it had fed aerial photos of North Carolina farmland into a deep-learning system to find almost 600 industrial livestock farms that manual mapping had missed.

Such concentrated feeding operations are a major source of freshwater pollution, in part because 60 percent of them operate without discharge permits, according to the EPA. In principle, regulators could use the AI to survey other states as well and to identify new operations as they pop up.

All of these examples, and many others like them, are tremendously encouraging. They tempt us to envision a happier future in which the instrumentation of nature draws humans into a more synoptic, and yet more intimate, connection to our home planet—one where Gaia itself gains a voice and a Facebook account. These systems could help people routinely band together to watch over ecosystems and organisms they care about deeply, despite never having experienced them directly.

Yet Mariel Borowitz, a space-policy researcher at Georgia Tech and author of the book Open Space, sounds a note of caution. The corporations building these technologies “are creating new kinds of data, new use cases, new users. But they are companies, so they are selling that data to make a profit.” And the revenue generated by environmental protection projects shows up on their books as little more than rounding error.

We should take care not to overestimate the protective power of public awareness nor underestimate how the technology will amplify the power of big industry and bad actors. That is essentially the mistake we made with social media.

Beneath the silver lining, a dark cloud

In the 20 years since Amos founded SkyTruth, his team has exposed rampant fracking activity, illegal gas-flaring, a decades-long oil spill, illegal fishing around Easter Island, and numerous other kinds of violations. Some of those exposés have triggered official responses. Yet the latter half of SkyTruth’s “If you can see it, you can change it” motto remains mostly aspirational.

“Although we’ve been effective at raising public awareness about certain issues,” Amos says, “we haven’t had a big impact in altering corporate behavior.” Environmental exploitation remains highly profitable, and profitable businesses find ways to protect themselves.

The same can be said of the technology industry. Among the tech giants, “everyone wants to be central, essential, and in control of your world,” security expert Bruce Schneier writes in his recent book Click Here to Kill Everyone, because “control equals profits.”

Google and Baidu became behemoths by erecting themselves as the gateways to the Web; Amazon and Alibaba as the gateways to commerce; Facebook as the gateway to friends and family. How much more valuable would it be to occupy the position of gateway to the planet?

That idea seems not to have escaped Amazon and Google—which, according to Borowitz and Woods, have been downloading essentially all the Earth observations that NASA, NOAA, the European Space Agency, and other agencies make available to the public. Amazon is even building a global network of 24 large antennas to download data directly from some of the satellites that gather it.

Planet Labs has been open about its long-term commercial strategy. CEO Will Marshall said last year that “Planet will index physical change on Earth the same way Google indexed the Internet.”

Not to be left out, Facebook’s AI team has been combining space imagery with public records to map the population of every community on Earth. That effort would seem to serve the company’s long-held goal to get the billions of people who currently lack Internet access online—and on Facebook.

None of this is necessarily a bad thing. Who could resist the convenience of Amazon’s Alexa observing from orbit that your roof needs replacing or your windows could use washing, and offering to schedule the work? If Google noticed you heading out on a backcountry hike and offered to automatically summon help if you appear to get lost or injured, would you refuse?

There will be countless ways that the tech giants can use the view from everywhere to make our lives slightly safer, cheaper, or more convenient. Most have not yet been conceived.

But let us pause to remember that Amazon, Google, and Facebook grew to become the third-, fourth-, and fifth-most valuable companies in the world by pitching ads directly at the people most likely to act on them. Inference equals influence: the product that the tech companies sell to their customers is their ability to infer how we live, where we go, what we do.  Imagine the value added to that product when it also captures our interactions with the physical world.

A rush to exploit remote sensing for advertising may inevitably allow bad actors to target us in more harmful ways. Witness Facebook’s Cambridge Analytica fiasco and data breaches in September 2018, and again in April 2019, that exposed records—including location data—on hundreds of millions of its users. And recall how in 2017 Google tracked Android users, even when they had disabled location sharing. The day may arrive when “going off the grid” is no longer possible.

The biggest customers for sensing data are governments and resource-extraction industries, and that is unlikely to change, Borowitz says.

Sixty percent of Maxar’s business comes from the US military, she notes. Planet Labs has earned tens of millions of dollars selling imagery to the US National Geospatial-Intelligence Agency. Governments have often demanded exclusive access to the imagery they purchase. It stands to reason that as environmental sensing becomes commercially more valuable or politically more embarrassing, those who pay will want to keep it to themselves.

It doesn’t take much imagination to envision ways in which this technology could work against conservation efforts even more effectively than it aids the watchdogs. As the costs of drones and satellite images fall and the performance of AI-driven identification rises, whalers, fishers, and poachers may find them to be powerful tools for guiding their hunts. “That’s definitely a concern,” Woods says, “particularly for elephants, rare whales, or rhinos.”

Commodities traders and financial analysts already hone their forecasts by analyzing Planet and Maxar data to quantify the levels of fuel-storage tanks at refineries, the movement of shipping containers at ports, and the heat emitted by factories. Might land speculators use similar techniques to drive up the prices of wind and solar farms by scooping up the best sites and transmission rights-of-way?

If past is prologue, regulators will lag far behind the tech giants and the resource extractors in constraining any destructive practices. “The big tech companies are spending record amounts of money lobbying in Washington,” Schneier writes—“many times more than oil companies, defense contractors, and everyone else.” One result, he observes, is that “regulations are consistently co-opted. Instead of promoting the common good, they’re aimed at promoting some private agenda.”

Watching the watchers

The good news is that none of these bad things has happened yet. Nor are they inevitable. Several models suggest how to prevent environmental surveillance from going sideways. Space exploration and scientific research offer two useful examples. And the wireless-communications and financial industries provide complementary ideas worth considering.

The earliest satellites, from Sputnik on, were launched in a race to space, and a spirit of free competition has kept space open ever since. So far, 35 countries have lofted Earth-facing satellites. “Everyone with a reasonably legitimate need gets to access low-Earth orbit and put hardware in space,” Amos notes. “It’s just as important that there won’t be shutter control imposed on [those] remote-sensing systems.”

The open-skies policy of space exploration could be extended to guarantee that no big player can exclude its competitors or critics from access to unfiltered observations—or to the computer storage and processing capacity needed to analyze them. There’s no avoiding the fact that we look at the world through a prism. But we should be free to switch prisms and compare different perspectives.

Scientific research has long embraced a similar principle—and it has been one of the greatest strengths of that enterprise. And financial regulators require public companies to be transparent about their performance so that the playing field is level for all investors. Similar rules could protect sensor data against undisclosed conflicts of interest and outright fraud.

Today, however, open access to both raw data and open-source software to manipulate it is not the norm. Borowitz surveyed the data policies on the use of unclassified imagery from government satellites and found that data was freely available for fewer than half of them. Governments even hold back data crucial to updating climate models because they want to sell it for commercial use, she reports.

Whereas Europe puts all of its Sentinel imagery in the public domain, only 10 percent of NOAA data was available online as of 2016. Since then, NOAA has implemented the Big Data Project, through which cloud providers host NOAA data and make it available to the public. They cannot charge for the actual data, but they can charge end users for distribution of the data and for processing and applications.

The volume of data presents one hurdle to meaningful open access. NOAA satellites alone generate 20 terabytes of data daily. “The amount of Earth-sensing data released by NASA is going from 20 petabytes today to 250 PB by 2020,” she says.

One petabyte—a billion megabytes—would take several years to download over a fast fiber-optic broadband connection. Amazon, Google, Microsoft, and others already offer competing cloud-computing platforms that can do the job.

But this raises a second, thornier issue. When data that is too massive for nonprofits and individuals to download is generated by government, stored on private servers, and digested by profit-making companies, who controls it? And who pays?

“Space is a global public commons,” Amos points out. He suggests that governments could demand license fees for the right to collect data from space or other public areas, much as broadcasters and wireless communications companies pay to use the electromagnetic spectrum. The fees could then help cover the costs of hosting the data for all to use.

Though Amos, Woods, and Borowitz all recognize the potential unintended downsides of the emerging view from everywhere, they remain optimistic about what it means for conservation. “The environmental bad guys already have ways to do what they want,” Woods says. “As long as the information stays open, I believe these tools will benefit the small players more.”

This article was originally published on Anthropocene Magazine by W. Wayt Gibbs,  a freelance science writer and editor based in Seattle. He is a contributing editor with Scientific American and editorial director at Intellectual Ventures. His work has appeared in Science, Nature, Discover, IEEE Spectrum,and The Economist.