Managing Big Data in Defense

“The goal is to let the commercial providers scoop the data, let the algorithms and industry partners give it the first few passes. Then have our analysts do what they do best – understand the world.” Robert Cardillo, Director of the US National Geospatial Intelligence Agency (NGA). Rick Adams writes.

Analysts of geospatial intelligence, aka geoint, are drowning in data. And the pipeline of available information continues to get bigger.

“In five years, there may be a million times more than the amount of geospatial data that we have today,” said Robert Cardillo, Director of the US National Geospatial Intelligence Agency (NGA). “If we were to attempt to manually exploit the commercial satellite imagery we expect to have over the next 20 years, we would need eight million imagery analysts.”

That number of trained analysts is probably not available, and not likely to be budgeted, so Cardillo’s goal is “to automate 75 percent of their tasks, so they have more time to analyze … so they can look much harder at our toughest problems – the 25 percent that require the most attention. The movement from pictures to pixels to data and the shift away from some of the highly manual work is a significant change not only to our processes, but also to our workforce.”

The current level of information captured from satellites, aircraft and other sources is further compounded by a tsunami of video and an increasing volume of three-dimensional data. Cardillo said, “Where we truly need augmentation most right now is Full Motion Video. FMV, as currently practiced, is a critical challenge to NGA and our entire profession. It’s time-consuming, manually intensive, redundantly exploited, poorly integrated, and it leaves a great deal of useful data unexploited and undiscovered. In other words, while it remains essential to national security, it’s both extremely costly and extremely inefficient.”

“This data deluge is not something to be afraid of,” he added. “The data itself isn’t the threat. Managed smartly and efficiently, it’s the solution – but it’s going to require us to change.”

“The Department of Defense already knows that they need to automate,” said Curt Davis, the founder of the University of Missouri’s Center for Geospatial Intelligence (CGI). “They now have more satellite imagery than their analysts can look over.”

Davis and CGI researchers adapted “deep learning models” such as GoogleNet and Microsoft’s ResNet, used for internet facial recognition, to deal with the peculiar challenges of satellite imagery, and were able to identify surface-to-air missile (SAM) sites in a nearly 90,000-square kilometer area of southeastern China (roughly the size of North Korea, coincidentally). The software algorithm approach was about 85 times more efficient than traditional human visual searching, reducing the time from 60 hours to 42 minutes while maintaining the 90 percent accuracy rate of imagery analysts and reducing the number of false positive identifications.

“All machine learning creates false positives,” Davis said. “But the focus of the study was to measure the deep-learning models against human performance. We wanted to test these deep learning methods on a realistic, real-world image analysis problem to critically assess their utility and potential impact. Historically machine-learning algorithms haven’t performed well when they have been applied to large satellite imagery datasets. The results were much better than we anticipated.”

In December, the US DoD began delivering artificial intelligence (AI) algorithms from an accelerated prototype programme, Project Maven, created in April to automate analysis of FMV gathered by drones. Lt. Gen. Jack Shanahan, Director for Defense Intelligence for Warfighter Support in the Office of the Undersecretary of Defense for Intelligence, described the goal as converting “the enormous volume of data available to DoD into actionable intelligence and insights. It’s about moving from the hardware industrial age to a software data-driven information environment and doing it fast and at scale.”

“Artificial intelligence is really a bit of a misnomer. It’s really machine learning that is applied to unstructured data, whether it’s text or imagery, and has reached a point where it’s able to convert vast quantities of unstructured data into analyzable bits and pieces of information – where changes have taken place within the last week or month or even in some cases within the last day,” Dr. Walter Scott told Military Simulation & Training. Scott is Executive Vice President and Chief Technology Officer of Maxar, the new name for an amalgam of leading geoint companies. “Coupled with cloud computing, it’s possible to apply those machine-learning algorithms at country or continent scale, so you’re really getting a picture of what’s happening across the entire planet.”

Satellite images complicate the problem for deep-learning algorithms, which typically work best with fixed image sizes. Human analysts must decide whether to resize the image and lose detail in the lower resolution, or crop the image to focus on one part. Before a computer can analyze a video, the project team must “train” the machine by “data tagging” each type of feature – as often as 100,000 times. Once taught, the machine autonomously recognizes features in images, such as a type of truck or a person carrying a weapon.

“I think the state of the technology right now enables a combination of man and machine to actually get to the answer,” noted Mike Warren, Co-founder and Chief Technology Officer of Descartes Labs, a spin-off of the US Department of Energy Los Alamos National Laboratory.

More than 100 defence and government geoint respondents agreed it is necessary to maintain both automated and manual control over intelligence analysis in a survey by British company WBR Research. They called for faster and more mobile applications of geoint to commanders on the ground and rated the automatic highlighting of significant changes on the ground as most desirable.

The US simulation and training community, frustrated with modelling the same “digital dirt” dozens of times for different programmes, is seeking to construct a single 3D geospatial database – known as One World Terrain (OWT), “the most realistic, accurate and informative representations of the physical and non-physical landscape” – for use in next-generation simulations and virtual environments. The “Netflix of training” scheme would utilize internet cloud solutions for storage and distribution, and should save time and funds when creating geo-specific datasets. The intent is also to incorporate social media data in real- or near-real-time in virtual or constructive simulation environments.

The US Intelligence Advanced Research Projects Agency (IARPA) conducted a competition last year for companies and academics to apply AI solutions to automatically identify objects of interest in satellite images. Competitors were provided with a publicly available data set of satellite imagery containing a million labeled objects, such as buildings and facilities.

The NGA has set up an “Outpost Silicon Valley” recruiting office in northern California to address the agency’s “human capital gap.” They are seeking to hire up to 100 data scientists and analysts to solve big data problems, “the kind that end up in presidential daily briefings or save lives during disasters or military conflicts,” according to Andy Brooks, chief data scientist at NGA. They also intend to hire several hundred full-time software engineers, developers and coders for the agency’s in-house software development team, and another 50 “explorers” who could be brought in for temporary stretches. “You’ll get to work on some crazy, high-stakes decisions and amazing projects. You’ll have an interesting story to tell,” said Brooks. “We’ll give them an experience for a year or two, and then send them back to industry.”

DigitalGlobe announced in early 2017 it will build a next generation constellation of earth observation satellites called WorldView Legion. Image credit: DigitalGlobe.

The Commercial Orbit

“Geoint is very much at a turning point,” explained Maxar’s Scott. “Historically, geoint has been an internal government exercise. The balance of capability in the geoint space has gone pretty hard over to commercial at this point. You have a commercial imagery provider like DigitalGlobe (which Scott founded and is now part of Maxar) that’s now providing 90 percent of the foundation geoint for the US government.”

DigitalGlobe is the NGA’s mainstay high-resolution imagery and geospatial solutions provider with a USD3.5-billion Enhanced View contract in 2010 that will be re-competed in 2020. The NGA also uses medium-resolution imagery from San Francisco-based Planet to monitor changes across large geographic areas for humanitarian and intelligence missions. Planet’s 190 remote-sensing “cubesats,” known as Doves, constitute the largest constellation in orbit.

“Today we can see a given spot maybe five times a day,” Maxar’s Scott noted. “When we launch our next-generation constellation, WorldView Legion [beginning in 2020], combined with Scout, a partnership we’re doing with the Kingdom of Saudi Arabia [six one-meter-resolution satellites scheduled for 2018-19], and our existing constellation, we’ll be able to see a given point on the ground several dozen times a day and small regions up to 40 times per day. That’s game-changing. That’s a level of persistence that wasn’t possible before.”

UK company Earth-i is planning a constellation of commercial satellites to provide high-frame rate images (with resolutions better than one meter for any location on Earth), as well as the ability to film moving objects such as vehicles, vessels and aircraft in ultra-high-definition color video. Imagery will be managed, catalogued and geometrically corrected using software from Swedish photogrammetry specialist, Spacemetric.

The UK government is seeking to get into the geoint imagery commercialisation game, announcing a new Geospatial Data Commission to develop a strategy for using public body mapping data to support economic growth. Officials from the Land Registry, Ordnance Survey, Valuation Office Agency and other government bodies have as a first task how to give small businesses free access to OS MasterMap data. Professor Sir Nigel Shadbolt, chair of the Open Data Institute, said, “Opening up the OS MasterMap will stimulate growth and investment in the UK economy, generate jobs and improve services.”

Even NASA, the US space agency, is planning to purchase earth science data from commercial providers, issuing a request for information in December to companies with satellite constellations, a reprise of an RFI issued in 2016 which was delayed in the Washington budgeting process. Michael Freilich, director of NASA’s Earth Science division, told an American Geophysical Union meeting, “We would like to work with you to purchase those data products in a pilot to then evaluate their worth in advancing our NASA programs. This is an attempt to develop a relationship, with money coming from us to you and data coming from you to us, to allow us to figure out whether what you are producing is useful for our science or not.”

Short-Sighted, Outdated

US government regulations restrict American companies from selling geospatial imagery better than one-quarter-meter resolution (objects of 10 inches or larger are identifiable). There are also limitations on displaying images of Israel, tied to the 1997 Kyl-Bingaman Amendment. Of course, there are no such restrictions on, say, European or Russian or Turkish or Chinese imagery agents. “It’s not a consistent environment around the world,” said Dr. Scott. “Holding US industry back thinking that will cause the rest of the world to be held back as well just doesn’t work.”

The Maxar CTO said innovation in short-wave infrared (which senses regions of the light spectrum that provide an indication of the chemical properties of a material) has been restrained by current regulations, which are 25 years old. “You’re seeing tremendous innovation in the commercial space which was not even contemplated back in 1992.”

Scott told us, “We’re only able to make about a quarter of the short-wave infrared information available that we’re actually able to collect. We’re forced to basically fuzz it out.”

DigitalGlobe has been lobbying for four years to have the restriction relaxed, and there may soon be relief. The Pentagon has developed a list of “exclusion zones,” 68 for commercial shortwave infrared and 85 for nighttime imaging, mostly overlapping, which could not be shown. “They are primarily military installations where we conduct training, where we prepare to go to war, where we are employing our force in direct preparation for a mission overseas, or a location overseas where they are currently operating,” explained US Army Lt. Col. Mark Cobos, who led the effort to whittle down the original list of about 5,000 “national security risk” areas.

The DoD, while planning to update the unclassified list of exclusion zones on an annual basis, does not intend to publicly release it. As this was written, the list had not yet been implemented.

Originally published in Issue 1, 2018 of MS&T.