Law enforcement agencies often deploy airborne unit aerial assets when ground officers are in hot pursuit of a suspect. The situation may be tough to monitor from the ground, but from the sky, tactical flight officers (TFO) have the entire scene in their view.
TFOs can easily track the real-time speed of the vehicle in pursuit and give commands to ground officers, such as upcoming street addresses, suspect movements, or imminent obstacles or dangers. One of the tools at their disposal is the latest in mission mapping solutions.
“When I first had the opportunity to use real-time augmented reality mapping, it was like having a tactical flight officer cheat code,” said Jon Gray, a retired law enforcement helicopter pilot from Southern California. “It made what was once complex, simple and easy to use. Many criminals have been caught due to the quality of data that is available.”
Across the airborne parapublic and military sectors, advances in mission mapping solutions have been a game-changer in the industry, helping operators improve mission outcome.
When it comes to enhancing situational awareness, more and more operators are turning to augmented reality mapping systems — first developed to solve a common issue in special ops aviation.
Previously, an operator’s mission mapping solution might consist of a camera displayed on one monitor and a map on another. It was a complicated system that often went underutilized.
“In the time it took for [operators] to look at the video feed to the monitor, and then do the mental calculus to figure out which area corresponds to which area, when they got back to the video feed, the suspect might not be on screen anymore,” said Tom Churchill, CEO of Shotover Systems.
As a solution, Churchill and others like him sought inspiration from none other than the NFL.
“Why can I see the first down line on a football game, but I don’t have that awareness of what a street name is?” said Luke Savoie, president of the intelligence, surveillance and reconnaissance sector at L3Harris. “Why don’t I have that in what is now a world filled with video from aircraft?”
Headquartered in Melbourne, Florida, L3Harris has a long history in creating mapping solutions and other situational awareness systems for both fixed- and rotary-wing platforms.
Thanks for the NFL sport network’s high-powered long-focal length lenses, along with a virtual first down line, fans have grown used to watching football with information superimposed directly on top of the video.
“If you could do the same with an aircraft, that would solve all these problems,” Churchill said. “You’d be able to have all the geospatial information that you need directly on top of the video.”
Shotover develops high-performance gyrostabilized camera systems and real-time augmented reality mapping and mission management software. In fact, Churchill said its M2 EO/IR multi-sensor camera system is the first of its kind specifically designed for augmented reality.
The company is one of a number in this space offering mission mapping solutions for special mission operators. No matter what shape, form or manufacturer they come from, these mapping solutions set out to improve situational awareness and pilot navigation, enabling flight crews to get the information they need to make better decisions.
Building blocks of augmented reality mapping
In essence, a mission mapping system integrates moving maps with EO/IR camera systems. Using augmented reality capability, vector data such as street names, businesses, parks, or hiking trails are overlaid on top of a live video feed.
The mapping system can also work with dynamic information, pulling in data from sources like the android tactical assault kit (ATAK), automatic identification systems (AIS), automatic dependent surveillance-broadcast (ADS-B), or Link 16 to display real-time geospatial data showing the location of ground crews, maritime ships, aircraft, or military assets.
It’s this type of data “which is often the most interesting,” Churchill said. Not only can vector data be overlaid on video, but it’s also what industry experts call actionable, meaning the system can use the data to perform some sort of action automatically.
For example, if a law enforcement agency operates over multiple jurisdictions, they may need to communicate with different agencies at different times.
“Imagine a state highway patrol that’s flying over one county, and then over a different county,” Churchill said. “We can make it so that as they fly from one county to another, the system can automatically bring up a user interface dialog that says, ‘I see you’re now in this new county, would you like me to program the radios to the correct frequencies?’”
The same idea can be applied to search-and-rescue (SAR) operations where the system can automatically steer the camera to follow a specific search path.
“Another piece that is very valuable is being able to overlay planning data on top of that mapping environment,” said Scott Richardson, vice president of product at CarteNav. “As an example, knowing where my coverage was for a search pattern.”
In its simplest function, Richardson said mission mapping solutions essentially enhance two key components: providing operators with an accurate understanding of where EO/IR cameras are looking, and giving them the ability to command the camera to behave a certain way.
“That really is the primary mission operators’ goal — to get the camera where it needs to be and understand where it’s looking accurately,” Richardson said.
But that’s not where its capabilities end.
Mapping systems with sensor integration
Any type of systems and geospatial data that can be generated and collected could, in theory, be integrated into a mission mapping solution, Richardson said. This includes searchlights, downlink systems, radars, radio systems, direction finders, GPS systems, and more. It also includes data from commercial partners like TomTom or open sources like OpenStreetMap.
A newcomer to the mission mapping scene, Trakka Systems is known for developing EO/IR cameras and high-intensity searchlights. The mapping system came into the picture later as a way to allow mission operators to integrate all its Trakka products together for a total mission solution from one supplier.
The team is currently working on integrating with other camera systems outside the Trakka product line to increase its market share.
“We’ve also been integrating with downlink companies to be able to steer their antennas and tune frequencies,” said Mike Kipphorn, Trakka’s director of sales for the west coast. “We are also working with a radio company that’s interested in integrating with TrakkaMaps as well.”
Trakka calls the maps the “hub” of the system, touching all other components, “so you can control any component from the mapping system,” Kipphorn said.
The system can also work with customized data collected by mission operators. The Riverside County Sheriff’s Department, for example, was the first known operator to collect and map out school information, such as classroom locations and numbers, for every school in its patrol area.
“[Now], if they ever have to respond to [a school shooting], they’ll be able to direct the officers on the ground to the proper location [in the school],” Churchill said.
Since then, both the Calgary Police Service and Prince George’s County Police Department in Maryland have added school data overlays to their mapping system. The South Carolina Law Enforcement Division (SLED) and New Jersey State Police are also in the process of doing the same.
“We have a team of people that source data from all over the world, and we include that in every system that we deliver,” Churchill said. “We collect data on a global basis, and we deliver it on a continental basis so somebody in a regional sheriff’s office will get data for the entire United States. That way, if there ever is an event that requires their rapid deployment, they have everything they need.”
Mission enhancer for special operations
In special operations, helicopters allow operators to gain a better perspective of the scene than those who are on the ground. But the perspective from the sky can also make it challenging for aerial operators to identify what they’re looking at. That’s where augmented reality really becomes valuable.
“From a law enforcement aspect, it’s hard to look out the window and simultaneously see what’s happening on the screen,” Kipphorn said. “So it’s important to have a mapping system that can give you the big picture.”
Mission mapping systems give operators more authority in the sky and the “ability to rapidly provide definitive directions to the folks on the ground,” Churchill said. “It’s always the person on the ground that we’re doing this for. The operator in the air, he’s not arresting anyone, he’s there for the benefit of the folks that are on the ground.”
The goal of a mission mapping system is to allow operators to focus on their task without being pulled away to another monitor, Richardson said. To do this, the CarteNav system can pull up tactical panels or overlays — what CarteNav calls TacVision — to show operators their location and other data without having to look at another monitor.
“It’s about bringing in information and allowing the operator the ability to control what they see and don’t see in their augmented reality,” Richardson said.
In essence, the system provides improved situational awareness for pilots and reduction in workload for the entire unit. For Savoie, improved situational awareness translates to safety in flight, which translates to reduced crew fatigue and reduced risk.
Savoie said this technology can be used in any application that “involves flying close to the earth.” This includes responding to disasters in adverse weather conditions or environments where communication infrastructure is down and situational awareness is low.
“We utilized the moving map and video system during Hurricane Katrina, when the Department of Defense assets came into New Orleans to provide support, manage the airspace, and coordinate the recovery,” Savoie said. “In the emergency implementation, it’s all about speed and the ability to respond quickly and more accurately. Minutes and seconds matter in finding someone who is hurt and then getting them to a hospital.”
What’s on the horizon?
The future scope of this evolving technology is being driven largely by customers’ needs, experts said.
“We are very driven by operator feedback,” Richardson said. “As cool as some of the technologies I talked about are, we’re not going to develop the technology without operator input to understand how to leverage those tools.”
Along with these “evolutionary improvements” from customer feedback, as Churchill called it, the technology will also evolve through revolutionary improvements, or internal innovations.
“It’s our job to be aware of what’s technically or theoretically possible in a way that we can’t expect customers to necessarily know,” Churchill said. “That’s what I believe our role is — to constantly be aware of what new technologies are available, and to think about whether they have the potential to be useful for our customers.”
Along with working on improved diagnostics, Churchill said Shotover is now focused on
taking the information that’s in a mission mapping system and making it available in the operators’ eyepiece.
“If you don’t have to look at a monitor but the information is directly available as you move around the window, that’s a useful technology for certain classes of users,” he said.
The CarteNav team also has a couple of projects in its pipeline.
“One of the things that’s going to be coming out with our next round of this technology is an ability to directly interact with sensors to command them specifically,” Richardson said, referring to AI learning algorithms. This includes integrating object detection into the mapping system, such as identifying a red truck and flagging that back to the operator.
Daniel Allen, chief software manager at Trakka, believes the future of mission mapping solutions will see more connectivity between the air crew and ground crew.
“One of the things that we’ve started working on is having stations on the ground and in the air that communicate back and forth,” Allen said. “You can message the users in the air and exchange data with them.”
Savoie believes the technology itself will become more of a primary flight instrument in a helicopter cockpit rather than a tool simply to help with situational awareness.
“We’ve started to see uses of our technology where it is becoming the primary instrument, not the secondary, not the backup, and not just something that’s situational awareness only,” he said. “Everyone uses it as primary now.”