It’s Sea Air Space week, baby, and the news is already coming in hot and fast.
This morning, maritime perception company Tocaro Blue (you’ll remember them from their team-up with Havoc AI) unveiled ApolloCore, a new camera-based machine learning software, in an exclusive release to Tectonic.
The stack—designed to work with the company’s ProteusCore radar processing software—basically takes off-the-shelf point-tilt-zoom (PTZ) marine cameras and turns them into “high performance perception systems” for things like USVs.
“ApolloCore…basically turbo-charges affordable commercial-off-the-shelf cameras into military-grade tools used for autonomous vessels,” Tocaro Blue CEO John Minor told Tectonic.
Baby blue: Tocaro Blue is a startup safely living on the nuts-and-bolts side of things.
The company was founded back in 2019 in Florida with a pretty nifty mission: To take commercial off-the-shelf tech and turn it into the “perception layer that powers the future of intelligent vessels” using AI.
- Their first, flagship product, ProteusCore, uses AI and machine learning to analyze radar sensor data and autonomously detect and track other vessels and maritime threats. Basically, it takes cheaper, COTS radars and turns them into ultra-powerful perception tools.
- Even without connectivity, the software can share this threat data across an autonomous fleet.
- That’s what Havoc AI—and other customers, including Magnet Defense/Metal Shark—are using right now.
Push it to the limit: But the thing is, radar is only one part of the perception layer when you’re way, way out at sea.
- Radar has a bit of a physics problem—because of the curvature of the Earth (it ain’t flat, folks), radar systems can only “see” out to the radar horizon.
- That means that low-flying or surface-level targets or threats (think small USVs) can slip below that line of sight.
- Waves, spray, and foam can also create “sea clutter” that makes smaller threats hard to detect—that gets worse the tougher a sea state is.
Look deeper: Cameras—specifically PTZ cameras—can bring new depth to that perception layer, especially in terms of ID-ing and classifying threats. And the good news is, a lot of vessels (even small USVs) already have these fun, rotating cameras on board.
“We really started realizing that builders and integrators were not getting much value from the PTZ camera that’s installed on a lot of those unmanned surface vessels…basically just using it like a security camera,” Minor said. “Some had some object detection, but nothing more than just simple bounding boxes, and there were no ranging or tracking capabilities.”
What ApolloCore does is take those cameras (which can cost as little as a few thousand bucks) and pump up their capability with AI. Here’s how it works:
- First, Minor said, the company’s software stabilizes cameras and images they produce. “That’s really important for remote operation, and also target detection and ranging,” he said.
- Then, the software classifies what it’s seeing. The team has trained a model on a database of about 3.5M vessel images to create 15 different marine object classes (boats, buoys, markers, etc.). “Apollo core can operate completely passively to track and range targets,” Minor added.
- The whole thing also works alongside Proteus—so you’ve got simultaneous radar and onboard camera and radar processing. “You’re combining the power of the radar for ranging, targeting, and detecting or tracking things with the camera, which is really good at classification and understanding what’s being seen in the field of view,” he said.
- Tocaro Blue Chief of Research and Development Austin Gurley told Tectonic that the Apollo-Proteus perception cocktail takes the range of vision for a surface vessel out to “many miles or kilometers into the distance.”
“When you combine that radar with the camera, it’s a comprehensive perception suite for an unmanned surface vessel,” Minor added.
What you lookin’ at: Gurley said that ApolloCore’s threat classification ability is the real game changer. Not only do you know what’s coming towards you, but you also have an idea of how you could respond.
“[The software can say] this is a power boat. It’s a sailboat. It’s a ship,” he said. “But the value of [ApolloCore] is in identifying the dynamics of that target. You can say, ‘If this is a small power boat, I know it can accelerate quickly, and it’s kind of agile and nimble, whereas the ship has a lot of momentum and it behaves differently.’”
This kind of information is especially relevant for autonomous vessels. “Perception is very difficult,” Tocaro Blue Senior Sales Director Andrew Rains told Tectonic. “Once perception is solved, the autonomy piece is a much simpler problem.”
Plug it in: Minor described ApolloCore as “middleware…[that] sits in between the sensor and whatever the system is that needs to use that data.”
That means that most of the company’s customers are companies building autonomous vessels—both those building autonomy stacks and those building the vessels themselves (like Havoc and MetalShark).
“We are giving an autonomy developer the right front-end perception capability so that they can then go build the obstacle avoidance, the track and follow capability, the mission set execution from [that] information,” Rains said. “We’re not designing the mission. We’re just designing the platform for them to get the best sensor data possible to feed into their system.”
And at least by defense standards, all that data and perception goodness comes fairly cheap:
- According to the company, ProteusCore costs about $10-20K per license, depending on the type of radar it’s connected to.
- ApolloCore costs $15-50K a license, depending on the type of camera it’s connected to (from lower-cost COTS cameras to higher-end defense-specific models).