As both drone technology and AI continue to evolve, aerial data analytics company AirFusion looks to build on its success in the wind sector, writes Tildy Bayar

Three and a half years ago, US-headquartered AirFusion launched its first product, a drone-based data analytics tool for wind turbine operators. Building on the success of this product, the firm is currently developing a new inspection system for high-voltage power transmission lines.

This development is focused not on the drone itself, but on the data analysis software behind it. Chief strategy officer Kevin Wells explains why: “Drone technology is a highly competitive market where we’d have difficulty differentiating ourselves, while with our software we’re much more able to differentiate.

“On the drone side there’s plenty of great equipment out there that keeps getting better,” he says. “There’s lots of specialization in terms of navigational packages, sense-and-avoid, and now of course the advent of autonomous flight, which is very exciting and a terrific driver for our business.

“Similarly, on the sensor side, it’s a very specialized group of companies. Typically what’s happening is that larger sensors are becoming miniaturized for easy deployment on drones.”

So rather than attempting to compete in the data collection space, AirFusion chose to focus on data processing – with the goal of replacing not just human inspection of power machinery, but also human analysis of the data gathered via drone.

“What we saw when we formed the company,” Wells says, “was the sheer amount of data that was coming down the pipe from this amazingly efficient method of collection from drones carrying all manner of sensors. But all of that data was still being analyzed by humans, really by hand, frame by frame.”

Edward Mier, managing director for Europe, says the artificial intelligence (AI) element of AirFusion’s software is crucial to the company’s goal of “making the analyst’s work 95 per cent more efficient”. And he says it’s also what differentiates the company from the competition.

“In most of the instances [of drone-based inspection firms] we’ve come across, the AI element is missing. Particularly if you talk about wind, I don’t think we’ve really found anybody who’s actually got an active AI capability.”

While AirFusion’s AI engine is “essentially a generic engine that is available for any vertical market,” he says, the key is that it can cross-correlate the data gathered by multiple drone-mounted sensors with a database of potential wind turbine damage cues.

“Our differentiator is really the patent that we have, holding a multiple-sensor process and the ability to merge, overlap those particular images and then have the machine detect, with great accuracy, the fault that we’re seeing.”

For the power industry, Mier believes thermal sensors “are going to be absolutely crucial” because multiple sensors offer the possibility of cross-correlation.

“If you see a particular fault with the thermal camera which generally indicates that the power line is overheating – an insulator breakdown, for example – the additional optical sensor which would be mounted on our drone (or however the data is being collected, but primarily and most likely by drone) would give us a cross-correlation that would say, not only do we believe that hotspot is something we need to be looking for, but we also would be cross-correlating that with actual sensory visual data to see whether we can see a crack in the insulator or some other defect.

“You may end up using up to four sensors. You could imagine that maybe a LIDaR sensor would be additionally useful, and indeed we believe that, in some cases, a sonic sensor may be useful as well. Typically, in a high-powered transmission situation when the conductor is breaking down, you’ll hear a noise from the short-circuiting, and that third or fourth sensor would give us that further correlation.”

Assistive AI

“Our convolutional neural net system for AI is really the solution to an image recognition problem,” Wells says. “In image recognition, the way AI systems typically work is about your image training set. Every AI system is about as smart as a bag of hammers until you train it – it doesn’t know what to look for or what’s important. We use subject matter experts from around world in whatever vertical market we’re working in – in this case it was wind.

“The questions are: what is the damage set we’re looking for, what does it look like, and what are the different instantiations of how it looks? So we trained the database with hundreds of thousands of images that show these remarkable varieties of common damage you get on a wind turbine blade: lightning strikes, delamination, leading edge erosion, etc.

“Once those are coded properly into the system, it can become self-learning if you set it up the right way. It takes a great deal of time to train the engine, though, and you have to put in a bunch of heuristics around the images because there are things that only a human expert could know go together. Ultimately, you hope to get to a system that’s very accurate.”

According to Wells, there are “many different flavours of AI”. AirFusion refers to theirs as ‘assistive AI’ rather than ‘autonomous AI’.

“Autonomous AI seeks to take the human being out of the equation altogether,” he explains. “Circuit board inspection is an example. Because the circuit board itself is very precise and uniform, you can control all the variables and every photo will be identical. If there are any anomalies, they will be instantly recognizable by the machine.

“Our technology is more of an assisted version of AI. What we’re trying to do is reduce the bulk of time a human would need to spend doing damage discovery and classification.

“First, identify that there is damage, and then say what kind of damage it is. It’s really a two-step process of discovery and assessment.

“At the end of all that, what we do is eliminate the images that are clear of damage, because they’ll still take valuable time for the human to go through. With the remainder, perhaps ten to 12 images, we’ll say ‘Ok, we think this is a lightning strike, and we think it’s severity 1-5’, and then we present that to the user. But we’ll always present it to a human because there’s so much subjectivity involved.

“In a meeting with one of the world’s leading wind OEMs, the person who is responsible for blade assessment across the enterprise – that’s tens of thousands of wind turbines – said that if you take a single image of a piece of damage and put it in front of five human analysts, you might get five different answers as to what it actually is. So because of that great subjectivity, what we tend to do is tee up our machine’s best guess as to what it is – and sometimes we put in guess #2 if there’s a question mark for us, and let them make the final decision. But that takes them a fraction of the time it would take them to do the whole process themselves without machine assistance.”

Autonomous flight

Where the drone sector’s move toward autonomous flight really benefits AirFusion is in improving the consistency of the data that’s gathered. “The point is to try to establish a good baseline and then have consistent follow-up results,” says Wells. “Autonomy helps with that.

“Image variability is the thing you try to avoid if you can. The new drones coming out in the last few years can make adjustments to their path in microseconds based on factors like wind gusts etc. If a drone is flying autonomously, it can correct its path much faster than a human could. And if drones are always flying a certain distance away from the infrastructure, you’re taking out so much of the variability of the images.

“Reducing the number of variables in the data coming in is very helpful, and moves from that assistive place where we are today as an industry. The more autonomous you can get with your AI results the better. But that will be a development that happens over a decade.”

The sky’s the limit

In the nearer future, AirFusion is poised to both continue to expand its offering for the wind sector and move into the T&D space, focused primarily on the European market. The firm is seeing growing interest from the wind industry because, according to Mier, “operators are looking for every single dollar saved to maximize revenue. Because we can make the inspection process more efficient, but also come up with consistent results which improve the efficiency of the turbines, they are really seeking us out.”

Adds Wells, “A tonne of world-class brands have approached us and want to work with us, but it’s wide open at this point – the sky is really the limit.”

Tildy Bayar is features editor of PEi magazine