Training drones:- How do you map something that is both invisible and odorless? Researchers at the University of Oslo have trained drones to find the best places to measure greenhouse gases on their own.

“Estimating such gas fluxes is not easy. We are really at the forefront of what is done in this field”, says Ph.D. candidate Alouette van Hove at UiO's Department of Geosciences.

Imagine the tundra on Svalbard. Or the vast, frozen peatlands of Siberia. For thousands of years, the permafrost has ensured that the carbon in the mires has remained undisturbed, but now it's getting warmer. Methane and CO2 gas are being released. Now the gases are rising from the ground into the atmosphere.

“Being able to map fluxes, or exchanges, of greenhouse gases on the earth's surface is necessary to quality assure and calibrate climate models”, says van Hove.

Mile after mile of peatland. Here, between the peaty ground and the air above, there is an exchange, a flux, of gases. They are important pieces in the global climate equation, but the estimates the researchers use in climate models are uncertain.

The fact that the gases dilute as soon as they come out into the air, and are carried away by wind and weather, does not make it easier. Part of the solution might be to measure the emissions close to the ground, using drones.

“What we can do is estimate the fluxes by making observations. This way, we can adjust the models with actual measurements”, says van Hove.

Intelligent measurement systems

Imagine that it is your task to measure these gas fluxes. But the area you are looking over is hundreds of square kilometers. Where are they? Where should you measure?

“The gases are invisible, and they do not smell. They can only be detected with a gas analyzer. But if you have an area of 100 x 100 square kilometers, you cannot investigate every meter”, says researcher Norbert Pirk.

He leads the research project ACTIVATE, which stands for "Actively learning experimental design in terrestrial climate science". The project aims to research and develop intelligent measurement systems for use in climate research.

Drones are used to perform the atmospheric measurements. These will be used to estimate the exchange of carbon, water, and energy between the earth's surface and the atmosphere. The measurements are combined with data from satellites, as well as mobile or stationary measuring installations.

“We are concerned with the interaction between the earth's surface and the atmosphere. Between these, there is an exchange of important greenhouse gases. This exchange is not homogeneously distributed over the globe. It typically occurs in localized "hot-spots". It is these we must find,” says Pirk.

Gone with the wind

Joining in the hunt for such hotspots, the researchers have the Drone Lab at the University of Oslo. Here, several drones are ready to go out on missions in the service of climate research. But first, they need training. Ph.D. candidate van Hove has made sure of that.

“You can't just go into an area and do a sweep with the drone. There is simply too much to measure. Besides, the weather conditions will make it so that if you measure ten minutes later, everything will look different”, says van Hove.

To get the most accurate estimate of the gas fluxes, they must measure at the most informative localities and times.

“We must optimize the time we use with the drone”, says van Hove.

She has developed a method where they use reward-driven learning - "reinforcement learning" - to train the drones to know where to look for the best places to measure.

“To train the drones, we create an artificial environment, where the drones can practice. They get a reward every time they make a movement that turns out to be useful”.

This way, the drone can learn whether turning one way instead of the other way was a good decision.

“It can be compared to training dogs. We use rewards to train the drone to choose the best action”, says van Hove.

Tries, fails and learns

In practice, this all happens inside a computer program, where the drones' "rewards" are nothing more than a specific function in the program. The drones are run in "trial and error" experiments, where the drone can move within a given area. In this area, the drones can perform a given number of actions (move forward, backward, upward, downward, etc.), but they are not allowed to move out of the area.

“So "rewards" are given for choices of actions that, after a certain time, lead to a result that is as close to the truth as possible, that is, the gas flux”, says van Hove.

Through experiments, van Hove has been able to show that such trained drones can find and measure such hotspots of CO2 emissions more precisely than if the drone performs a pre-programmed search. This is even though the drone in the pre-programmed search is set to fly over the CO2 source.

“We have shown that it is possible to train drones to estimate a parameter, without having to have prior knowledge of the parameter's true value”, says van Hove.

Now, the trained drones will be tested in practice. Soon, Pirk and van Hove will take 12 drones to Svalbard.

“Now we are going to test the drones out in the field. Then they will get to practice making decisions while they are in the air”, says Pirk.

The goal is to be able to put the drones to work at various observatories in the Arctic, where there is currently a particular lack of observational data.

“The ACTIVATE project will span over five years, and I think the measurement campaigns will become larger and more complex during the course of the project”, says Pirk, who envisions having 12 drones in operation in Svalbard in the summer of 2025. AlphaGalileo/SP