How to detect faulty sensors – when you don't know the right value
Sensors suffer from accuracy problems, and are often located in places where it is difficult to find out if they function as intended.
Sensors are often made from cheap materials with some type of redundancy, so that if they run out of battery or are damaged, there will still be others that work.
But it is of course an advantage to identify the ones that are becoming inaccurate so that they can be replaced or repaired. How can we do this without knowing the right answer, or more precisely, without knowledge of the ground truth?
One way of doing it is to compare the sensors’ readings with measurements from other instruments, but there are many places where it is difficult to carry out such checks, for example underground or in enclosed tanks.
The first thought that popped into HiOA scientist Anis Yazidi's head was that this problem can be solved!
Deviations from the majority
‘To find this out, we can focus on deviations from the majority of the sensors used when we have no accurate measurements of the true value for comparison,’ was what Anis Yazidi arrived at together with scientists John Oommen and Morten Goodwin.
‘We have also defined some conditions for deviation: If a sensor deviates from the majority more than 50% of the time over time, it is no longer reliable,’ says Anis Yazidi.
‘We can assume that a normally reliable sensor can occasionally give incorrect measurements, perhaps due to factors in the surrounding environment.’
The system that the scientists have developed take the measurements from all the sensors and compare them. The more a sensor deviates from the others, the more certain they are that the sensor in question is unreliable much of the time.
The scientists have conducted experiments involving simulations in different models.
Distinguishing between reliable and unreliable sensors
On the basis of the observations made, they have developed an algorithm that is capable of distinguishing between a reliable and an unreliable sensor.
‘Are the simulations based on practical problems?’
‘The simulations are general because they are intended to test if our model is correct. And they show that the problem can be solved quickly so that we don’t have to spend a lot of time determining which sensors are unreliable.’
Accuracy can be improved
‘We are also trying to find out whether we can filter out the unreliable ones, and how we can use their incorrect measurements to improve accuracy. If I know how they contradict the other sensors, I will also know more about how they can be replaced.’
A world first
Even though it is a simple idea, it appears that no other solutions exist that can identify unreliable sensors without knowing the underlying cause, and Anis Yazidi and his colleagues may be the first in the world to come up with a solution.
‘If an alternative existed, that would have been discovered during the peer review of our solution by such a reputable research journal as IEEE Transactions on Cybernetics.’
Continues with continuous measurements
‘How will you take this work forward?’
‘So far, we have taken as our point of departure that the measurements are binary, either zero or one, and not a continuous variable e.g. on a scale from 0 to 1, or a temperature scale from 0 to 100.’
This is another possibility to explore how the reliability of a sensor can be calculated in the absence of knowledge of the true value.
So far, the scientists have only measured if e.g. temperature readings are high or low, registered as ‘on’ or ‘off’.
‘We are now starting to look at continuous measurements. When the values go from one to a hundred, how can I filter out an unreliable sensor?’
One possibility is to use an aggregate of the comparisons, because reliable sensors will mostly be relatively homogenous.
Sorting reliable versus unreliable into groups that will contradict each other, and then looking at statistical correlations between measurements, can give a fair indication.
Yazidi, Oommen and Goodwin have presented their most recent research results at a conference in Singapore in November: AMDA 2017, The 13th International Conference on Advanced Data Mining and Applications.
References to scholarly articles:
A. Yazidi, B. J. Oommen and M. Goodwin, ‘On Solving the Problem of Identifying Unreliable Sensors Without a Knowledge of the Ground Truth: The Case of Stochastic Environments,’ in IEEE Transactions on Cybernetics, vol. 47, no. 7, pp. 1604–1617, July 2017.
Article published at AMDA 2017 in Singapore:
Anis Yazidi, B. John Oommen and Morten Goodwin: ‘Identifying Unreliable Sensors Without a Knowledge of the Ground Truth in Deceptive Environments’ .
A sensor is an instrument that detects a stimulus, e.g. heat or cold, and converts the registration into a signal that can be read. Examples include sensors that detect temperature, light (photocells), distance, force and movement.