InnerWeb
Skip to main content

Detecting duplicate transmitters from route data: best practices for analysts

A duplicate transmitter gives itself away in the data before anyone goes into the field. It is enough to compare neighbouring route points, set bounce thresholds and filter out vehicles that stopped and set off from another place. We describe the method used by the data analyst who maintains the location system.

over 1 million
route points in the 14-day scan
Source: karta
151 of 651
bounces disappeared after one silencing
Source: karta
approx. 5 s
gap between points of real duplicates
Source: karta

Principle: the true position follows from neighbouring points

The true position of a device is established from neighbouring route points. A point that jumps away from the previous one and to which the next point returns is an artefact. The previous point shows the actual place of the vehicle.

This principle has a practical consequence for analysis. You do not need to know the physical location of a transmitter to notice that something is wrong with it. A triplet of consecutive points is enough: previous, suspicious and next. The pattern 'bounce and return' is characteristic of a duplicated identifier.

Bounce detection rule

The rule used has two thresholds. A point is treated as a bounce when it is more than 55 units away from the previous one, and the next point returns close to the previous one, meaning closer than 21 units. The two thresholds work together: the first catches a large jump, the second confirms that it was a return and not a change of place.

The scan was run on over one million route points from 14 days. A sample this large distinguishes constant errors from one-off disturbances and gives a ranking of transmitters that is repeatable from day to day.

  • For each device, sort its route points in time.
  • Compute the distance between consecutive points.
  • Mark a point as a bounce: distance from the previous one above 55 units, and distance of the next one from the previous one below 21 units.
  • Assign the bounce to the transmitter that generated the suspicious point.
  • Build a ranking of transmitters by number of bounces.

Thresholds of the bounce detection rule (units)

Thresholds of the bounce detection rule (units)Bar chart of two thresholds: 55 units of distance and 21 units of return.0204060unitsDistance from previous (above): 55 units55 unitsDistance from previous (above)Return near previous (below): 21 units21 unitsReturn near previous (below)
Source: Internal record: bounce detection rule

Three sources of false alarms

The number of bounces alone is not enough for a decision. Before silencing any transmitter, three of the most common sources of error must be ruled out.

Return points of one culprit

Many apparently guilty transmitters are merely the return points of one real culprit. In one scan, 151 of 651 bounces disappeared after silencing a single transmitter. Had the analyst silenced the whole top of the ranking, many working devices would have been switched off.

That is why we work iteratively: silence the leader of the ranking, recalculate the scan, and only then assess the rest.

A vehicle that stopped and set off from another place

A vehicle that stopped and then set off from a different place is not evidence of a duplicate. The time gap between the points decides. Real duplicates had a gap on the order of 5 seconds, too short to cover 400 to 650 m. If the gap is an hour, the observation may simply be an ordinary trip.

A position change caused by signal interference

A single signal disturbance produces a single bounce, not a series. A duplicate produces bounces systematically, on every trip past the same place. That is why repeatability over time matters, not a one-off hit.

What to do after a duplicate is detected

The duplicate is silenced by changing its major number to a value that no device transmits. Position, minor and location stay, so reversing it means changing one number. Note, however, that both transmitters of the pair drop out of positioning until a site visit.

With a physical duplicate, changing the stored position alone does not remove the error, it only moves it to the other place. For that reason a position correction must never be treated as a remedy for a duplicate.

After the change, synchronise the positioning cache without clearing it entirely and check in the data whether the silenced transmitter records no new points and the others do.

Control rhythm

The duplicate scan should be run periodically, not only when a report arrives. After every installation of a new batch of transmitters, run the scan on the data from the first days, and then every quarter. This way numbering errors surface while the transmitters are still new and it is easy to establish who installed them and when.

Save the scan results together with the date and the thresholds. When someone changes the thresholds, comparison with the previous scan will still be possible.

Checklist

Video

Frequently asked questions

How do I know it is a duplicate and not a trip?

Check the time gap between the points. Real duplicates had a gap of about 5 seconds, too little to cover 400 to 650 m. A trip or a stop gives a much longer gap.

Why not silence all the transmitters on the ranking?

Many of them are only return points of one culprit. In one scan, 151 of 651 bounces disappeared after silencing a single transmitter.

Is silencing reversible?

Yes. Only the major number changes, so reversing it means changing one number. Until the site visit, neither transmitter of the pair takes part in positioning.

Want to run a duplicate scan on your own data?

We will present the method using routes from your plant. Book a conversation with the InnerWeb team.

Let's talk