InnerWeb
Skip to main content

Location telemetry retention: good practices for plant management

Heartbeat telemetry has no default retention: the cleanup mechanism exists, but it stays switched off until the customer decides how long history is kept. Without that decision the table grows without limit. We describe who should make it before the first rollout and which practices help justify it.

26 s
example heartbeat interval used in the calculation
Source: karta
about 606 million
heartbeats a year for 500 devices, example calculation
Source: example calculation

Why retention is the customer's decision, not a default setting

The retention period of location history touches several areas at once: work safety, personal data protection, infrastructure cost and evidence after incidents. The system vendor cannot decide on behalf of the plant which of these interests comes first. That is why the heartbeat cleanup is prepared but remains switched off until the customer decides.

The effect of having no decision is predictable: the heartbeat table grows with every heartbeat. After two years of operation it can hold tens of millions of records, which affects query performance and storage cost.

Practice 1: name the decision owner before the first rollout

The decision should be made by a team made up of the OHS manager, the data protection officer or the person in that role, and an IT representative. The OHS manager states how long history is needed to analyse incidents. The data protection officer assesses how long location data of employees may be kept. IT presents the storage cost and the impact on performance.

The outcome of the meeting is one figure: the heartbeat retention period in days or months, recorded in the plant's documentation.

Growth of the heartbeat table without retention (500 devices)

Growth of the heartbeat table without retention (500 devices)Line chart: the heartbeat table grows linearly to about 606 million records after one year and 1212 million after two years.025050075010001250million recordsMonth 3Month 6Month 12Month 24Month 3: 150 million records150Month 6: 303 million records303Month 12: 606 million records606Month 24: 1212 million records1212
Source: Example calculation: 500 devices, heartbeat every 26 s, about 1.66 million heartbeats a day

Practice 2: separate the retention period for different data

Not all telemetry data has the same value. Raw heartbeats, which are created every few dozen seconds, grow fastest and lose analytical value fastest. The device event log and alarm records have much higher evidential value and should usually be kept longer.

We recommend that the policy contains at least two periods: a shorter one for raw heartbeats and a longer one for safety events. Example periods require justification in the risk analysis of the specific plant.

  • Raw location heartbeats: a shorter period, based on the needs of current analysis.
  • Safety events (alarm, no movement, SOS): a longer period, in line with the post-incident procedure.
  • Device log: a period sufficient to settle an incident and to pass an inspection.

Practice 3: count the data growth before the decision

It is good to base the retention decision on figures. Example calculation: assuming 500 devices and a heartbeat every 26 seconds, each device sends about 3,323 heartbeats a day (86,400 seconds divided by 26 seconds), and the whole fleet about 1.66 million heartbeats a day. Over a year this gives about 606 million records.

This result shows that without cleanup the table reaches a size at which even properly indexed queries start to cost. The numbers depend on the number of devices and the interval, so each plant should count its own.

Practice 4: switch on the cleanup deliberately and test it

Once the decision is made, the cleanup mechanism should be switched on with the chosen period. The first run on a large table may delete millions of records, so it is worth doing it during a period of lower traffic, after a backup has been made, and watching the database load. Deleting data without a backup is an operation that should not be done lightly.

After the start, check that the following cleanups run on schedule and that the table size stabilises.

Practice 5: document the decision and review it every year

The retention period is not a decision for ever. Regulations, the number of devices and auditor requirements change. Record the date of the decision, the people responsible and the justification, and plan a review once a year. This document is evidence for an audit that data storage has been considered.

Example calculation

Heartbeat growth for a device fleet

Assumptions
ParameterValue
Number of devices500 pcs
Heartbeat interval26 s

Result: About 3,323 heartbeats per device a day, about 1.66 million for the fleet, about 606 million a year.

Note: Example calculation on assumed values; the actual growth depends on the number of devices, the interval and the operating time.

Checklist

Video

Frequently asked questions

Does the system delete old heartbeats by default?

No. The cleanup mechanism exists, but it stays switched off until the customer decides on the retention period.

Who should decide on the retention period?

A team made up of the OHS manager, the person responsible for data protection and an IT representative, before the first rollout.

Do all telemetry data have to be kept for the same time?

No. Raw heartbeats grow fastest and have the shortest analytical value, while safety events are worth keeping longer.

Let's settle retention before the rollout

We will help prepare the data growth calculation and a proposal of retention periods for your team's decision.

Let's talk