Factory floor data collection architecture with MQTT and time series

Collecting factory floor data is simple until the first network incident. This is the reference architecture we use when the requirement is continuity, not a demo.

Four-layer topology

PLC / sensors
   │  OPC UA, Modbus TCP, serial
   ▼
Edge gateway (local buffer)
   │  MQTT / Sparkplug B, TLS
   ▼
MQTT broker (cluster)
   │  ingestion
   ▼
Time series database  →  API  →  Dashboards and alerts

The edge layer is what prevents data loss. It keeps an on-disk buffer and retransmits when the link comes back — without it, every link drop becomes a permanent gap in the history.

Topic modeling

Improvised topics age badly. Adopt a stable hierarchy from day one:

plant/line/cell/equipment/metric
e.g.: itu/line-03/cell-a/press-12/temperature

• Use only lowercase, no accents, no spaces.
• Never put units or data type in the topic — that goes in the payload.
• Reserve a level for contract version when the payload may evolve.

If the fleet is heterogeneous, Sparkplug B handles device discovery, session birth/death, and metric typing, at the cost of a binary payload and less trivial debugging tools.

Quality of service and ordering

QoS 1 with deduplication by idempotency key usually costs less than QoS 2 in throughput.

Writing to the time series database

A few precautions that avoid rewriting the ingestion pipeline six months later:

1. 1.Write in batches. Group by a 1-to-5-second window; single-row inserts tank performance.
2. 2.Cardinality is the enemy. Don't use batch ID or production order ID as a tag; store it as a field.
3. 3.Set retention by resolution. Raw for 30 days, 1-minute aggregate for 1 year, 1-hour aggregate indefinitely.
4. 4.Timestamp at the edge. The ingestion server's clock does not represent the moment of measurement.

Observability of the collection itself

Monitor the pipeline as if it were a piece of equipment: latency between source timestamp and write, buffer retransmission rate, count of devices in Sparkplug death state, and cardinality growth. A silent pipeline is not a healthy pipeline — it's a pipeline with no instrumentation.

Conclusion

The hard part of industrial telemetry isn't publishing the first metric: it's keeping five years of coherent history as the plant changes. Edge buffering, a stable topic hierarchy, cardinality control, and an explicit retention policy solve most of this before it turns into debt.