Sustained vs. Peak Compute: Why Your Edge AI Model Slows to a Crawl After 30 Seconds on a BLE MCU
Your model hits its target latency. You benchmark it on the bench, it runs a clean 12 ms per inference, and you ship it. Then the field reports come in: the device is fine for about half a minute, then latency creeps up, frame rate drops, and the thing gets warm to the touch.
You didn’t change the model or the firmware. So what changed?
Nothing changed. You measured the wrong thing. The number in your benchmark was peak compute, a short burst your silicon can hit before physics catches up. What you’re shipping is sustained compute, and on a BLE MCU those two numbers can be wildly far apart. Almost no benchmark reports it.
Peak vs. Sustained Compute, Defined
- Peak compute: the throughput your MCU delivers for a few seconds, before thermal or power limits engage. Clocks are at max, the die is cold, the battery is fresh.
- Sustained compute: the steady-state throughput the silicon holds indefinitely, after thermal mass saturates and voltage regulation settles.
Datasheets and TinyML benchmark suites report peak, and they’re not lying exactly, they’re just leaving out the interesting part. MLPerf Tiny, for instance, measures inference latency and energy over short runs. That’s a fair burst spec. It says nothing about what happens at minute 3.
Inference/sec
60 |*****
50 | ****
40 | ***** <- sustained plateau
30 | **------------
20 |
+----+----+----+----+----+---> time (s)
0 30 60 90 120 150
^ peak ^ thermal/power throttle kicks inThe plateau is your real spec.
Why BLE MCUs Hit the Wall Early
Two mechanisms cause this, and on BLE parts they compound.
Thermal throttling. An nRF52 or EFR32 lives in a tiny QFN package with no heatsink and almost no thermal mass. Run a MAC-heavy convolution loop and the junction temperature climbs fast, often tens of degrees C in seconds. The datasheet gives you a max junction temp (frequently 105°C for these parts, but check yours). Once you approach it, the part scales clocks down or you risk hitting the limit. The radio and the compute core share the same die and the same thermal budget, so heat from inference eats into headroom the radio also wants. There’s a big burst of heat in a small package with nowhere to go.
Power budget throttling. A CR2032 coin cell has real internal resistance, often on the order of 10 ohms and rising as it drains. Pull a sustained 15 mA for inference and the terminal voltage sags. Your LDO has a current ceiling. Your brownout detector has a trip point. Cross it and the part resets or the DVFS controller steps the clock down to keep the rail alive. A small LiPo is better but not immune; sustained draw plus a cold cell equals voltage droop. Getting this right means mapping your device’s real current profile before you blame the model.
BLE makes it worse. The radio needs reserved current and voltage headroom to transmit cleanly, so when the budget tightens, inference gets squeezed first. The radio duty cycle is non-negotiable if you want your packets to land, which means inference is always the second-class citizen fighting for whatever’s left.
Sustained heavy inference
|
+--------+--------+
| |
Die heats up VDD rail sags
| |
Thermal throttle Brownout/DVFS
| |
+--------+--------+
|
Clocks scale down
|
Latency climbs (the cliff)The 30-second cliff. Why 30 seconds and not 5, or 90? Thermal mass. A tiny package has so little of it that the die saturates quickly, but not instantly. The first burst runs on a cold die. Heat accumulates over roughly 20 to 60 seconds depending on package, board copper, and load. When the die hits the throttle threshold, clocks scale, and your latency jumps. That timeline is why the bench looks great and the field looks broken.
Diagnosing Your Specific Cliff
Before you fix anything, find out which mechanism you’re fighting. They need different fixes.
Start by logging. Don’t take one measurement, log inference latency continuously for at least 5 minutes. One-shot numbers are how you got here.
Then correlate two signals:
- Die temperature, from the on-chip temp sensor (most of these parts have one).
- The VDD rail, measured under load with a scope or a fast ADC, not a multimeter.
The decisive test: turn the radio off and rerun. If throttling still happens, you’re thermal-bound. If it goes away, you’re power-bound, the radio TX bursts were tipping you over the rail budget.
Throttling observed?
|
+-- Persists with radio OFF? --> YES --> Thermal-bound
| -> duty cycle, quantize
+-- Only with radio ON? --> Power-bound
| -> DVFS, schedule vs TX
+-- Recovers after sleep? --> Thermal mass saturation
-> burst + cooldownIf a 30-second sleep restores full speed, the die cooled off and gave you headroom back. That single observation tells you duty cycling will help.
Mitigations, Keeping the Model On-Device
These are ordered roughly by effort. Each manages the constraint; none removes it.
Duty cycling. Run inference in bursts, sleep between them, and let thermal mass recover during the gap. If your use case tolerates it (say, you only need a classification every few seconds), this is the cheapest win. The trade-off is responsiveness: you can’t infer during the cooldown, so event latency goes up.
Clock and voltage scaling. Instead of running flat out until the part throttles itself, deliberately hold a lower, sustainable clock. You give up peak latency but you stop the thermal ping-pong where the part surges, overheats, throttles hard, recovers, and surges again. A steady 48 MHz you can hold beats a 64 MHz you lose 30 seconds in.
Quantization. Moving from FP32 to INT8 cuts the energy per MAC substantially. Published TinyML studies put INT8 MAC energy at a fraction of FP32, often quoted around 4x lower, though the exact ratio depends on your part’s multiplier and memory hierarchy (measure yours). Less energy per inference means less heat and less current draw. INT4 goes further where accuracy allows. This has the best heat-per-inference payoff, so review your quantization options if you haven’t already gone below FP32.
Operator scheduling. Don’t run your heaviest layers concurrently with a radio TX event. Spread the big convolutions across time and interleave them around the BLE connection intervals so peak compute and peak radio current never stack on the same instant. This directly helps when you’re power-bound.
Every one of these has a ceiling. You’re rationing a fixed thermal and power envelope. If your model genuinely needs sustained heavy compute the envelope can’t supply, no amount of scheduling saves you. That’s when you change the problem itself.
The Architectural Escape Hatch
Sometimes the honest answer is: don’t sustain heavy compute at all.
Instead of running the full model continuously on-device, do the minimum locally, feature extraction, event detection, a cheap first-stage classifier, and push the hard part somewhere with a real power budget.
FULL ON-DEVICE OFFLOAD PATTERN
[sensor] [sensor]
| |
[heavy model] [tiny feature extract]
| (sustained MAC) | (short burst)
[result] [BLE beacon ~13 bytes]
| |
throttles @30s gateway/cloud inference
-> no sustained computeA BLE transmit burst is cheap. A short advertising packet carrying a ~13-byte feature vector costs you a few milliamps for a few milliseconds, then you’re back to sleep. Compare that to a MAC loop hammering the die for seconds on end. The radio burst never saturates your thermal mass, so there’s no cliff to fall off.
This is one valid pattern, not a universal answer. Platforms like Hubble are built around it: the device extracts and beacons a tiny payload upstream, and a gateway or cloud tier runs the heavy inference where compute and cooling are free. You can see how the advertising packet is structured to size your feature vector against it. Whether it fits depends on your latency tolerance and connectivity, but if your on-device model keeps throttling, offloading sidesteps the whole thermal and power fight rather than managing it.
The Call, in Three Lines
Throttles with radio off? -> thermal -> duty cycle + quantize
Throttles only with radio on? -> power -> DVFS + schedule TX
Neither fits the envelope? -> offload, beacon a tiny payloadMeasure sustained throughput, not the peak number from the bench. Run the radio-off test to separate thermal from power. And when the envelope won’t hold your model, extract a small feature vector and beacon it upstream instead of grinding MACs until the die saturates.
Hubble Network lets constrained BLE devices beacon feature vectors directly to satellites, so offloaded inference works even without local connectivity. See how it works →