How to Deploy a TinyML Anomaly Detection Model on a Cortex-M4

Most TinyML projects die somewhere between “the model works in a Jupyter notebook” and “the model works on the board.” The training gets all the attention. The deployment gets all the bugs. You’ll spend 20% of your time picking and training an anomaly detection model, then 80% fighting quantization drift, tensor arena sizing, and mysterious HardFaults during inference. This guide covers that 80%.

We’re walking through the full deployment pipeline for running a TinyML anomaly detection model on a Nordic nRF54 series chip (Cortex-M4F), using Zephyr RTOS and TensorFlow Lite for Microcontrollers (TFLM). We’re skipping model training entirely. You either have a pre-trained model or you can grab one from the open-source options below. The goal: go from a .tflite file on your laptop to real-time inference on-device in a single working session.

What You’ll Need Before Starting

Hardware: nRF54L15-DK. If you don’t have one, an nRF5340-DK works as a fallback (target the application core, the Cortex-M33, which shares most of the TFLM integration steps).

Software:

• nRF Connect SDK v2.7+
• Zephyr RTOS (bundled with nRF Connect SDK)
• ARM GNU Toolchain (arm-none-eabi-gcc)
• Python 3.9+ with TensorFlow 2.x installed (for quantization)

Model: A pre-trained autoencoder in .tflite or TensorFlow SavedModel format. Good starting points:

• Edge Impulse’s autoencoder examples (exportable as .tflite)
• A simple Keras dense autoencoder trained on the MIMII dataset (machine sound anomaly detection)
• NASA bearing dataset autoencoders from various TinyML community repos

Here’s the pipeline we’re following:

┌─────────────┐    ┌─────────────┐    ┌──────────────┐    ┌────────────┐    ┌───────────┐
│  Pre-trained │───▶│  Quantize   │───▶│ Convert to   │───▶│ Integrate  │───▶│ Validate  │
│  Model       │    │  (INT8)     │    │ C Array      │    │ in Zephyr  │    │ On-Device │
│  (.tflite)   │    │  (TFLite)   │    │ (xxd/python) │    │ (CMake)    │    │ (RTT/UART)│
└─────────────┘    └─────────────┘    └──────────────┘    └────────────┘    └───────────┘

Model Selection and Constraints for Cortex-M4

Picking the right model architecture isn’t about accuracy alone. A 98%-accurate autoencoder that blows your flash budget is useless regardless of its reconstruction quality. You need to think in hardware budgets.

nRF54L15 resource budget:

nRF54L15 Flash (1.5 MB)
├──████████████████░░░░░░░░░░░░░░░░░░░░  Zephyr + BLE Stack (~400 KB)
├──████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  TFLM Runtime (~80 KB)
├──██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  Model (target: <50 KB)
├──██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  Application Code (~40 KB)
└──░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  Free (~930 KB)

• Flash: ~200 KB max for the model, but aim for under 50 KB
• RAM: 256 KB total; your tensor arena (the scratch memory TFLM uses during inference) should stay under 30-60 KB
• Clock: 128 MHz Cortex-M4F; target under 10 ms inference for real-time vibration monitoring at typical sample rates

Architecture tradeoffs:

An autoencoder is a neural network trained to reconstruct its input. Anomalies produce high reconstruction error, which makes autoencoders the safest bet for TFLM deployment. Isolation forests lack native TFLM support, though the emlearn library can help. Mahalanobis distance is dead simple and tiny but struggles with complex signal patterns.

My recommendation: start with a dense autoencoder. 3-4 layers, bottleneck of 8-16 neurons, input window of 128-256 samples. It’ll fit comfortably and give you a working baseline before you try anything fancier.

Quantization to INT8 for Edge Inference

The Cortex-M4F has a hardware FPU, but it’s single-precision and scalar, so it won’t help much with the matrix multiplications that dominate neural network inference. INT8 quantization gets you two things. First, smaller model size. Second, access to CMSIS-NN’s SIMD instructions, which operate on 4 packed 8-bit values in a single cycle.

Here’s the Python snippet for post-training quantization:

import tensorflow as tf
import numpy as np

# Load your trained model
model = tf.keras.models.load_model('anomaly_ae.h5')

# Representative dataset for calibration (use normal operating data)
def representative_dataset():
    for i in range(200):
        sample = calibration_data[i:i+1].astype(np.float32)
        yield [sample]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()
with open('anomaly_ae_int8.tflite', 'wb') as f:
    f.write(tflite_model)

Two things will bite you here. First, your reconstruction error threshold from float model evaluation is now wrong. Quantization shifts the output distribution. You must recalibrate the threshold against the quantized model. Run your test set through the quantized .tflite in Python and pick a new threshold.

Second, very small models (under ~1K parameters) can lose meaningful precision during quantization. Always compare quantized accuracy against the float baseline. If accuracy drops more than 2%, consider quantization-aware training (QAT), where the model learns to tolerate quantization noise during training. But try post-training quantization first; it works fine for most autoencoders.

Converting to C and Integrating with Zephyr

Turn the quantized .tflite into a C header:

xxd -i anomaly_ae_int8.tflite > model_data.h

This gives you a const unsigned char array you can compile directly into your firmware.

Project structure:

project/
├── CMakeLists.txt
├── prj.conf
├── src/
│   ├── main.c
│   ├── model_data.h        ← quantized model as C array
│   ├── inference.c          ← TFLM interpreter setup
│   └── inference.h
├── tflite-micro/            ← TFLM as Zephyr module
└── boards/
    └── nrf54l15dk.overlay   ← devicetree overlay for sensor/ADC

The critical C code has 5 parts. Here’s the skeleton for inference.c:

#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "model_data.h"

// 1. Tensor arena — static allocation, sized for your model
static uint8_t tensor_arena[20 * 1024];  // start at 20 KB, adjust

// 2. Op resolver — register ONLY what your model needs
static tflite::MicroMutableOpResolver<5> resolver;
resolver.AddFullyConnected();
resolver.AddRelu();
resolver.AddReshape();
// Add others as needed. Fewer ops = less flash.

// 3. Build interpreter
const tflite::Model* model = tflite::GetModel(anomaly_ae_int8_tflite);
tflite::MicroInterpreter interpreter(model, resolver, tensor_arena,
                                      sizeof(tensor_arena));
interpreter.AllocateTensors();

// 4. Fill input tensor from sensor buffer
TfLiteTensor* input = interpreter.input(0);
for (int i = 0; i < INPUT_SIZE; i++) {
    input->data.int8[i] = quantize(sensor_buffer[i],
                                    input->params.scale,
                                    input->params.zero_point);
}

// 5. Run inference and compute reconstruction error
interpreter.Invoke();
TfLiteTensor* output = interpreter.output(0);
float mse = compute_reconstruction_error(input, output);
bool is_anomaly = (mse > THRESHOLD);

Use MicroMutableOpResolver with only the ops your model actually uses, not AllOpsResolver. The difference can be 40-60 KB of flash.

The single most important line in your prj.conf:

CONFIG_CMSIS_NN=y
CONFIG_TENSORFLOW_LITE_MICRO_CMSIS_NN_ACCELERATION=y

Enabling CMSIS-NN kernels is your biggest performance lever. On Cortex-M4, this typically yields 2-4x speedup over the reference TFLM kernels. No additional code changes required, just flip the config.

If you’re building a custom device integration with Hubble’s BLE-based connectivity, the device integration guide covers the firmware-side setup for getting your sensor data off the board and into the cloud, which pairs well with local anomaly detection as a filter.

Build, Flash, and Validate On-Device

Build and flash:

west build -b nrf54l15dk/nrf54l15/cpuapp
west flash

On-device validation is where most people cut corners, and it costs them later. You need to verify the on-device model produces the same outputs as your desktop Python evaluation:

1. Preload test vectors. Embed 5-10 known-normal and 5-10 known-anomalous input windows as const arrays in your firmware.
2. Run inference at boot. Loop through each test vector, run Invoke(), print the reconstruction error over RTT or UART.
3. Compare against Python. Run the same test vectors through the quantized .tflite in Python. Results should match within INT8 rounding tolerance (typically ±1 on the int8 output scale).

Common failures and how to fix them:

• “Arena too small” at AllocateTensors(): bump your tensor_arena size by 20% increments until it works, then add 10% margin.
• “Op not found”: you forgot to register an op in your MicroMutableOpResolver. Check your model’s ops with flatc or Netron.
• Inference output is constant regardless of input: your input scaling is wrong. The int8 zero-point and scale must match the quantization parameters baked into the model. Pull them from input->params.
• HardFault on Invoke(): almost always a stack overflow. Increase your Zephyr main thread stack size in prj.conf (CONFIG_MAIN_STACK_SIZE=8192 or higher).

Profiling and Squeezing Out More Performance

Wrap your Invoke() call with Zephyr’s cycle counter:

uint32_t start = k_cycle_get_32();
interpreter.Invoke();
uint32_t end = k_cycle_get_32();
uint32_t us = k_cyc_to_us_floor32(end - start);
printk("Inference: %u us\n", us);

Check flash and RAM with:

arm-none-eabi-size build/zephyr/zephyr.elf

Optimization levers, ranked by impact:

1. CMSIS-NN (already enabled, right? This is 2-4x alone)
2. Shrink input window (256 → 128 samples cuts compute roughly in half)
3. Prune model layers and re-quantize
4. Minimal op resolver (vs. AllOpsResolver; saves flash, not inference time)

On power: if you’re running on battery, duty-cycle your inference. Running anomaly detection every 500 ms instead of continuously can drop average current from milliamps to microamps. For wireless sensor nodes that need to transmit anomaly alerts, you can pair on-device detection with Hubble’s BLE advertising packet format to get alerts off the device without maintaining a persistent connection.

Taking This to Production

A working demo isn’t a product. Here’s what bridges the gap:

Threshold calibration. Run the model on several hours of known-normal operating data on the actual deployed hardware. Set your anomaly threshold at 3σ above the mean reconstruction error. This accounts for real-world sensor noise that your training data might not have captured.

OTA model updates. You will retrain and requantize models as you collect more field data. MCUboot plus the nRF Connect SDK’s DFU support lets you push updated model binaries over BLE without touching the device.

Anomaly logging. Store timestamps and reconstruction error scores in a flash ring buffer. Transmit periodically via BLE or LoRa. This gives you the data to refine your model and validate that your threshold is working.

Alternative path. If manual TFLM integration feels like too much plumbing for your timeline, Edge Impulse’s EON Compiler can generate optimized C++ inference code from your model. You trade some flexibility for deployment speed, especially useful for prototyping.

The Hubble Device SDK provides firmware-level BLE integration for Nordic chips, which can handle the connectivity layer while your application code focuses on inference and anomaly logic.

Where Deployment Engineering Breaks Down

The model is the easy part. Quantization recalibration, tensor arena debugging, input scaling bugs: that’s where TinyML projects stall or ship broken. This pipeline is deliberately boring: select, quantize, convert, integrate, validate. Each step is mechanical. The discipline is doing each one carefully, verifying before moving to the next.

You’ve got a Cortex-M4 running at 128 MHz with enough flash and RAM for a useful anomaly detector, and CMSIS-NN kernels that make inference fast enough for real-time vibration monitoring. The toolchain works. Go build something.

Hubble Network connects your edge devices from anywhere—no gateways, no line of sight—so your anomaly detector can report the moment it matters. See how it works →