ESP32 Manufacturing Provisioning: Flashing Credentials and Configs at Scale
You’ve got firmware running on your desk. Wi-Fi connects, TLS handshake works, OTA pulls down updates. Everything’s great on that one dev board you’ve been nursing for 3 months. Then procurement tells you 5,000 modules are arriving from the CM next Thursday, and someone asks: “How do we get unique certificates, serial numbers, and Wi-Fi configs onto each one?”
That question tends to land about 4 weeks before the first production run. And the answer can’t be “plug in a USB cable and type idf.py flash really fast.”
This guide covers building a provisioning pipeline that takes a master database of per-device credentials and configs, generates flashable binaries, burns them onto ESP32 modules in a scripted loop, and logs every result. We’ll cover partition strategy, nvs_partition_gen.py, scripted flashing with esptool.py, secure boot/flash encryption, and validation, all ESP-IDF toolchain, all serial flashing, with notes on scaling to gang programmers.
What Goes Where: Partition Strategy for Manufacturing
The golden rule: one firmware binary, many NVS binaries. Your compiled application (bootloader, app, partition table) stays identical across every unit. Per-device data (Wi-Fi creds, device certificate, serial number, calibration values) lives in a separate NVS partition that gets generated and flashed individually.
Here’s the flash map for a typical 4 MB module:
Flash Map (4 MB)
+---------------------+ 0x000000
| Bootloader | (shared across all units)
+---------------------+ 0x008000
| Partition Table | (shared)
+---------------------+ 0x00D000
| NVS (per-device) | ← creds, serial, config
+---------------------+ 0x027000
| NVS Keys | (if NVS encryption enabled)
+---------------------+ 0x028000
| OTA Data | (shared)
+---------------------+ 0x030000
| App (OTA_0) | (shared)
+---------------------+ 0x1F0000
| App (OTA_1) | (shared)
+---------------------+ 0x3B0000
| Custom fctry data | (optional, per-device)
+---------------------+ 0x400000The shared blobs (bootloader, partition table, app) get built once from your CI pipeline and treated as release artifacts. The NVS binary is the only thing that changes per unit.
When should you use a custom fctry data partition instead of NVS? If you’re storing large binary blobs (firmware signing keys, big calibration tables) that exceed what NVS handles comfortably, a raw data partition with your own read logic makes sense. For most provisioning data (strings, small certs, numeric configs), NVS is the right call. It gives you key-value access in firmware with zero custom parsing code.
Generating Per-Device NVS Binaries
nvs_partition_gen.py ships with every ESP-IDF install, sitting under components/nvs_flash/nvs_partition_generator/. It takes a CSV file describing key-value pairs and spits out a flashable .bin file.
The CSV schema uses 4 columns: key, type, encoding, value. Namespace rows group keys logically (matching how your firmware reads them with nvs_open).
Here’s a minimal example for a device with Wi-Fi credentials, a serial number, and a TLS client certificate:
key,type,encoding,value
wifi_ns,namespace,,
ssid,data,string,MyNetwork
pass,data,string,hunter2
device_ns,namespace,,
serial,data,string,UNIT-00042
cert,file,binary,/certs/device_00042.pemGenerate a single NVS binary:
python $IDF_PATH/components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py generate \
device_00042.csv device_00042.bin 0x6000 --version 2That 0x6000 matches your NVS partition size (24 KB in this example). Get this wrong and you’ll get silent failures or corrupted reads. Always pull the size from your partition table CSV; don’t guess.
For batch generation, loop over a master database. A simple Python script works:
import csv, subprocess, os
TEMPLATE = "nvs_template.csv" # has placeholders
IDF_NVS_TOOL = os.path.join(os.environ["IDF_PATH"],
"components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py")
with open("device_manifest.csv") as f:
reader = csv.DictReader(f)
for row in reader:
device_csv = f"build/nvs_{row['serial']}.csv"
device_bin = f"build/nvs_{row['serial']}.bin"
# Write per-device CSV from template + row data
with open(device_csv, "w") as out:
out.write("key,type,encoding,value\n")
out.write("wifi_ns,namespace,,\n")
out.write(f"ssid,data,string,{row['ssid']}\n")
out.write(f"pass,data,string,{row['password']}\n")
out.write("device_ns,namespace,,\n")
out.write(f"serial,data,string,{row['serial']}\n")
out.write(f"cert,file,binary,{row['cert_path']}\n")
subprocess.run([
"python", IDF_NVS_TOOL, "generate",
device_csv, device_bin, "0x6000", "--version", "2"
], check=True)Run this once before production day. You’ll end up with a directory of .bin files, one per unit, ready to flash.
Scripting the Flash Pipeline
Here’s the conceptual flow for each unit on the line:
[Master DB / CSV]
|
v
[Pre-generated NVS .bin] ── per-device
|
v
[esptool.py write_flash]
- bootloader.bin @ 0x0
- partition.bin @ 0x8000
- nvs_XXXX.bin @ 0xD000 ← per-device
- app.bin @ 0x30000 ← shared
|
v
[Verify / read-back]
|
v
[Log result to DB]The flash script wraps esptool.py with per-device NVS selection, serial port detection, and logging:
import subprocess, sys, time, json
SHARED_ARGS = [
"0x0", "build/bootloader.bin",
"0x8000", "build/partition-table.bin",
"0x30000", "build/app.bin",
]
def flash_device(port, serial_number):
nvs_bin = f"build/nvs_{serial_number}.bin"
flash_args = [
"esptool.py", "--port", port, "--baud", "921600",
"write_flash",
"0xD000", nvs_bin,
] + SHARED_ARGS
result = subprocess.run(flash_args, capture_output=True, text=True)
log_entry = {
"serial": serial_number,
"port": port,
"timestamp": time.time(),
"success": result.returncode == 0,
"output": result.stdout[-500:] if result.stdout else "",
}
if result.returncode != 0:
log_entry["error"] = result.stderr[-500:]
return log_entryA few practical notes on the flash station:
Baud rate matters. At 921600 baud, a 4 MB flash takes roughly 35 to 45 seconds. At 460800, double that. For a 5,000-unit run, the difference is ~24 hours of line time. Use the fastest stable rate your jig supports.
Serial port detection. If your test jig uses a fixed USB-serial adapter, hardcode the port. If operators swap boards on a hub, use esptool.py with --port auto or scan /dev/ttyUSB* before each cycle. Assign ports by USB topology (hub port number) for multi-port setups.
Reset mode. Most test jigs pull GPIO0 low and toggle EN via RTS/DTR. If your jig uses a different reset mechanism, pass --before no_reset and handle it in your fixture controller.
The Secure Provisioning Layer
This is where the stakes go up. Efuse operations are permanent and irreversible. Burn the wrong key or set the wrong flag, and that module is scrap. Test your entire secure provisioning flow on at least 10 sacrificial units before the production run.
Secure Boot v2
Secure Boot v2 ensures only firmware signed with your key can run on the device:
- Generate a signing key (do this once, store it offline, back it up):
espsecure.py generate_signing_key --version 2 secure_boot_signing_key.pemBuild with secure boot enabled in
menuconfig(Security Features → Enable Secure Boot v2).The build system signs the bootloader and app automatically.
On first boot (or via
espefuse.py), the public key digest gets burned to efuse. This cannot be undone. After this, unsigned firmware won’t boot.
Flash Encryption
Flash encryption protects your firmware IP at rest. There are 2 modes:
Development mode lets you re-flash over serial (the FLASH_CRYPT_CNT efuse has remaining bits). Good for internal testing. Release mode permanently disables serial re-flash of encrypted partitions. Use release mode only when you’re confident in your OTA pipeline.
The provisioning sequence matters: flash plaintext firmware first, then enable flash encryption on first boot. You can also encrypt offline with espsecure.py encrypt_flash_data and flash the ciphertext directly. That’s faster on the line but requires careful key management.
NVS Encryption
NVS encryption protects your credentials (Wi-Fi passwords, TLS keys) at rest. It uses a separate NVS key partition. Generate the key:
python $IDF_PATH/components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py \
generate-key --keyfile nvs_keys.bin --outdir build/Flash nvs_keys.bin to the NVS key partition offset. Your NVS data partition must then be generated with the --inputkey flag pointing to the same key file.
Per-Device TLS Certificates
For mutual TLS auth to your cloud backend, each device needs a unique client certificate and private key. The typical approach: generate key pairs and CSRs on your provisioning host, sign them with your CA, and inject the cert + key into NVS via the CSV. You can generate keys on-device and export CSRs, but it’s slow and awkward at production scale.
If you’d rather not run your own PKI, hardware secure elements like the Microchip ATECC608 can hold device keys in tamper-resistant storage. You’ll pay roughly $0.50 to $1.00 per unit in BOM cost, and you’ll need to write additional I2C driver code.
Here’s the decision matrix:
Security Feature | When to Use | Efuse Impact
-----------------------|---------------------------|-------------------
Secure Boot v2 | Prevent unauthorized FW | Irreversible
Flash Encryption | Protect FW IP at rest | Irreversible (Release)
NVS Encryption | Protect credentials | Requires key partition
Per-device TLS cert | Mutual auth to cloud | None (stored in NVS)If your devices connect to a cloud platform via BLE rather than Wi-Fi, some of these concerns shift. Devices provisioned through the Hubble device SDK for Espressif follow a similar per-device credential injection pattern but with BLE advertising payloads instead of Wi-Fi configs. The partition strategy and NVS generation workflow still apply.
Validation, Logging, and Handling Failures
Flashing without verification is gambling. Build these checks into every cycle:
Read-back verification. After flashing, run esptool.py verify_flash against each partition. This catches corrupted writes, bad connections, and flaky jig pins, adding ~10 seconds per unit.
Functional smoke test. Flash a lightweight test firmware (or add a “factory test” mode to your production firmware) that boots, reads NVS values, attempts a Wi-Fi connection or TLS handshake, and reports PASS/FAIL over the serial console. Parse the output in your script.
Logging everything. Every unit’s MAC address, serial number, flash result, test result, and timestamp goes into a database or structured CSV. Eight months from now, when a customer returns a unit, you need to trace it back to the exact provisioning run.
Failure handling. Set a retry count (2 retries is reasonable) before quarantining a unit. If your failure rate exceeds 2 to 3%, stop the line. Check the jig, USB cables, and module batch. A spike in failures usually points to a mechanical problem, not a firmware bug.
Scaling Beyond a Single USB Cable
For runs over a few hundred units, a single serial port becomes a bottleneck.
Multi-port USB hubs with 4 to 8 ports let you run parallel esptool.py instances, each mapped to a jig slot. A Python script with threading or asyncio can flash all 8 slots simultaneously, cutting line time by 8x. Use a powered hub; ESP32 modules draw 300 to 500 mA during flash.
Dedicated gang programmers from Espressif partners (or SEGGER J-Link-based setups with JTAG) handle higher throughput. These are common in contract manufacturer environments.
Third-party provisioning platforms like Espressif’s ESP RainMaker provisioning, AWS IoT ExpressLink, or Kudelski IoT keySTREAM manage the PKI and credential lifecycle for you. These make sense if you’re shipping millions of units, working with multiple CMs, or need regulatory compliance around key management. For modest volumes where you own the cloud backend and run your own CA, DIY works fine.
For devices that report data to a cloud backend, you’ll want to register devices and configure webhooks on the platform side as part of the same provisioning pipeline. Tying device registration to your flash script keeps the cloud inventory in sync with what’s actually coming off the line.
Pre-Production Checklist
Before the first unit hits the jig, confirm these artifacts and processes are locked down:
- [ ] Partition table finalized and tested with OTA
- [ ] NVS CSV schema defined; sample generated and verified on dev board
- [ ] Master credential DB populated for the full batch
- [ ] Flash script tested end-to-end on 10+ units (not just 1)
- [ ] Secure boot signing key generated and stored offline (with backup)
- [ ] Flash encryption mode decided (Development vs. Release)
- [ ] NVS encryption key generated (if applicable)
- [ ] Functional test firmware or test mode validated
- [ ] Logging and MES integration confirmed (writes to DB, not just stdout)
- [ ] Failure handling and retry logic tested (unplug a board mid-flash, see what happens)
- [ ] Jig pin alignment and USB cable quality verified across all slotsThe path from “works on my desk” to “works on 5,000 units” is mostly about separating what’s shared from what’s unique, scripting the boring parts, and logging everything. The tooling already exists in ESP-IDF. Your job is to bolt it together into a pipeline that an operator can run without thinking about partition offsets.
Hubble Network connects your ESP32 devices directly to satellites, eliminating terrestrial infrastructure entirely. See how it works →