Fetch Device Data from Hubble Cloud with ESP32-C6

The Hubble Network provides a global infrastructure for IoT devices to send data to the cloud using standard Bluetooth LE. If you followed our previous tutorial on setting up a Hubble beacon, you will know how to turn an ESP32-C6 DevKitC into a Hubble Network-compatible device. However, that was just the first step. The true power of the Hubble Network comes from gathering and processing this data.

To enable this, Hubble provides a Cloud API that lets any user automatically retrieve IoT data from their device(s) onto their PC. From there, they can visualize their data, identify trends, and make data-backed decisions to optimize their workloads.

In this guide, you’ll learn how to use the Hubble Cloud API to automatically fetch data from your ESP32-C6 Hubble beacon.

Getting Ready

Before building the retrieval pipeline, ensure you have a working Hubble beacon by following our previous tutorial. In this guide, you will use the same beacon sample firmware for your ESP32-C6 board.

You will use Hubble’s REST API to fetch data from Hubble Cloud. To do so, you need two pieces of information: your Organization ID and an API token.

If you completed our previous tutorial to create a working Hubble beacon, you should already have these two pieces of information.

If you don’t have an API token, follow the instructions here to create one.

The script below also requires a device ID. In our previous tutorial you will have received your device ID during the device registration process.

Automate Packet Retrieval

Fetching packets manually through the dashboard is useful for initial testing, but production applications require automation. The following Python script uses the requests library to poll the Cloud API for the packets from the last 5 minutes from a specific device. It follows the GET request and response formats outlined in the Hubble API documentation.

Note: Before you run this script, make sure you replace the Org ID, device ID, and API key placeholders with your real values.

import requests
import time
from datetime import datetime, timezone

# Configuration
ORG_ID = "INSERT_YOUR_ORG_ID_HERE"
DEVICE_ID = "INSERT_YOUR_DEVICE_ID_HERE"
API_KEY = "INSERT_YOUR_API_KEY_HERE"

URL = f"https://api.hubble.com/api/org/{ORG_ID}/packets"

def fetch_packets():
    # Only pull packets from the last 5 minutes
    start_ts = int(time.time()) - 300
    
    # Define the parameters of your API request
    params = {
        "device_id": DEVICE_ID,
        "start": start_ts
    }
    
    # Define the headers of your API request
    headers = {"Authorization": f"Bearer {API_KEY}"}

    try:
        # Send the API request
        response = requests.get(URL, headers=headers, params=params)
        if response.status_code == 200:
            packets = response.json().get("packets", [])[:10]
            
            # Set up headers for the terminal output
            print(f"{'TIME (UTC)':<10} | {'RSSI':<5} | {'PAYLOAD'}")
            print("-" * 41)

            # Parse the packet data from the API response
            for p in packets:
                dev = p.get("device", {})
                ts = dev.get("timestamp", "N/A")
                rssi = dev.get("rssi", "N/A")
                data = dev.get("payload", "")

                # Reformat the timestamp into a human-readable format hours:minutes:seconds
                timestr = datetime.fromtimestamp(ts, tz=timezone.utc).strftime('%H:%M:%S')

                # Print the packet data
                print(f"{timestr:<10} | {str(rssi):<5} | {data}")
    except Exception as e:
        print(f"Error: {e}")

if __name__ == "__main__":
    fetch_packets()

Verify End to End Data Flow

With the script running and your ESP32-C6 broadcasting, you can observe the entire data lifecycle. Your device constructs its packet, encrypts it using the SDK, and transmits it via Bluetooth LE. A nearby terrestrial gateway, like a smartphone running the Hubble Connect app, picks up the advertisement and forwards it to the Hubble Cloud.

The Hubble backend identifies the device, decrypts the 10-byte protocol header, and stores the entire contents of every packet. Your script then pulls this packet data, and outputs it to your PC’s terminal.

You should notice the 3 fields the script pulls from the packet:

• Time (UTC)
• RSSI
• Payload

These fields are part of every Hubble packet. The time indicates when your device constructed the packet, the RSSI indicates the strength of your device’s signal when it reaches a gateway (like a smartphone), and the payload is the custom data that your device can populate with just about anything, from an algorithm to connected sensor data.

Don’t worry if your payload is empty for your device. This example just shows a filler payload to demonstrate what it would look like. In a future tutorial, you will learn how to set a custom payload on your ESP32-C6 board and send that data to Hubble Cloud.

Troubleshooting Problems

If your script outputs incorrect or no data, be sure to check the following.

Credentials

Assuming you use the example script as-is, your Org ID, API key, and Device ID all need to be correct.

Org ID: You can find your Org ID on your Hubble Dashboard, under Developer Tools > API tokens in the left sidebar.

API Key: Unfortunately, you won’t be able to access the API key after you initially create it. If you are not confident you copied down the right key initially, create a new API key and save it somewhere safe.

Device ID: If you have a working Hubble beacon, you can find your device ID on the Hubble Dashboard. Navigate to your target device, expand the contents of one of the packets, and look for the “id” field.

Once you’ve checked all your credentials, you can use this tool in the Hubble API documentation to test them out. On the right sidebar, you can input your credentials and verify the Hubble Cloud correctly validates your request and gives you the expected response. This way you can isolate the problem to your script’s internal logic, instead of fighting with both that and the Cloud API at the same time.

System Time

The ESP32-C6 does not have built-in time management. In other words, you need to manually modify the firmware to set the system time, which is what the device uses to set its timestamp when constructing its Hubble packets.

There are many ways to correctly set the system time on your device. However, the easiest way is to just manually set it during the Hubble beacon firmware build process. That way, once you flash your firmware, the system time continues to update as long as your device remains powered on.

To set the system time manually, all you need to do is to provision your beacon. During that process, your PC automatically assigns the current time to your ESP32-C6 firmware. Once you build and flash that firmware, your device will boot with that system time. Your device’s time will now be largely accurate, though technically it will be behind the real time by a minute or two (or the duration of time from provision to boot).

Since the example script only retrieves packets from the last 5 minutes, you will need to provision your beacon firmware within 5 minutes of booting your device with this firmware. If that timeframe is too short, please update the line start_ts = int(time.time()) - 300 in the script to match the duration of your local firmware build process.

Scale the Implementation

Hubble’s Terrestrial Network detects advertisements from over 100 million gateways. This allows your ESP32-C6 to move through urban environments while still delivering data to your cloud script without any manual intervention. For high-volume applications, you can use the Cloud API to fetch packets for all devices in your organization simultaneously by omitting the device ID from the URL path. This returns a batch of packets that you can filter by device name or tag within your backend logic, giving you the power to track and monitor your devices all over the world.

Ready to connect your devices anywhere on Earth? Get started with Hubble Network for free →