BTHome: An open standard for broadcasting sensor data

Many wireless sensors broadcast their data using Bluetooth Low Energy (BLE). Their data is easy to receive, but decoding it can be a challenge. Each manufacturer uses its own format, often tied to its own mobile apps. Integrating all of these sensors into a home-automation system requires a lot of custom decoders, which are generally developed by reverse-engineering the protocols. The goal of the BTHome project is to change this: it offers a standardized format for sensors to broadcast their measurements using BLE. BTHome is supported by the Home Assistant home-automation software and by a few open-firmware and open-hardware projects.

The chances are high that the manufacturer of a BLE device requires the use of a smartphone app to remotely view its data. But, technically, there's no need to use the app. The device advertises its name and some data; anyone with a BLE receiver in the neighborhood is able to pick up those BLE advertisements. What those apps do is to convert the raw data to information such as a temperature or humidity value using a protocol decoder for the proprietary data format.

Reverse-engineering the protocol used by these BLE sensors is often not that difficult: just pick up some BLE advertisements, look at the decoded information in the app or on the display if the device has one, and try to figure out the mapping between the data and the decoded information. Most of these protocols aren't encrypted or obfuscated in any way. Once the protocol has been figured out, a decoder can be developed and the app isn't needed anymore.

However, for all of those sensors to be supported in home-automation systems, these systems have to add decoders for all of those proprietary formats. For example, Home Assistant has decoders for BLE devices made by Xiaomi, Govee, Mopeka, ThermoPro, Inkbird, and more. Every time one of these manufacturers launches a new device or changes their protocol, the decoder has to be adapted, not only in Home Assistant, but also in many other home-automation projects.

One decoder to parse them all

One can't hope to change the way that commercial BLE device manufacturers transmit their data, but a few open-source do-it-yourself projects for BLE sensors have started to use their own formats too. So Ernst Klamer, who had been working on Bluetooth support in Home Assistant for a long time, created his own standardized data format for devices to send sensor data over BLE. This was added to Home Assistant 2022.9 and launched as a new open standard for sending sensor data: BTHome.

Last year, Raphael Baron's soil-moisture sensor b-parasite and pvvx's ATC_MiThermometer firmware (for various BLE temperature sensors made by Xiaomi) adopted the BTHome format. So these two devices don't need a custom protocol decoder anymore in Home Assistant, just the BTHome integration. Any other devices that implement BLE advertisements using the BTHome format are also automatically recognized by Home Assistant.

It's important to know BTHome's scope. It only defines a format for BLE advertisements, which means it is for devices that simply broadcast their data. Advertisements are the fundamental way that BLE devices communicate; they are also used in the discovery of these devices. Another BLE communication mechanism uses a connection: one device connects to the other one and accesses the latter's services to read and write data. BTHome does not use these BLE connections.

As a result, BTHome is for one-to-many unidirectional communication: a device broadcasts its sensor data and many other devices can pick up that data. There's no way to return data to a BTHome sensor or to control the device by sending commands. So to light an LED or turn on a switch, something other than BTHome is needed.

Flexible format

An advertising packet in BLE is a sequence of one or more advertising data structures. Each of these starts with a length byte, which specifies the number of bytes following, a byte describing the type of data structure, and then the data itself. For example, there's a "Flags" type indicating the capabilities of the device; there are also "Shortened Local Name" or "Complete Local Name" types to advertise the device's name.

Another type of advertising data structure is the "Service Data with 16-bit UUID", which starts with a 16-bit UUID designating the type of data or the company advertising the data; there are other types of Service Data with UUIDs of different lengths. These UUIDs are assigned by the Bluetooth Special Interest Group that oversees the Bluetooth standards. After the UUID, there is data that follows a format defined by either the BLE standard or the company, depending on the UUID.

A BTHome device broadcasts its sensor measurements as part of the Service Data. Allterco Robotics, the manufacturer of Shelly devices for home automation, has sponsored the UUID 0xFCD2 for BTHome. It's free to use for everyone with a license that restricts the use of the UUID to implementations of the BTHome protocol.

The data format after the UUID bytes is described in detail on BTHome's web site. It supports dozens of sensor types (including temperature, humidity, speed, and voltage), binary sensor types (which can be true or false, such as door open or closed, or power on or off), and events (button presses, double-presses, long-presses, and others).

The payload always starts with an object ID designating the sensor type and then the data. One packet of BTHome Service Data can include multiple measurements of the same or different types. Because each object ID corresponds to data with a known, fixed length (1 to 4 bytes), a parser just has to read the object ID, the known number of bytes for the data, then the next object ID, the corresponding data, and so on, until the end of the Service Data. An example is shown in the diagram below.

The specification has some extra requirements that enable compatibility with future versions. Object IDs need to occur in numerical order (from low to high) in a BTHome advertisement. If a new version of BTHome adds a new sensor type, its object ID will be a higher number than the ones supported in earlier versions. And if a BTHome parser encounters an unknown object ID, it has to stop parsing the advertisement. Thanks to the ordering of the object IDs, the supported sensor data is still parsed, because it comes before the unknown object ID.

Projects using BTHome

Apart from the already mentioned b-parasite and ATC_MiThermometer projects, there are not many devices that currently implement the BTHome standard. But there are some that can be used as an example for other implementations in various programming languages.

A simple but interesting example is the BTHome Electricity Meter for the Puck.js device. It is a Bluetooth beacon with lots of sensors and I/O pins that is running the Espruino firmware, which allows the microcontroller to be programmed using JavaScript. The BTHome Electricity Meter code reads total energy usage (in kWh) and live power consumption (in W) from an electricity meter with an impulse light, using a light-dependent resistor connected to one of the Puck.js's I/O pins. The JavaScript code is easy to follow as an example of how to encode the data in BTHome advertisements. For other JavaScript projects, Ron Šmeral's bthome.js is an interesting library that could be used to build on.

BTHome Weather Station is an ESP-IDF project for the ESP32, a popular microcontroller with WiFi and Bluetooth built in. The project's code reads the temperature, humidity, and pressure from a BME280 sensor connected over I²C to the microcontroller board, and advertises the sensor measurements over BLE using BTHome. The same code can be used for a lot of other types of sensors. The bthome-bme680 project has borrowed this code to advertise measurements from the BME680 gas sensor, also on an ESP32 board. And Ole Sæther, working for Nordic Semiconductor, has created an implementation using the Zephyr real-time operating system that works on his employer's nRF52840 BLE SoC: bthome_nrf_template.

However, when researching BTHome implementations, make sure that they're not using the BTHome legacy (v1) format, as this should not be used for new projects. For example, the ESP32_BTHome project advertising a temperature and counter with the NimBLE-Arduino library on an ESP32 microcontroller board is still using the legacy format, as is the bthome_nrf52840 project for Nordic Semiconductor's nRF52840 SoC.

While the previous examples are for devices that advertise BTHome payloads, for parsing there are even fewer options. The canonical parser implementation is Home Assistant's bthome-ble Python library. This can be used to decode BLE advertisements in other Python programs and to decode BTHome payloads. Ardelean Călin Petru has done some initial work on a Rust implementation: bthome-rs. As an exercise in understanding the BTHome format, I created a BTHome format description in Kaitai Struct. Because Kaitai Struct is also a parser generator, this description can be turned into a BTHome parser for the 11 programming languages it supports.

Encrypted advertisements

By default anyone in the neighborhood of a BTHome device can pick up its sensor measurements and decode them. For more security-conscious applications, the BTHome format also supports encrypted advertisements. This works using a pre-shared key that is set up in the device and then also configured in the receiver (such as in Home Assistant's BTHome integration). The sensor data in the advertisements cannot be decrypted without the key. Encryption is done using the Advanced Encryption Standard (AES) algorithm in CCM mode.

None of the example projects seem to implement encryption, so unfortunately there's no clear example to build on. The BTHome Weather Station project mentions encryption under Future Work, but explains that the documentation is pretty scarce and that there's only one example Python script, which is contained in the bthome-ble repository. It encodes some test data with temperature and humidity values, encrypts the payload, shows the encryption algorithm's parameters, decrypts the result, and decodes the data, showing the same temperature and humidity values.

Integrating BTHome sensors in Home Assistant

As a test for how easy it would be to implement the BTHome format, I created a BTHome project in CircuitPython that advertises movement detected by the inertial sensor built into the Seeed Studio XIAO nRF52840 Sense board. The BTHome format specification was quite clear and, with CircuitPython's low-level BLE module _bleio, it was easy to translate the sensor data to the corresponding advertising payload.

After adding the BTHome integration, my Home Assistant installation immediately recognized the Seeed Studio XIAO nRF52840 Sense board. When the board gets moved, its status is shown in Home Assistant's dashboard. So BTHome delivers on its promise of out-of-the-box support for do-it-yourself BLE sensors using the format for their advertisements.

Building an ecosystem around an open standard

The BTHome format is still quite young and, so far at least, there is no large ecosystem around it. However, with two popular device projects adopting it, there's hope that it will get some visibility. Instead of everyone inventing their own custom data format, having a flexible format specification such as BTHome for sensor measurements can help a lot of projects.

For device makers, it means out-of-the-box support in Home Assistant and potentially other home-automation controllers. The BTHome project is also quite responsive to suggestions for adding new sensor types. For developers of home-automation controllers or BLE apps, the format means support for a lot of different device types. Seeing a random BTHome sensor automatically recognized in Home Assistant provides an example for other home-automation projects to aspire to.