ESP-Brookesia is a human-machine interaction development framework designed for AIoT devices. It aims to simplify the processes of application development and AI capability integration. Built on ESP-IDF, it provides developers with full-stack support — from hardware abstraction and system services to AI agents — through a component-based architecture, accelerating the development and time-to-market of HMI and AI application products.
Note
"Brookesia" is a genus of chameleons known for their ability to camouflage and adapt to their surroundings, which closely aligns with the goals of ESP-Brookesia. This framework aims to provide a flexible and scalable solution that can adapt to various hardware devices and application requirements, much like the Brookesia chameleon with its high degree of adaptability and flexibility.
The key features of ESP-Brookesia include:
- Native ESP-IDF Support: Developed in C/C++, deeply integrated with the ESP-IDF development ecosystem and ESP Registry component registry, fully leveraging the Espressif open-source component ecosystem;
- Extensible Hardware Abstraction: Defines unified hardware interfaces (audio, display, touch, storage, etc.) and provides a board-level adaptation layer for quick porting to different hardware platforms;
- Rich System Services: Provides out-of-the-box system-level services including Wi-Fi connectivity, audio/video processing, using a Manager + Helper architecture for decoupling and extensibility, providing support for Agent CLI;
- Multi-LLM Backend Integration: Built-in adapters for mainstream AI platforms including OpenAI, Coze, Xiaozhi, providing unified agent management and lifecycle control;
- MCP Protocol Support: Exposes device service capabilities to large language models via Function Calling / MCP protocol, enabling unified communication between LLMs and system services;
- AI Expression Capability: Supports visual AI expression through emoji sets, animation sets, and more, providing rich visual feedback for anthropomorphic interactions.
The functional framework of ESP-Brookesia is shown below, consisting of three layers from bottom to top: Environment & Dependencies, Services & Framework, and Application Layer:
- Utils: Provides common foundational capabilities for upper-layer modules.
General Utilsincludes the logging system, error checking, state machine, task scheduler, plugin manager, and memory/thread/time profilers.MCP Utilsacts as the bridge between ESP-Brookesia services and the MCP engine, exposing registered service functions as standard MCP tools so large language models can call device capabilities. - HAL: Defines unified hardware access interfaces and provides board-level adaptation.
Interfacedefines standardized hardware APIs for audio playback/recording, display panels and touch, status LEDs, and storage file systems.Adaptorprovides implementations for specific development boards and completes hardware resource initialization and mapping.Boardsprovides board-level YAML configuration that describes the peripheral topology, pin assignments, and driver parameters of each board. - General Service: Provides system-level foundational services, including
Wi-Ficonnection management,Audiocapture and playback,Videocodec processing,NVSnon-volatile storage,SNTPnetwork time synchronization, and aCustomservice extension mechanism. All services use the Manager + Helper architecture and support both local calls and RPC-based remote communication. - AI Agent Framework: Provides a unified management framework for AI agents, with built-in adapters for mainstream AI platforms such as
Coze,OpenAI, andXiaoZhi. Through theFunction Calling / MCPprotocol, it enables bidirectional communication between large language models and system services, allowing LLMs to perceive and invoke device capabilities. - AI Expression: Provides visual expression capabilities for AI interaction scenarios, including
Emotesets and animation control, delivering rich visual feedback for anthropomorphic interaction. - System (planned): Provides GUI, system management, and application framework support for different product forms such as mobile devices, speakers, and robots.
- Runtime (planned): Provides runtime support for WebAssembly, Python, Lua, and more, enabling dynamic application loading and execution.
- 中文：https://docs.espressif.com/projects/esp-brookesia/zh_CN
- English: https://docs.espressif.com/projects/esp-brookesia/en
- ESP-Brookesia Versioning
- Development Environment Setup
- Hardware Preparation
- How to Obtain and Use Components
- How to Use Example Projects
| Component | Description | Version |
|---|---|---|
| brookesia_lib_utils | Core utility library providing task scheduler, thread configuration, performance profilers, logging system, state machine, plugin system, and other foundational capabilities |  |
| brookesia_mcp_utils | MCP bridge library that registers Brookesia service functions or custom callbacks as MCP tools for invocation by large language models |  |
| Component | Description | Version |
|---|---|---|
| brookesia_hal_adaptor | HAL board-level adaptation component providing audio and LCD device interface implementations for specific development boards |  |
| brookesia_hal_boards | HAL board configuration component providing board-specific device configurations, peripheral mappings, and hardware setup hooks for esp_board_manager |  |
| brookesia_hal_interface | HAL foundation component that defines a unified device/interface model, lifecycle management, and hardware interface abstractions (audio, display, touch, storage, etc.) |  |
| Component | Description | Version |
|---|---|---|
| brookesia_service_audio | Audio service providing audio playback, encoding/decoding, volume control, and playback state management |  |
| brookesia_service_custom | Custom service extension template for developing custom services based on the Brookesia service framework |  |
| brookesia_service_helper | Service helper library providing type-safe definitions, schemas, and unified service calling interfaces (based on CRTP pattern) |  |
| brookesia_service_manager | Service manager providing plugin-based service lifecycle management, dual communication modes (local calls & TCP RPC), and event pub/sub mechanism |  |
| brookesia_service_nvs | NVS (Non-Volatile Storage) service providing namespace-based key-value storage management |  |
| brookesia_service_sntp | SNTP network time service providing NTP server management, timezone settings, and automatic time synchronization |  |
| brookesia_service_video | Video service supporting local capture encoding (H.264, MJPEG, RGB/YUV) and compressed-stream decoding to pixel formats |  |
| brookesia_service_wifi | Wi-Fi service providing state machine management, AP scanning, connection management, and SoftAP functionality |  |
| Component | Description | Version |
|---|---|---|
| brookesia_agent_coze | Coze agent adapter providing Coze API integration, WebSocket communication, and agent management |  |
| brookesia_agent_helper | Agent helper library providing type-safe definitions, schemas, and unified agent calling interfaces |  |
| brookesia_agent_manager | Agent manager providing AI agent lifecycle management, state machine control, and unified operation interfaces |  |
| brookesia_agent_openai | OpenAI agent adapter providing OpenAI API integration, peer-to-peer communication, and agent management |  |
| brookesia_agent_xiaozhi | Xiaozhi agent adapter providing Xiaozhi API integration |  |
| Component | Description | Version |
|---|---|---|
| brookesia_expression_emote | AI emote and animation management providing emoji sets, animation sets, and event message sets for rich visual feedback in anthropomorphic interactions |  |
If you would like to contribute to ESP-Brookesia or build on the examples in this repository, you can get the master branch with the following command:
bash git clone https://github.com/espressif/esp-brookesia
We currently welcome contributions in the following areas:
- Fixing bugs
- Adding support for new development boards to brookesia_hal_boards
For new component ideas, we recommend starting a discussion in GitHub Issues first so we can align on the use case, interface boundaries, and overall roadmap together. Once there is agreement on the direction, the follow-up contribution process is usually much smoother.
- Latest documentation: https://docs.espressif.com/projects/esp-brookesia/en, built from the docs directory of this repository;
- ESP-IDF Programming Guide: https://docs.espressif.com/projects/esp-idf/en;
- ESP-Brookesia components can be found on ESP Component Registry;
- Visit the esp32.com forum to ask questions and explore community resources;
- If you find a bug or need a new feature, please check GitHub Issues first to avoid duplicate submissions;