Reachy Mini × Gemma 4 — a fully on-device robot companion
A small desk robot that hears, sees, talks and moves, with every model running on the robot's own Jetson.
What it is
A Reachy Mini desk robot driven by Gemma 4 E4B running entirely on a 16 GB NVIDIA Jetson Orin NX. There is no cloud service in the loop, and it keeps working offline.
Gemma 4 everywhere
- On-device LLM. Gemma 4 E4B, using Google's QAT int4 weights, served by a fork of TensorRT Edge-LLM.
- Native audio input. Gemma 4 hears the user's raw audio directly; the Whisper transcript is passed along only as a hint.
- Multimodal vision. When asked, it looks through the robot's camera and answers about what it actually sees.
- Tool calls. It calls tools to move its head and base, take a look, and check who is present.
- Character. It is playful: it jokes about having no hands, and then actually performs the move it offered.
Look-to-talk
Listening can be triggered by gaze instead of a wake word. Face detection, facial landmarks and an eye-gaze model run on the GPU; when the recognized owner looks at the robot, it does its "waking" move and starts listening. Looking away cancels.
In a noisy room, if speech recognition only hears noise, Gemma 4 listens to the raw audio itself, and stays silent if nobody actually spoke.
Honest by design
- It won't claim to see something without taking a fresh look through the camera.
- Face identity is a personalization hint, never authentication.
- A quiet mode suppresses all speech.
On-device stack
- NVIDIA Jetson Orin NX 16 GB, JetPack 7.2
- Gemma 4 E4B on TensorRT Edge-LLM
- whisper.cpp speech recognition
- Phonon on-device text-to-speech (CPU)
- DeepStream 9.1 perception at 20 fps: YOLO26n people tracking, YuNet face detection, MobileGaze eye gaze, SFace recognition
- Reachy Mini body, driven through a collision-clamped motion service
Where it's going: the Jetson inside the robot
Today the Jetson sits next to the Reachy Mini and drives it over USB. The goal is a single self-contained desk robot: replace the Reachy's base with a custom base that houses the Jetson Orin NX, so the brain lives inside the body.
That turns heat into the main design problem. Under load the board draws about 10–20 W, and in open air at a busy hackathon it still reached the low 80s °C with its fan at full speed. A closed base around motors and plastic leaves less room for that.
- Heat budget first. We already cut the vision pipeline's work (20 fps instead of ~49, tracking only people, the debug video stream off by default), which took the idle board from roughly 20 W to about 9 W and from the low 80s to around 60 °C.
- Airflow and heat spreading. The base needs an intake and exhaust path, and a heatsink coupled to the enclosure, without blowing hot air over the motors.
- Acoustics. A fan inside the body sits close to the microphone array, so fan placement, isolation and noise suppression have to be designed together with the voice pipeline.
- Graceful when hot. A thermal gate pauses new answers before the chip throttles, and the robot says so in character: “Phew, it's so hot in here my brain stopped working. Give me a moment to cool off!”