Listen
A lit button and spoken invitation turn voice capture into a clear, single action.
Embodied AI · Physical Computing · 2026
Hey, Cili
A physical joke critic that listens to a visitor, interprets the humor, delivers a dry response, and turns an AI judgment into light, voice, expression, and movement.
Awaiting input
Tell me
something
funny.
Cili does not just answer. It listens, rates the joke from 0–10, labels its humor, talks back, and physically reveals the verdict.
VOICE CHANNEL READY
01 · The Premise
Can a machine judge a joke without killing it?
Humor is contextual, social, and often unexplainable. Hey, Cili turns that ambiguity into a short encounter between a visitor and a character: offer a joke, wait for the machine to make sense of it, then receive a score and a deadpan reply.
The goal was not to build a neutral assistant. It was to give an AI system a visible point of view — and make its interpretation feel like a small physical performance.
Interpret
The system reduces an open-ended joke to a score, a humor keyword, and a short commentary.
Perform
The answer leaves the console and becomes a face, a voice, a glowing number, and a moving arm.
02 · Interaction Flow
One joke, six coordinated responses
The experience is deliberately short, but the implementation coordinates speech recognition, language-model output, synthesized voice, screen animation, an LED matrix, and a servo. Each stage hands off one small, predictable artifact to the next.
Invite
The robot asks for a joke and arms the exhibition interaction.
Record
The visitor holds the illuminated control to speak and releases it to finish.
Transcribe
A local whisper.cpp model converts the recorded voice into text.
Judge
DeepSeek returns structured JSON: score, keyword, and commentary.
Speak
TTS generates the reply while TouchDesigner gives the robot a responsive face.
Reveal
Arduino displays the 0–10 score and moves the arm, then resets for the next visitor.
03 · Working System
Designed to survive an open exhibition
The final repository is a working Windows/Python installation rather than a visual prototype. Its exhibition mode guards against repeated button presses, waits for each spoken reply to finish, clears temporary audio, and returns the robot to a ready state for the next person.
Local speech recognition
whisper.cpp handles transcription locally, producing a clean text handoff for the model.
A strict response contract
The LLM must return valid JSON with a 0–10 score, a concise humor label, and 2–3 short sentences.
One shared voice file
Python renders the reply as WAV; TouchDesigner detects, copies, and plays it with the on-screen character.
A resilient physical link
PING/PONG handshakes, reconnect logic, and SHOW/HIDE commands keep the LED score and servo synchronized.
VOICE→WHISPER→DEEPSEEK→TTS→TOUCHDESIGNER+ARDUINO
04 · Physical Character
The interface is the object
The screen becomes Cili's face, the side-mounted LED matrix makes the judgment public, and the servo-driven arm gives the score a small mechanical flourish. The retro monitor form makes the system approachable while keeping its machine identity intact.
FACE
Animated visual feedback gives attention and timing to the exchange.
SCORE
The 8×8 matrix makes an abstract model output immediately legible.
GESTURE
A servo lifts the arm when the verdict appears, then returns to rest.
05 · In the Wild
Every conversation from the exhibition
Once exhibited, the work became a collection of unscripted exchanges. Visitors brought different languages, rhythms, confidence, and definitions of what counts as funny. The video below preserves the complete set of conversations recorded after the installation went on display.
Full exhibition conversation archive · Open on Bilibili ↗
06 · Reflection
What the robot taught me
Latency has a personality
In conversation, a pause is never neutral. Audio cues, facial feedback, and clear state changes make the system's thinking time feel intentional.
Embodiment raises the stakes
A score on a console is data; a glowing number, spoken roast, and moving arm feel like a public judgment.
Reliability is part of the design
Reconnection, input guards, cleanup, and reset behavior are invisible until they fail — and essential when strangers control the work.
Next iteration
Use the conversation archive to study scoring consistency, language bias, and how different audiences perform humor for a machine.
Explore the working code, hardware protocol, and installation setup.
Open Joke-robot on GitHub