Autonomous DriveBot

01

Why we built it

AI Smart came from imagining where human-robot interaction is going: controlling sophisticated hardware should feel as natural as having a conversation. We wanted to show how real-time communication technology (LiveKit) could turn a Raspberry Pi robot car into an agent that sees, listens and responds to commands instantly.

The challenge that drove us was combining autonomous navigation, computer vision and natural voice interaction in one system, on accessible hardware and modern web tools.

02

What it does

AI Smart is a LiveKit-powered robotic car with two-way, low-latency voice conversation and autonomous navigation, controlled through a web-based operator console.

1. Voice and visual interaction

   Talk to the robot naturally and get instant replies from a LiveKit agent. The Raspberry Pi streams its camera to the web UI with minimal latency, and an optional OpenAI vision step can have the robot describe what it sees.

2. Remote control

   Drive forward, backward, left, right or stop. Pan, tilt and centre the camera, look left, right, up or down, and open or close the camera feed on demand.

3. Autonomous navigation (demo mode)

   Using OpenCV, the robot detects ArUco markers, estimates distance, and navigates by itself to the object a marker identifies: a Monster drink (marker 0), a water bottle (marker 1), a cube (marker 2) or a laptop (marker 3).

03

How we built it

The system is distributed across three main parts.

1. Raspberry Pi agent

   The LiveKit Agents runtime orchestrates room connections and real-time communication. Python scripts drive the PiCar-X motors, Picamera2 feeds LiveKit video tracks, and demo mode adds ArUco detection with OpenCV and autonomous navigation.

2. LiveKit Cloud

   Room and session orchestration, WebRTC audio and video streaming, and data channels that carry commands (topic: car-control), on scalable cloud infrastructure.

3. Web console

   A Next.js and React control console using the LiveKit JavaScript SDK, with token-based authentication through a connection-details endpoint and a responsive UI that renders live video next to the command controls.

• Back end: Python 3 with the LiveKit Agents SDK, Picamera2, OpenCV for computer vision and ArUco detection, NumPy for distance estimation, and the OpenAI API (optional) for vision descriptions.
• Front end: Next.js, the LiveKit client SDK, modern web media APIs, and pnpm.

04

Challenges

• Real-time complexity: keeping audio, video and data channels in sync while holding latency low took careful orchestration.
• Raspberry Pi limits: camera processing, OpenCV and motor control all compete for a small CPU.
• Smooth autonomous navigation: path planning that goes to a marker without oscillating.

05

What we’re proud of

• Genuinely low-latency two-way voice: talking to the robot feels natural and immediate.
• A working autonomous navigation system that identifies objects and drives to them on its own.
• A polished, intuitive web console that makes controlling a robot feel accessible.

06

What we learned

• A deep understanding of WebRTC and how to use it for robotics.
• How to drive Raspberry Pi GPIO and motor controllers from Python.
• Managing several hardware peripherals (camera, motors, servos) at once.

07

What’s next

• SLAM: map and localise indoors without needing markers.
• Advanced path planning: obstacle avoidance and dynamic re-routing.
• Natural language: understand complex, multi-step commands.
• Emotion detection: read vocal tone and adapt the robot’s behaviour.