Guide to Edge-AI on Wheels: Deploying Local Ollama and Vision Models to a Tracked Robot Chassis for Object Sorting

Edge‑AI on Wheels: Deploying Local Ollama and Vision Models to a Tracked Robot Chassis for Object Sorting

In this step‑by‑step tutorial you’ll learn how to turn a simple tracked robot chassis into an autonomous edge‑AI sorter. We’ll install Ollama locally, run a lightweight vision model, and integrate the inference pipeline with robot motion control—all without relying on cloud services.

---

1️⃣ Overview – Why Edge‑AI on a Tracked Platform?

Edge‑AI gives you:

• Zero latency – inference runs directly on the robot.
• Privacy – no data leaves the device.
• Reliability – works offline and in low‑connectivity environments.

Using Ollama means you can pull and run LLM or vision models with a single command, while a compact GPU (e.g., NVIDIA Jetson Nano) handles the heavy lifting for visual detection.

2️⃣ Recommended Hardware

Component	Suggested Model	Key Specs
Chassis	2‑WD Tracked Kit (e.g., DFRobot 4WD Smart Robot Car Kit)	Aluminum frame, 12 V DC motors, encoder feedback
Compute	NVIDIA Jetson Nano 4 GB	CUDA 6.2, 4 GB LPDDR4, 2 × USB‑3.0
Camera	Raspberry Pi HQ Camera (12 MP) + CS‑mount lens	1080p @30 fps, adjustable focus
Power	2 × 12 V 5 Ah Li‑Po packs + DC‑DC buck (5 V for Jetson)	~30 min runtime @ max speed
Middleware	ROS 2 Foxy (Ubuntu 20.04)	Pub/Sub, real‑time safety nodes

3️⃣ Software Stack & Installation

3.1 Install Ubuntu 20.04 (LTS)

Flash the Jetson Nano with NVIDIA JetPack 4.6. Choose the “Ubuntu Desktop” image – it includes CUDA, cuDNN, and the necessary drivers.

3.2 Set Up ROS 2 Foxy

sudo apt update && sudo apt install -y curl gnupg2 lsb-release
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
sudo apt update
sudo apt install -y ros-foxy-desktop
echo "source /opt/ros/foxy/setup.bash" >> ~/.bashrc
source ~/.bashrc
            

3.3 Install Ollama (CPU/GPU hybrid)

Ollama provides a deb package that works on Ubuntu.

curl -fsSL https://ollama.com/install.sh | sh
# Verify
ollama --version
            

Optional – enable GPU acceleration for vision models:

ollama serve --gpu
            

3.4 Pull a Vision Model (YOLO‑v8 Tiny)

Ollama hosts community‑built models. The tiny variant fits the Jetson Nano’s memory budget.

ollama pull yolov8-tiny
            

Test the model on a sample image:

ollama run yolov8-tiny < sample.jpg
            

4️⃣ Integrating Vision with Robot Motion (ROS 2 Nodes)

We create two ROS 2 nodes:

1. camera_node – publishes raw sensor_msgs/Image at 15 fps.
2. vision_node – subscribes to the image, runs Ollama inference, and publishes detection results (class, confidence, bbox).

4.1 camera_node (Python)

import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
import cv2
from cv_bridge import CvBridge

class CameraNode(Node):
    def __init__(self):
        super().__init__('camera_node')
        self.pub = self.create_publisher(Image, 'camera/image_raw', 10)
        self.timer = self.create_timer(0.066, self.publish_frame)  # ~15 fps
        self.cap = cv2.VideoCapture(0)
        self.bridge = CvBridge()

    def publish_frame(self):
        ret, frame = self.cap.read()
        if ret:
            msg = self.bridge.cv2_to_imgmsg(frame, encoding='bgr8')
            self.pub.publish(msg)

def main(args=None):
    rclpy.init(args=args)
    rclpy.spin(CameraNode())
    rclpy.shutdown()

if __name__ == '__main__':
    main()
            

4.2 vision_node (Python + Ollama CLI)

import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from std_msgs.msg import String
from cv_bridge import CvBridge
import subprocess
import json
import base64

class VisionNode(Node):
    def __init__(self):
        super().__init__('vision_node')
        self.sub = self.create_subscription(Image, 'camera/image_raw', self.callback, 10)
        self.pub = self.create_publisher(String, 'detections', 10)
        self.bridge = CvBridge()

    def callback(self, msg):
        # Convert ROS Image → JPEG bytes
        cv_img = self.bridge.imgmsg_to_cv2(msg, desired_encoding='bgr8')
        _, buf = cv2.imencode('.jpg', cv_img)
        img_b64 = base64.b64encode(buf).decode('utf-8')

        # Run Ollama model (runs locally, very fast)
        process = subprocess.run(
            ['ollama', 'run', 'yolov8-tiny', '--image', img_b64],
            capture_output=True, text=True
        )
        if process.returncode != 0:
            self.get_logger().error('Ollama inference failed')
            return

        detections = json.loads(process.stdout)
        # Publish as a simple JSON string
        out_msg = String()
        out_msg.data = json.dumps(detections)
        self.pub.publish(out_msg)

def main(args=None):
    rclpy.init(args=args)
    rclpy.spin(VisionNode())
    rclpy.shutdown()

if __name__ == '__main__':
    main()
            

> Tip: Wrap the Ollama call in an async thread if you need higher frame rates.

Now create a third node, sorter_node, that subscribes to /detections and commands the motor drivers based on object class (e.g., “plastic”, “metal”, “trash”).

5️⃣ Real‑Time Sorting Logic

5.1 Motor Controller Interface (PWM)

import RPi.GPIO as GPIO
import time

LEFT_PWM  = 12   # GPIO12 (PWM0)
RIGHT_PWM = 13   # GPIO13 (PWM1)

GPIO.setmode(GPIO.BCM)
GPIO.setup(LEFT_PWM, GPIO.OUT)
GPIO.setup(RIGHT_PWM, GPIO.OUT)

left_motor  = GPIO.PWM(LEFT_PWM, 1000)   # 1 kHz
right_motor = GPIO.PWM(RIGHT_PWM, 1000)

left_motor.start(0)
right_motor.start(0)

def set_speed(l, r):
    left_motor.ChangeDutyCycle(l)   # 0‑100 %
    right_motor.ChangeDutyCycle(r)

def stop():
    set_speed(0, 0)
            

5.2 sorter_node (Decision Engine)

import rclpy
from rclpy.node import Node
from std_msgs.msg import String
import json
from motor_control import set_speed, stop   # import from 5.1

class SorterNode(Node):
    def __init__(self):
        super().__init__('sorter_node')
        self.sub = self.create_subscription(String, 'detections', self.handle_detections, 10)

        # Simple mapping: class → motor speed pattern
        self.policy = {
            'plastic': (30, 30),   # forward slow
            'metal'  : (60, 30),   # turn right
            'paper'  : (30, 60),   # turn left
            'trash'  : (0, 0)      # stop for manual removal
        }

    def handle_detections(self, msg):
        detections = json.loads(msg.data)
        if not detections:
            self.get_logger().info('No objects detected')
            stop()
            return

        # Choose the highest‑confidence object
        top = max(detections, key=lambda d: d['confidence'])
        cls = top['class']
        speeds = self.policy.get(cls, (20