Design an Autonomous Robot
Every concept in this module — the limits of hand-coding, supervised and reinforcement learning, imitation, simulation, the autonomy stack, human oversight, and safety — comes together when you design a real robot system from scratch. This lesson is a design session. You will work through the complete engineering specification of an autonomous robot, making principled decisions at each layer and defending them using the frameworks you have built over the past eight lessons. There are no single correct answers, but there are well-reasoned answers and poorly-reasoned ones. The goal is to think like a robotics engineer.
How to Approach a Robot Design Problem
Designing an autonomous robot is a process of resolving trade-offs under constraints. The constraints come from three directions: the task (what the robot must accomplish), the environment (where and with whom the robot will operate), and the resources available (data, compute, time, budget, human oversight). Good robot design makes these constraints explicit, then selects technical approaches that match the constraints rather than the ones that sound most impressive. A principled design process follows this order: First, define success. What does it mean for the robot to have completed its task? What metrics matter, and which trade-offs are acceptable? A robot whose failure mode is 'misses some items' is very different from one whose failure mode is 'causes physical harm.' Second, characterize the environment. Is it structured or unstructured? Static or dynamic? Does it include humans? How much does it vary? The degree of environmental uncertainty determines how much learning is needed and at which layers. Third, select learning approaches by matching them to sub-problems. Supervised learning for perception tasks where labeled data can be collected. Reinforcement learning for control tasks where reward can be defined and simulation is feasible. Imitation learning for complex tasks where human demonstrations are the richest data source. Fourth, design the autonomy stack. Define each layer, its inputs and outputs, its operating frequency, and whether it uses a classical or learned approach. Specify the interfaces between layers. Fifth, define the safety and oversight policy. What autonomy level is appropriate at launch? What evidence is required to increase it? What are the hard constraints? What is the fallback behavior when the robot is uncertain?
When designing a robot system, identify the hardest scenario the robot will encounter — the worst weather, the most cluttered environment, the most unexpected human behavior — and verify that your design handles it adequately. Systems designed only for average cases fail spectacularly in edge cases. In robotics, edge cases are not rare — they are the norm in unstructured environments.
Design Scenarios
Choose one of the following scenarios for the main design activity, or propose your own with instructor approval. Each scenario has different learning demands, safety requirements, and autonomy trade-offs. Scenario A — Last-Mile Delivery Robot: a wheeled robot that autonomously delivers packages on sidewalks in a mixed pedestrian environment, from a loading point to residential front doors, in an urban neighborhood with varying weather. Scenario B — Agricultural Harvesting Robot: a robot arm on a mobile base that identifies ripe fruit on trees or vines, picks it without bruising, and deposits it in a collection bin, operating in open fields across an entire growing season. Scenario C — Disaster Response Scout: a quadruped robot that enters a collapsed building after an earthquake, maps the interior, identifies survivors using thermal and audio sensors, and relays information to rescue teams, without any pre-existing map of the structure. Scenario D — In-Home Physical Rehabilitation Robot: a robot arm that guides a stroke patient through prescribed rehabilitation exercises, adapts the assistance level to the patient's current strength, and reports progress to a remote physical therapist. Each scenario involves: perception under uncertainty, some form of learned control or decision-making, a sim-to-real challenge, an autonomy level decision, and meaningful safety requirements.
Full Autonomous Robot Design
- Choose one scenario from the Design Scenarios section above (or propose your own). Complete all seven parts below. This is a substantial design exercise — budget at least 45-60 minutes. Work individually, then compare and critique designs with a partner.
- Part 1 — Mission Definition (5 minutes)
- Write a precise mission statement: what the robot must do, under what conditions, with what acceptable failure rate, and what failure modes are acceptable versus unacceptable. Be specific enough that you could test whether the mission is accomplished.
- Part 2 — Environment Characterization (5 minutes)
- Describe the operating environment in detail: structured vs. unstructured, static vs. dynamic, presence of humans, sensor challenges, sources of variability. Rate the overall environment complexity 1-5 and justify the rating.
- Part 3 — Learning Approach Map (10 minutes)
- List at least five distinct sub-problems your robot must solve. For each:
- - State the sub-problem precisely
- - Choose a learning approach: supervised learning, RL, imitation learning, or classical (no learning)
- - Justify your choice by connecting it to the specific structure of the sub-problem
- - Describe what training data or reward signal you would use
- Part 4 — Autonomy Stack Design (10 minutes)
- Draw or describe your three-layer autonomy stack (Perception → Planning → Control). For each layer:
- - List the specific components
- - Specify inputs and outputs
- - State operating frequency
- - Identify which components are learned vs. classical and why
- Part 5 — Sim-to-Real Strategy (5 minutes)
- Describe how you will use simulation in your development pipeline. What simulator would you use? What physical parameters would you randomize? What aspects of the environment are hardest to simulate, and how will you handle the resulting sim-to-real gap?
- Part 6 — Autonomy Level and Oversight (5 minutes)
- Assign an initial autonomy level (1-5) to each major capability. Design the human oversight interface for the most safety-critical capability. What information does the human see, and what actions can they take? What criteria must be met before you would raise the autonomy level of any component?
- Part 7 — Safety and Verification (10 minutes)
- List three hard constraints your robot must never violate. For each, specify your safety mechanism (CBF, hardware limit, organizational procedure, or other). Then design a pre-deployment verification test for your robot: what scenarios do you test, how many trials, and what pass/fail criteria do you use?
- Peer Review: Exchange designs with a partner. Review their design using this checklist: (1) Is the mission specific enough to be testable? (2) Is the learning approach for each sub-problem well-matched to the sub-problem structure? (3) Are the hard safety constraints comprehensive and enforced by appropriate mechanisms? (4) Is the autonomy level appropriate given the demonstrated competence and the stakes? Provide written feedback on at least two specific improvements.
A brilliant robot that you cannot verify is safe should not be deployed. A modest robot that reliably does its narrow task, within verified constraints, with appropriate human oversight, delivers real value while posing manageable risk. As robot capabilities increase and verification methods mature, autonomy can be extended incrementally. This conservative-then-expand approach has characterized every successful real-world robot deployment.
A student designs a disaster response robot and chooses full autonomy (Level 5) at launch, arguing the robot must operate without radio contact in collapsed buildings. What is the most important design question the student has not addressed?