Capstone targetListen → Plan → Navigate → See → Manipulate

Physical AI field manual

Teach robots to understand the room, not just the prompt.

A technical textbook for the jump from AI agents to embodied systems: ROS 2 control, simulation, Isaac workflows, VLA planning, hardware constraints, and an autonomous humanoid capstone.

13: week course path
7: robotics modules
1: humanoid capstone

Start the textbook Inspect the course map

Course map

Six systems a humanoid must coordinate before a demo is credible.

Core robotics systems / textbook architecture

EMB

Embodied intelligence

Why agents need bodies, sensors, timing, space, force, and feedback before they can act in the real world.

ROS

Robot middleware

ROS 2 nodes, topics, services, actions, parameters, launch files, and Python control paths with rclpy.

SIM

Digital twin labs

Gazebo, Unity, URDF, SDF, collision, gravity, depth cameras, LiDAR, IMUs, and testable robot scenes.

ISA

Isaac acceleration

Isaac Sim, Isaac ROS, synthetic data, VSLAM, Nav2, GPU constraints, and sim-to-real transfer habits.

VLA

Vision-language-action

Speech and natural language become bounded robot goals through perception, planning, navigation, and checks.

CAP

Humanoid capstone

A simulated robot listens, plans, navigates, identifies an object, and manipulates it inside one integrated demo.

Learning sequence

The book follows the signal from speech to physical consequence.

01Command

Capture intent without trusting language blindly.

02Plan

Convert the goal into bounded robot actions.

03Simulate

Test physics, sensors, scenes, and failure modes first.

04Deploy

Move only validated behavior toward edge hardware.

Module deck

A 13-week route from embodied foundations to a simulated humanoid.

Scroll to advanceModule cards move horizontally as the page moves downward

Weeks 1-2

Physical AI Foundations

Embodied intelligence, sensors, physical laws, and the shift from software agent to acting system.

Open moduleWeeks 3-5

ROS 2 Control Layer

Nodes, topics, services, actions, launch files, parameters, and Python bridges into robot behavior.

Open moduleWeeks 6-7

Gazebo and Unity Twins

Physics simulation, visual environments, robot descriptions, and sensor testing before hardware risk.

Open moduleWeeks 8-10

NVIDIA Isaac Brain

Isaac Sim, Isaac ROS, VSLAM, Nav2, synthetic data, and edge deployment tradeoffs.

Open moduleWeek 13

Conversational Robotics

Voice-to-action workflows that turn natural language into grounded, validated robot movement.

Open module

Final studio

The capstone is not a page of theory. It is a robot workflow under constraints.

Students design a simulated humanoid that receives a voice command, plans a path, avoids obstacles, identifies an object, and manipulates it. The book keeps the engineering boundaries visible: simulation first, local control for physical motion, and safety notes at every hardware edge.

Preview the capstone