Date Lecture Readings Logistics
Module 1: Introduction and Background
1/19 Lecture #1 (Shenlong):
Introduction to Robot Perception
[ slides | video | notes ]

1/21 Lecture #2 (Shenlong):
Poses, Transforms
- 3D Transformations
- Rotation Representations
[ slides | video | notes ]

Module 2: Sensing
1/26 Lecture #3 (Shenlong):
Camera I
- Image Formation
- Perspective Geometry

[ slides | video | notes ]

1/28 Lecture #4 (Shenlong):
Camera II
- Epipolar Geometry
- Stereo, Event Cameras

[ slides | video | notes ]

Assignment 1 out

2/1 Lecture #5 (Shenlong):
Camera III
- Multi-view Geometry
- Calibration

[ slides | video | notes ]

2/2 Lecture #6 (Shenlong):
Other Sensors I
- LiDAR, Radar, Sonar

[ slides | video | notes ]

2/9 Lecture #7 (Shenlong):
Other Sensors II
- GPS, IMU, Odometer
- Touch, Tactile, etc
[ slides | video | notes ]

2/11 Lecture #8 (Shenlong):
Other Sensors III
- Sound
- Tactile
[ slides | video | notes ]

Assignment 1 due
Assignment 2 out

Module 3: State Estimation
2/16 Lecture #9 (Shenlong):
State Estimation I
- State Estimation Theory
- Bayes Filtering, Kalman Filters
- Particle Filters, Histogram Filters

[ slides | video | notes ]

2/18 Lecture #10 (Shenlong):
State Estimation II
- Bayes Filtering, Kalman Filters
- Particle Filters, Histogram Filters

[ slides | video | notes ]

2/23 Lecture #11 (Shenlong):
3D Representations
- Voxel, Mesh, Points, SDFs
- Representation Learning
[ slides | video | notes ]

2/25 Lecture #12 (Shenlong):
Map-based Localization
- Map Representations
- Registration and Matching
[ slides | video | notes ]
  • Probabilistic Robotics, Ch. 9

Assignment 2 due

3/2 Lecture #13 (Shenlong):
SLAM I
- RGBD and LiDAR SLAM
[ slides | video | notes ]

3/4 Lecture #14 (Shenlong):
SLAM II
- Visual Odometry
- Visual SLAM
[ slides | video | notes ]

Project Proposal due
Assignment 3 out

Module 3: Learning-based Perception
3/9 Lecture #15 (Shenlong):
Deep Learning I
- MLP, Backprop
- CNNs
[ slides | video | notes ]

3/11 Lecture #16 (Shenlong):
Deep Learning II
- RNNs, GNNs, Transformers

[ slides | video | notes ]

3/16 No class, spring break
3/18 No class, spring break
3/23 Lecture #17 (Shenlong):
Motion Understanding
- Optical Flow
- Nonrigid Tracking
[ slides | video | notes ]

3/25 Lecture #18 (Shenlong):
Semantic Segmentation
- 2D Semantic Segmentation
- 3D Semantic Segmentation
[ slides | video | notes ]

Assignment 3 due
Assignment 4 out

3/30 Lecture #19 (Shenlong):
Object Detection
- 2D and 3D Detection

[ slides | video | notes ]

4/1 Lecture #20 (Shenlong):
Object Tracking
- 2D and 3D Tracking
[ slides | video | notes ]

4/6 Lecture #21 (Shenlong):
Object Pose Estimation
- 6-DoF Pose Estimation
- Articulated Pose Estimation
[ slides | video | notes ]

4/8 Lecture #22 (Shenlong):
Forecasting
- Motion Forecasting

[ slides | video | notes ]

Assignment 4 due
Assignment 5 out

4/13 Lecture #23 (Shenlong):
Simulation I
- Intro to Simulation
- Sensor Simulation
- Sim2Real
[ slides | video | notes ]

4/20 Lecture #24 (Shenlong):
Simulation II
- Sensor Simulation
- Sim2Real
[ slides | video | notes ]

4/22 Lecture #25 (Shenlong):
Multi-Modal Perception
- Data Fusion
- Transfer Learning
[ slides | video | notes ]

Assignment 5 due

Module 4: Case Studies
4/27 Lecture #26 (TBD):
Applications
- Self-Driving

[ slides | video | notes ]

4/29 Lecture #27 (TBD):
Applications
- Mixed Reality

[ slides | video | notes ]

5/4 Lecture #28 :
no class, preparing final project
[ slides | video | notes ]

5/6 Final Project Report due