Schedule
Date | Lecture | Readings | Logistics | |
---|---|---|---|---|
Module 1: Introduction and Background | ||||
1/19 |
Lecture #1
(Shenlong):
Introduction to Robot Perception [ slides | video | notes ] |
|
||
1/21 |
Lecture #2
(Shenlong):
Poses, Transforms - 3D Transformations - Rotation Representations [ slides | video | notes ] |
|
||
Module 2: Sensing | ||||
1/26 |
Lecture #3
(Shenlong):
Camera I - Image Formation - Perspective Geometry [ slides | video | notes ] |
|
||
1/28 |
Lecture #4
(Shenlong):
Camera II - Epipolar Geometry - Stereo, Event Cameras [ slides | video | notes ] |
Assignment 1 out |
||
2/1 |
Lecture #5
(Shenlong):
Camera III - Multi-view Geometry - Calibration [ slides | video | notes ] |
|
||
2/2 |
Lecture #6
(Shenlong):
Other Sensors I - LiDAR, Radar, Sonar [ slides | video | notes ] |
|||
2/9 |
Lecture #7
(Shenlong):
Other Sensors II - GPS, IMU, Odometer - Touch, Tactile, etc [ slides | video | notes ] |
|
||
2/11 |
Lecture #8
(Shenlong):
Other Sensors III - Sound - Tactile [ slides | video | notes ] |
Assignment 1 due |
||
Module 3: State Estimation | ||||
2/16 |
Lecture #9
(Shenlong):
State Estimation I - State Estimation Theory - Bayes Filtering, Kalman Filters - Particle Filters, Histogram Filters [ slides | video | notes ] |
|
||
2/18 |
Lecture #10
(Shenlong):
State Estimation II - Bayes Filtering, Kalman Filters - Particle Filters, Histogram Filters [ slides | video | notes ] |
|
||
2/23 |
Lecture #11
(Shenlong):
3D Representations - Voxel, Mesh, Points, SDFs - Representation Learning [ slides | video | notes ] |
|
||
2/25 |
Lecture #12
(Shenlong):
Map-based Localization - Map Representations - Registration and Matching [ slides | video | notes ] |
|
Assignment 2 due |
|
3/2 |
Lecture #13
(Shenlong):
SLAM I - RGBD and LiDAR SLAM [ slides | video | notes ] |
|
||
3/4 |
Lecture #14
(Shenlong):
SLAM II - Visual Odometry - Visual SLAM [ slides | video | notes ] |
Project Proposal due |
||
Module 3: Learning-based Perception | ||||
3/9 |
Lecture #15
(Shenlong):
Deep Learning I - MLP, Backprop - CNNs [ slides | video | notes ] |
|
||
3/11 |
Lecture #16
(Shenlong):
Deep Learning II - RNNs, GNNs, Transformers [ slides | video | notes ] |
|
||
3/16 | No class, spring break | |||
3/18 | No class, spring break | |||
3/23 |
Lecture #17
(Shenlong):
Motion Understanding - Optical Flow - Nonrigid Tracking [ slides | video | notes ] |
|||
3/25 |
Lecture #18
(Shenlong):
Semantic Segmentation - 2D Semantic Segmentation - 3D Semantic Segmentation [ slides | video | notes ] |
Assignment 3 due |
||
3/30 |
Lecture #19
(Shenlong):
Object Detection - 2D and 3D Detection [ slides | video | notes ] |
|||
4/1 |
Lecture #20
(Shenlong):
Object Tracking - 2D and 3D Tracking [ slides | video | notes ] |
|||
4/6 |
Lecture #21
(Shenlong):
Object Pose Estimation - 6-DoF Pose Estimation - Articulated Pose Estimation [ slides | video | notes ] |
|||
4/8 |
Lecture #22
(Shenlong):
Forecasting - Motion Forecasting [ slides | video | notes ] |
Assignment 4 due |
||
4/13 |
Lecture #23
(Shenlong):
Simulation I - Intro to Simulation - Sensor Simulation - Sim2Real [ slides | video | notes ] |
|||
4/20 |
Lecture #24
(Shenlong):
Simulation II - Sensor Simulation - Sim2Real [ slides | video | notes ] |
|||
4/22 |
Lecture #25
(Shenlong):
Multi-Modal Perception - Data Fusion - Transfer Learning [ slides | video | notes ] |
Assignment 5 due |
||
Module 4: Case Studies | ||||
4/27 |
Lecture #26
(TBD):
Applications - Self-Driving [ slides | video | notes ] |
|||
4/29 |
Lecture #27
(TBD):
Applications - Mixed Reality [ slides | video | notes ] |
|||
5/4 |
Lecture #28
:
no class, preparing final project [ slides | video | notes ] |
|||
5/6 | Final Project Report due |