Schedule
Date | Lecture | Readings | Logistics | |
---|---|---|---|---|
Module 1: Introduction and Background | ||||
8/25 |
Lecture #1
(Shenlong):
Introduction to Robot Perception [ slides | video | notes ] |
|
||
8/27 |
Lecture #2
(Shenlong):
Poses, Transforms and Kinematics - 3D Transformations - Rotation Representations - Kinematics [ slides | video | notes ] |
|
||
9/1 | No class |
|||
9/3 |
Lecture #3
(Shenlong):
Sensing I - LiDAR, Radar, Sonar - GPS, IMU, Odometer - Touch, Tactile, etc [ slides | video | notes ] |
Presentation Preference due (11:59pm, Fri 8/27) |
||
9/8 |
Lecture #4
(Shenlong):
Sensing II - Cameras - Image Formation - Perspective Geometry [ slides | video | notes ] |
|
||
9/10 |
Lecture #5
(Shenlong):
Sensing III - Stereo, Structured Light and Event Cameras - Epipolar Geometry - Structure-from Motion [ slides | video | notes ] |
|
||
9/15 |
Lecture #6
(Shenlong):
Deep Learning Review - MLP, Backprop - CNNs, RNNs, Transformers, GNNs - Generative Models [ slides | video | notes ] |
|
||
9/17 |
Lecture #7
(Shenlong):
3D Representations - Voxel, Mesh, Points, SDFs - Representation Learning [ slides | video | notes ] |
|
Paper reviewing starts from this lecture. Choose from papers starting with [R] to review. |
|
9/22 |
Lecture #8
(Shenlong):
Graphical Models and Nonlinear Optimization Review - MRFs, Bayes Nets and Factor Graphs - Map Inference, Marginal Inference - Nonliner Optimization [ slides | video | notes ] |
|
Presentation assignments out (11:59pm, Fri 9/17) |
|
Module 2: Case Studies | ||||
9/24 |
Lecture #9
(Shenlong):
Spatial Perception: Map-based Localization - State Estimation Review - Map Representations - Bayes Filtering, Kalman Filters - Particle Filters, Histogram Filters - Registration and Matching [ slides | video | notes ] |
|
Project Proposal due (11:59pm, Fri 9/24) |
|
9/29 |
Lecture #10
(Shenlong):
Spatial Perception: SLAM - RGBD and LiDAR SLAM [ slides | video | notes ] |
|
||
10/1 |
Lecture #11
(Issac Ngui and Emersion Sie):
Spatial Perception: SLAM - Visual Odometry - Indirect SLAM - Direct SLAM [ slides | video | notes ] |
|
||
10/6 |
Lecture #12
(Abid and Jeffrey Zhang):
Object Perception: Detection and Tracking - 2D and 3D Detection - 2D and 3D Tracking [ slides | video | notes ] |
|
||
10/8 |
Lecture #13
(Zhijian Liu):
Efficient 3D Perception [ slides | video | notes ] |
|||
10/13 |
Lecture #14
(Shenlong Wang):
Spatial Perception: Semantic Scene Understanding - 2D Semantic Segmentation - Instance and Panoptic Segmentation [ slides | video | notes ] |
|
||
10/15 |
Lecture #15
(Joao Marcos Correia Marques, Shreya Gummadi):
Spatial Perception: Semantic Scene Understanding - 3D Segmentation - Semantic Fusion [ slides | video | notes ] |
|
||
10/20 |
Lecture #16
(Mengchao Zhang, James Motes):
Object Perception: Pose Estimation - 6-DoF Pose Estimation - Articulated Pose Estimation [ slides | video | notes ] |
|||
Module 3: Challenges and Frontiers | ||||
10/22 |
Lecture #17
(Yifan Zhao, Yixuan Wang):
Perception in a Moving World - Optical Flow - Nonrigid Tracking [ slides | video | notes ] |
|
||
10/27 |
Lecture #18
(Sayantani Basu, Jiachen Li):
Perception in a Moving World - Motion Forecasting - Streaming Perception [ slides | video | notes ] |
|
||
10/29 |
Lecture #19
(Shenlong Wang, Roger Qiu):
Perception in an Open World - Open-set Perception - Out-of-Distribution Robustness - Continual / Life-long Learning - Exploration [ slides | video | notes ] |
|
Mid-term Presentaion (4:15pm, Fri 10/29) |
|
TBD |
Lecture #20
:
Mid-term Project Presentations [ slides | video | notes ] |
|||
11/3 |
Lecture #21
(Mohit Goyal, Sara Aghajanzadeh):
Perception and Action - Active Perception [ slides | video | notes ] |
|||
11/5 |
Lecture #22
(Shaowei Liu, Aditya Prakash):
Perception and Action - Perceive for manipulation - Perceive for navigation [ slides | video | notes ] |
|
||
11/10 |
Lecture #23
(German Ros):
Invited Speaker Talk: TBD [ slides | video | notes ] |
|||
11/12 |
Lecture #24
(Kaiwen Hong, Kendall Koe):
Multi-Modal Perception - Multi-sensor Perception [ slides | video | notes ] |
|
||
11/17 |
Lecture #25
(Shenlong Wang, Arjun Gupta):
Perception and Simulation - Intro to Simulation - Sim2Real Domain Transfer [ slides | video | notes ] |
|
||
11/19 |
Lecture #26
(Yuan Shen, Vlas Zyrianov):
Learning to Sim - Realistic Sensor Simulation - Scenario Simulation [ slides | video | notes ] |
|
||
11/24 | No class. Thanksgiving | |||
11/26 | No class. Thanksgiving | |||
12/1 |
Lecture #27
(Luca Carlone):
Invited Speaker Talk: TBD [ slides | video | notes ] |
|||
12/3 |
Lecture #28
(Amish Goel, Patrick Naughton):
Perception with Guarantees - Uncertainty Modeling - Adversarial Robustness - Certifiable Perception [ slides | video | notes ] |
|
Extended office hour for final project (4:15pm, Fri 12/3) |
|
Module 4: Final Project Presentations | ||||
12/8 | Final presentations (2pm to 4pm) |