Shenlong Wang

Shenlong Wang

I am an Assistant Professor at the UIUC Department of Computer Science, affiliated with Computer Vision @ UIUC, Illinois Robotics Group and Center for Immersive Computing @ UIUC. I finished my PhD at Department of Computer Science, University of Toronto and worked as a research scientist at Uber, advised by Raquel Urtasun.

My research interests lie at the intersection of 3D computer vision and robotics, particularly in two areas: (a) developing algorithms to perceive and model the 3D world, and (b) creating accurate, realistic, and physics-informed digital twins of objects and scenes. Our research group is exploring various topics within this field, such as robust and uncertainty-aware robot perception, modeling articulated objects from videos, generating 3D scenes and sensors through machine learning, developing safe autonomy for self-driving, and creating physics-informed simulations for immersive computing and climate change.

News

Invited talk at AICE Spring Research Symposium
Three papers have been accepted by CVPR 2024. Congratulations to the team!
We are grateful to the NSF Robust Intelligence Program for suppporting our research on digital twins with a CAREER award.
Invited talk at IMMERSE Annual Symposium
Thank you Meta for the research gift support!
One paper has been accepted by RA-L. Congratulations to the team!
One paper has been accepted by NeurIPS 2023. Congratulations to Tianhang!
Invited talk at C3.ai Generative AI Workshop
We are grateful to the NSF Safe Learning-Enabled Systems Program for sponsoring our research on safe autonmous .
We are grateful to the NSF Robust Intelligence Program for sponsoring our research on neural scene models.
Thank you Amazon-Illinois AICE center for the research grant support!
Four papers have been accepted by ICCV 2023. Congratulations to Albert, Yuan, Zhi-Hao, Xiyue and Shaowei!
Thank you Intel for the research gift support!
Talks at CVPR 2023: Scene Understanding Workshop, Secure and Safe Autonomous Driving Workshop.
Co-host the End-to-End Autonomous Driving Workshop at CVPR 2023.
Thank you IBM for the research grant support!
Vlas won New Frontier Fellowship. Congratulations to Vlas!
Thank you Jump-Arches and NCSA for the research grant support!
One paper has been accepted by CVPR 2023. Congratulations to Shaowei!
One paper has been accepted by L4DC 2023 as spotlight. Congratulations to Sheng!
Two papers have been accepted by NeurIPS 2022. Congratulations to Yuan, Yuefan and Zeyuan!
Thank you Insper-Illinois fund for the research grant support!
Our group is honored to receive the Amazon Research Award.

Positions

We are always looking for motivated students to join us.

Prospective graduate students: please apply to the Ph.D. programs at UIUC CS/ECE and list me as a potential advisor.
Current UIUC grad and undergrad students: please send me an email with your CV and transcript if you are interested.
Visiting students: please send me your CV and transcript through email.

Current Group Members

PhD Students:

Hanxiao Jiang (CS PhD, joint with Yunzhu Li)
Zhi-Hao Lin (CS PhD)
Shaowei Liu (CS PhD, joint with Saurabh Gupta)
Yuan Shen (CS PhD)
Jing Wen (CS PhD, joint with Alex Schwing)
Albert Jianqiao Zhai (CS PhD)
Vlas Zyrianov (CS PhD)

Master Students:

Bhargav Chandaka (CS MS)
Tianhang Chen (CS MS, joint with Kaiyu Guan)
Max Hsu (CS MS)
David Yao (CS MS)

Visiting Students:

Hongchi Xia (BS, SJTU)

Undergraduate Students:

Gloria Wang (CS BS)
Emily Chen (CS BS)
Henry Che (ECE BS)

Past Members:

See this link

Teaching

Publications

		Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video Hongchi Xia, Zhi-Hao Lin, Wei-Chiu Ma, Shenlong Wang International Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [Paper] [Project] [Code]
		NeRF2Physics: Physical Property Understanding from Language-Embedded Feature Fields Albert Jianqiao Zhai, Yuan Shen, Emily Chen, Gloria Wang, Xinlei Wang, Sheng Wang, Kaiyu Guan, Shenlong Wang International Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [Paper] [Project] [Code]
		GoMAvatar: Efficient Animatable Human Modeling from Monocular VideoUsing Gaussians-on-Mesh Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alex Schwing, Shenlong Wang International Conference on Computer Vision and Pattern Recognition (CVPR), 2024 [Paper] [Project] [Code]
		Objects With Lighting: A Real-World Dataset for Evaluating Reconstruction and Rendering for Object Relighting Benjamin Ummenhofer, Sanskar Agrawal, Rene Sepulveda, Yixing Lao, Kai Zhang, Tianhang Cheng, Stephen Richter, Shenlong Wang, German Ros International Conference on 3D Vision (3DV), 2024 (Oral) [Paper] [Project] [Code]
		On the Overconfidence Problem in Semantic 3D Mapping Joao M C Marques, Albert Jianqiao Zhai, Shenlong Wang, Kris Hauser International Conference on Robotics and Automation (ICRA), 2024 [Paper] [Project] [Code]
		Sim-on-Wheels: Physical World in the Loop Simulation for Autonomous Driving Yuan Shen, Bhargav Chandaka, Zhi-Hao Lin, Albert Jianqiao Zhai, Hang Cui, David Forsyth, Shenlong Wang Robotics and Automation Letters (RAL), 2023 [Paper] [Project] [Code]
		Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects Tianhang Cheng, Wei-Chiu Ma, Kaiyu Guan, Antonio Torralba, Shenlong Wang Neural Information Processing Systems (NeurIPS), 2023 [Paper] [Project] [Code]
		ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field Yuan Li, Zhi-Hao Lin, Jia-Bin Huang, David Forsyth, Shenlong Wang International Conference on Computer Vision (ICCV), 2023 [Paper] [Project] [Code]
		PEANUT: Predicting and Navigating to Unseen Targets Albert Jianqiao Zhai, Shenlong Wang International Conference on Computer Vision (ICCV), 2023 [Paper] [Project] [Code]
		MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models Xiyue Zhu, Vlas Zyrianov, Zhijian Liu, Shenlong Wang International Conference on Computer Vision (ICCV), 2023 [Paper] [Project] [Code]
		ContactGen: Generative Contact Modeling for Grasp Generation Shaowei Liu, Yang Zhou, Jimei Yang, Saurabh Gupta, Shenlong Wang International Conference on Computer Vision (ICCV**), 2023 [Paper] [Project] [Code]
		Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Clouds Shaowei Liu, Saurabh Gupta, Shenlong Wang International Conference on Computer Vision and Pattern Recognition (CVPR**), 2023 [Paper] [Project] [Code]
		DiffTune+: Hyperparameter-Free Auto-Tuning using Auto-Differentiation Sheng Cheng, Lin Song, Minkyung Kim, Shenlong Wang, Naira Hovakimyan Annual Learning for Dynamics and Control Conference (L4DC), 2023 (Oral) [Paper] [Code]
		CASA: Category-agnostic Skeletal Animal Reconstruction Yuefan Wu, Zeyuan Chen, Shaowei Liu, Zhongzheng Ren, Shenlong Wang Neural Information Processing Systems (NeurIPS), 2022 [Paper] [Project] [Code]
		SGAM: Building a Virtual 3D World through Simultaneous Generation and Mapping Yuan Shen, Wei-Chiu Ma, Shenlong Wang Neural Information Processing Systems (NeurIPS), 2022 [Paper] [Project] [Code] [Video]
		Learning to Generate Realistic LiDAR Point Cloud Vlas Zyrianov, Xiyue Zhu, Shenlong Wang European Conference on Computer Vision (ECCV), 2022 [Paper] [Project] [Code] [Video]
		Virtual Correspondence Wei-Chiu Ma, Joyce Yang, Shenlong Wang, Raquel Urtasun, Antonio Torralba International Conference on Computer Vision and Pattern Recognition (CVPR), 2022 [Paper] [Project] [Media] [Video]
		NeurMiPs: Neural Mixture of Planar Experts for View Synthesis Zhi-Hao Lin, Wei-Chiu Ma, Yu-Chiang Frank Wang, Hao-Yu Max Hsu, Shenlong Wang International Conference on Computer Vision and Pattern Recognition (CVPR), 2022 [Paper] [Project] [Code] [Video]
		Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild Shivam Duggal, Zihao Wang, Wei-Chiu Ma, Siva Manivasagam, Justin Liang, Shenlong Wang Raquel Urtasun Winter Conference on Applications of Computer Vision(WACV), 2022 [Paper] [Video]
		GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Siva Manivasagam, Shangjie Xue, Ersin Yumer, Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (Oral, Best Paper Candidate*) [Paper] [Poster] [Video] [Project]
		SceneGen: Learning to Generate Realistic Traffic Scenes Shuhan Tan, Kelvin Wong, Shenlong Wang, Siva Manivasagam, Mengye Ren, Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), 2021 [Paper] [Video]
		S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling Ze Yang, Shenlong Wang, Siva Manivasagam, Zeng Huang, Wei-Chiu Ma, Xinchen Yan, Ersin Yumer, Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), 2021 [Paper] [Video]
		Asynchronous Multi-View SLAM Joyce Yang, Can Cui, Ioan Andrei Bârsan, Raquel Urtasun, Shenlong Wang* International Conference on Robotics and Automation (ICRA), Xi'an, 2021 [Paper] [Supp] [Video] [Project]
		MuSCLE: Multi Sweep Compression of LiDAR using Deep Entropy Models Sourav Biswas, Jerry Liu, Kelvin Wong, Shenlong Wang, Raquel Urtasun, Neural Information Processing Systems (NeurIPS*), 2020 [Paper] [Video] [Project]
		Pit30M: A Benchmark for Global Localization in the Age of Self-Driving Cars Julieta Martinez, Sasha Doubov, Jack Fan, Ioan Andrei Bârsan, Shenlong Wang Gellert Mattyus, Raquel Urtasun, International Conference on Intelligent Robots and Systems (IROS), 2020 (Best Application Paper Runner-up) [Paper] [Video] [Project] [Code & Data]
		Deep Feedback Inverse Problem Solver Wei-Chiu Ma, Shenlong Wang, Jiayuan Gu, Sivabalan Manivasagam, Antonio Torralba, Raquel Urtasun European Conference on Computer Vision (ECCV), Glasgow, 2020 (Spotlight) [Paper] [Video] [Project]
		Conditional Entropy Coding for Efficient Video Compression Jerry Liu, Shenlong Wang, Wei-Chiu Ma, Meet Shah, Rui Hu, Pranaab Dhawan, Raquel Urtasun European Conference on Computer Vision (ECCV), Glasgow, 2020 [Paper] [Video]
		Deep Structured self-Driving Network Wenyuan Zeng, Shenlong Wang, Renjie Liao, Yun Chen, Bin Yang, Raquel Urtasun European Conference on Computer Vision, Glasgow, 2020 [Paper]
		LidarSIM: Realistic LiDAR Simulation by Leveraging the Real World Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Bin Yang, Shuhan Tan, Mikita Sazanovich, Wei-Chiu Ma and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020 (Oral) [Paper]
		OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression Lila Huang, Shenlong Wang, Kelvin Wong, Jerry Liu, and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020 (Oral) [Paper]
		Identifying Unknown Instances for Autonomous Driving Kelvin Wong, Shenlong Wang, Mengye Ren, and Raquel Urtasun International Conference on Robot Learning (CoRL), Osaka, 2019 (Spotlight) [Paper]
		Efficient Graph Generation with Graph Recurrent Attention Networks Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, Charlie Nash, Yujia Li, William L. Hamilton, David Duvenaud, Raquel Urtasun, and Richard Zemel Neural Information Processing Systems (NeurIPS), Vancouver, 2019 [Paper] [Code]
		DSIC: Deep Stereo Image Compression Jerry Liu, Shenlong Wang, and Raquel Urtasun International Conference on Computer Vision (ICCV), Seoul, 2019 (Oral) [Paper]
		DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu, and Raquel Urtasun International Conference on Computer Vision (ICCV), Seoul, 2019 [Paper] [Code]
		Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization Wei-Chiu Ma, Ignacio Tartavull, Ioan Andrei Bârsan, Shenlong Wang, Gellert Mattyus, Min Bai, Namdar Homayounfar, Shrinidhi K. Lakshmikanth, Andrei Pokrovsky Raquel Urtasun, International Conference on Intelligent Robots and Systems (IROS**), 2020 [Paper] [Video]
		Deep Rigid Instance Scene Flow Wei-Chiu Ma, Shenlong Wang, Rui Hu, Yuwen Xiong, and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 2019 (1st place on KITTI Scene Flow) [Paper]
		Learning to Localize through Compressed Binary Maps Xinkai Wei, Ioan Andrei Barsan, Shenlong Wang, Julieta Martinez, and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR*), Long Beach, 2019 [Paper]
		Convolutional Recurrent Network for Road Boundary Extraction Justin Liang, Namdar Homayounfar, Wei-Chiu Ma, Shenlong Wang, and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 2019 [Paper]
		Learning to Localize Using a LiDAR Intensity Map Ioan Andrei Barsan, Shenlong Wang, Andrei Pokrovsky, and Raquel Urtasun Conference on Robot Learning (CORL), Zurich, 2018 (Spotlight) [Paper][Video]
		Deep Multi-Sensor Lane Detection Min Bai, Gellert Mattyus,Namdar Homayounfar,Shrinidhi Kowshika Lakshmikanth, Shenlong Wang, and Raquel Urtasun International Conference on Intelligent Robots and Systems (IROS), Madrid, 2018 [Paper]
		Deep Continuous Fusion for Multi-Sensor 3D Object Detection Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun European Conference on Computer Vision (ECCV), Munich, 2018 (1st place on KITTI BEV Detection) [Paper]
		Deep Parameteric Continuous Convolutional Neural Networks Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018 (Spotlight) [Paper]
		TorontoCity: Seeing the World with a Million Eyes Shenlong Wang, Min Bai, Gellert Mattyus, Hang Chu, Wenjie Luo, Bin Yang, Justin Liang, Joel Cheverie, Sanja Fidler, Raquel Urtasun International Conference on Computer Vision (ICCV), Venice, 2017 (Spotlight*) [Paper] [Update]
		AutoScaler: Scale-Attention Networks for Visual Correspondence Shenlong Wang, Linjie Luo, Ning Zhang and Jia Li British Machine Vision Conference (BMVC), London, 2017 (Oral) [Paper]
		Find Your Way by Observing the Sun and Other Semantic Cues Wei-Chiu Ma, Shenlong Wang, Sanja Fidler and Raquel Urtasun International Conference on Robotics and Automation (ICRA), Singapore, 2017 [Paper] [Video]
		Proximal Deep Structured Models Shenlong Wang, Sanja Fidler and Raquel Urtasun Neural Information Processing Systems (NeurIPS), Barcelona, 2016 [Paper][Supp][Code] [Data] [Update]
		HouseCraft: Building Houses from Rental Ads and Street Views Hang Chu, Shenlong Wang, Sanja Fidler and Raquel Urtasun European Conference on Computer Vision (ECCV), Amsterdam, 2016 [Paper] [Code] [Data]
		Holoportation: Virtual 3D Teleportation in Real-time Sergio Orts-Escolano et al. User Interface Software and Technology Symposium European (UIST), Tokyo, 2016 (Oral) [Paper] [Video]
		The Global Patch Collider Shenlong Wang, Sean Fanello, Christoph Rhemann, Shahram Izadi and Pushmeet Kohli International Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016 (Oral) [Paper] [OpenCV]
		HD Maps: Fine-grained Road Segmentation by Parsing Ground and Aerial Images Gellert Mattyus, Shenlong Wang, Sanja Fidler and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016 [Paper] [Data][Video]
		Lost Shopping! Monocular Localization in Large Indoor Spaces Shenlong Wang, Sanja Fidler and Raquel Urtasun International Conference on Computer Vision (ICCV), Santiago, 2015 (Oral) [Paper] [Data]
		Enhancing World Maps by Parsing Aerial Images Gellert Mattyus, Shenlong Wang, Sanja Fidler and Raquel Urtasun International Conference on Computer Vision (ICCV), Santiago, 2015 [Paper] [Code] [Data][Video]
		Holistic 3D Scene Understanding from a Single Geo-tagged Image Shenlong Wang, Sanja Fidler and Raquel Urtasun International Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 2015 (Oral) [Paper] [Code]

		Efficient Inference for Contiuous Markov Random Fields with Polynomial Potentials Shenlong Wang, Alex Schwing and Raquel Urtasun Neural Information Processing Systems (NeurIPS), Montreal, 2014 [Paper] [Code]

		Transductive Gaussian Process on Image Denoising Shenlong Wang, Lei Zhang and Raquel Urtasun International Conference on Computational Photography (ICCP), Santa Clara, 2014 (Oral) [Paper] [Bibtex]

		Semi-Coupled Dictionary Learning with Applications in Image Super-resolution and Photo-Sketch Synthesis Shenlong Wang, Lei Zhang and Yan Liang IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 2012 [Paper] [Code]

		Nonlocal Spectral Prior Model for Low-level Vision Shenlong Wang, Lei Zhang and Yan Liang Asian Conference on Computer Vision (ACCV), Korea, 2012 [Paper] [Bibtex]

		Relaxed Collaborative Representation for Pattern Classification Meng Yang, Lei Zhang, David Zhang and Shenlong Wang IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 2012 [Paper] [Bibtex]

Services

Area Chair / Associate Editor / SPC: CVPR 2021-now, ICCV 2023, ECCV 2024, NeurIPS 2023, IROS 2023, ICRA 2024, WACV 2023, AAAI 2021
Journal Reviewer: PAMI, IJCV, TRO, TMLR, TIP, TGRS, CVIU, PR, IVC, TCSVT, TASE, TNNLS
Conference PC Member / Reviewer: CVPR 2015-now, ECCV 2016-now, ICCV 2017-now, ICLR 2017-now, ICML 2016-now, Neurips 2016-now, UAI 2015-2018, ICRA 2016-now, IROS 2018-now, RSS 2021-now, Siggraph Asia 2023-now, Siggraph 2023-now
Organizers: CVPR 2021 Tutorial on Visual Compression, CVPR 2021 Tutorial on Self-Driving, CVPR 2023 Workshop on End-to-end Self-Driving

Honors and Awards

NSF CAREER Award, 2024
Amazon Research Award, 2022
CVPR Best Paper Award Candidate, 2021
IROS Best Application Paper Finalist, 2020
Neurips Top Reviewer, 2020
Facebook Fellowship (13 world-wide, declined), 2017
Adobe Fellowship (10 world-wide, declinded), 2017
RBC Fellowship (3 university-wide), 2017
Platform Computing Graduate Fellowship, UofT, 2016

Talks

Modelling, Editing and Generating Urban Digital Twins, Midwest Machine Learning Workshop, 2023
Visual Imagination in the 3D Physical World, China3DV, 2023
Safely Evaluate Autonomous Vehicles with Mixed Reality, Amazon, 2023
Modeling and Generating Digital Twins, Nvidia, 2022
Extreme 3D Reconstruction and Simulation, Autodesk, 2022
Towards Realistic and Scalable Simulation for Autonomy, CVPR/UIUC, 2021
Safely Train and Test Autonomous Vehicles with Augmented Reality, Kent Seminars, 2021
Neural Compression for Geometric Data, CVPR, 2021
Learning 3D Representations for Self-Driving, UofT/NVIDIA/UIUC, 2020
Learning to Drive with a Touch of Human Knowledge, UIUC/CMU/Cornell/GaTech/Maryland/UNC/SFU/etc, 2020
Towards More Robust and Data-Efficient Robotics in the Wild, Intel/NVIDIA, 2019
Deep Geometric Scene Understanding for Self-Driving Cars, CREATE Summer School, 2019
Deep Parameteric Continuous Conv, CVPR, 2018
TorontoCity, ICCV/Endless Summer School, 2017
The Global Patch Collider, CVPR/UofT, 2016
3D Scene Understanding with Knowledge about the World, VALSE Webinar/YorkU, 2016
Lost Shopping! Monocular Localization in Large Indoor Spaces, ICCV, 2015
Holistic 3D Scene Understanding from a Single Monocular Image, CVPR, 2015
Efficient Inference on Continuous MRFs with Polynomial Potentials, NCAP/UofT, 2014

Miscs

I take photos in my spare time:

Pentax Photo Gallery (It's a double-blind peer-review system.)
Flickr (It's more like arXiv, including some of my photo submissions rejected by pentax photo gallery.)

I have a lovely daughter, Dorie.