16-889: Learning for 3D Vision

Spring 2022

[ Home | Schedule | Assignments | Piazza | Canvas | Office Hour ]

This schedule is preliminary and subject to change as the term evolves.

Lecture	Date	Topic	Suggested Reading	Assignments
WEEK 1
1	Tue 01/18/2022	Introduction (slides)
2	Thu 01/20/2022	3D Representations (slides)
WEEK 2
3	Tue 01/25/2022	3D Representations: Conversions (slides)
4	Thu 01/27/2022	Image Formation (slides)		Assignment 1 Out
WEEK 3
5	Tue 02/01/2022	Single-view 3D: History + 2.5D (slides)	Depth Map Prediction from a Single Image using a Multi-Scale Deep Network Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
Office Hour	Wed 02/02, 4-5PM	PyTorch3D Tutorial
6	Thu 02/03/2022	Single-view 3D: Objects (slides)	Learning a predictable and generative vector representation for objects Occupancy Networks: Learning 3D Reconstruction in Function Space AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
WEEK 4
7	Tue 02/08/2022	Single-view 3D: Objects and Scenes (slides)	Semantic Scene Completion from a Single Depth Image Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
8	Thu 02/10/2022	3D Prediction without 3D Supervision (slides)	Unsupervised Learning of Depth and Ego-Motion from Video Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency Learning Category-Specific Mesh Reconstruction from Image Collections	Assignment 1 Due Assignment 2 Out
WEEK 5
9	Tue 02/15/2022	Single-view 3D: Humans and Animals (slides)	SMPL: A Skinned Multi-Person Linear Model End-to-end Recovery of Human Shape and Pose
10	Thu 02/17/2022	Project Proposal: Selection and Writing (slides - CMU access only)
WEEK 6
11	Tue 02/22/2022	Volume Rendering (slides)
12	Thu 02/24/2022	Neural Radiance Fields (slides)	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis	Assignment 2 Due Assignment 3 Out
WEEK 7
13	Tue 03/01/2022	Generalizable Neural Fields (slides)	DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation Learned Initializations for Optimizing Coordinate-Based Neural Representations pixelNerf: Neural Radiance Fields from One or Few Images
14	Thu 03/03/2022	Rendering Signed Distance Fields (slides)	Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance Volume Rendering of Neural Implicit Surfaces NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
WEEK 8 - Spring Break
WEEK 9
15	Tue 03/15/2022	Neural Light Fields (slides)(Ben)	Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering Learning Neural Light Fields with Ray-Space Embedding Light Field Neural Rendering
16	Thu 03/17/2022	Neural Surface Rendering (slides) (Jason)	De-rendering the World's Revolutionary Artefacts NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild NeROIC: Neural Rendering of Objects from Online Image Collections	Assignment 3 Due Assignment 4 Out
WEEK 10
17	Tue 03/22/2022	Point Clouds: Classification and Segmentation (slides)	PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation Vector Neurons: A General Framework for SO(3)-Equivariant Networks 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
18	Thu 03/24/2022	Point Clouds: Detection (slides)	Frustum PointNets for 3D Object Detection from RGB-D Data PointPillars: Fast Encoders for Object Detection from Point Clouds ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes
WEEK 11
19	Tue 03/29/2022	Processing Meshes (slides)	SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation MeshCNN: A Network with an Edge
20	Thu 03/31/2022	Shape Abstraction (slides)	Learning Shape Abstractions by Assembling Volumetric Primitives Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks	Assignment 4 Due Assignment 5 Out
WEEK 12
21	Tue 04/05/2022	Generative 3D Models (slides)	Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
	Thu 04/07/2022	No Class (Spring carnival)
WEEK 13
22	Tue 04/12/2022	Modeling 3D in Time (slides)	Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes Nerfies: Deformable Neural Radiance Fields
23	Thu 04/14/2022	Applications: Graphics, Robotics, and Beyond (slides)	Cognitive Mapping and Planning for Visual Navigation Where2Act: From Pixels to Actions for Articulated 3D Objects	Assignment 5 Due on 04/15/2022 (Fri), 11:59pm ET
WEEK 14
24	Tue 04/19/2022	Conclusion (slides)
25	Thu 04/21/2022	Project Presentations: I
WEEK 15
26	Tue 04/26/2022	Project Presentations: II
27	Thu 04/28/2022	Project Presentations: III
WEEK 16
	Fri 05/06/2022			Project Reports Due