16-825: Learning for 3D Vision

Spring 2023

[ Home | Schedule | Assignments | Piazza | Canvas | Office Hour ]

This schedule is preliminary and subject to change as the term evolves.

Lecture	Date	Topic	Suggested Reading	Assignments
WEEK 1
1	Wed 01/18/2023	Introduction (slides)		Assignment 0 Out
WEEK 2
2	Mon 01/23/2023	3D Representations (slides)
3	Wed 01/25/2023	3D Representations: Conversions (slides)		Assignment 0 Due Assignment 1 Out
Office Hour	Wed 01/25/2023 5-6PM	PyTorch3D Tutorial
WEEK 3
4	Mon 01/30/2023	Image Formation (slides)
5	Wed 02/01/2023	Single-view 3D: Overview + 2.5D (slides)	Depth Map Prediction from a Single Image using a Multi-Scale Deep Network Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
WEEK 4
6	Mon 02/06/2023	Single-view 3D: Objects (slides)	Learning a predictable and generative vector representation for objects Occupancy Networks: Learning 3D Reconstruction in Function Space AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
7	Wed 02/08/2023	3D Prediction without 3D Supervision (slides)	Unsupervised Learning of Depth and Ego-Motion from Video Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency Shape and Viewpoint without Keypoints Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstruction	Assignment 1 Due Assignment 2 Out
WEEK 5
8	Mon 02/13/2023	Single-view 3D: Humans, Hands, and Animals (slides)	SMPL: A Skinned Multi-Person Linear Model Embodied Hands: Modeling and Capturing Hands and Bodies Together End-to-end Recovery of Human Shape and Pose DOVE: Learning Deformable 3D Objects by Watching Videos
9	Wed 02/15/2023	Single-view 3D: Scenes (slides) Projects: Selection and Timeline (slides - CMU only)	Semantic Scene Completion from a Single Depth Image Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
WEEK 6
10	Mon 02/20/2023	Volume Rendering (slides)
11	Wed 02/22/2023	Neural Radiance Fields (slides)	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis Plenoxels: Radiance Fields without Neural Networks Instant Neural Graphics Primitives with a Multiresolution Hash Encoding	Assignment 2 Due Assignment 3 Out
WEEK 7
12	Mon 02/27/2023	Rendering Signed Distance Fields (slides)	Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance Volume Rendering of Neural Implicit Surfaces NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
13	Wed 03/01/2023	Neural Surface Rendering (slides)	NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting NeROIC: Neural Rendering of Objects from Online Image Collections	Assignment 4 Out
WEEK 8 - Spring Break
WEEK 9
14	Mon 03/13/2023	Generalizable Neural Fields (slides)	DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation Learned Initializations for Optimizing Coordinate-Based Neural Representations pixelNerf: Neural Radiance Fields from One or Few Images
15	Wed 03/15/2023	Sparse View 3D and View Synthesis (slides)	SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers SparsePose: Sparse-View Camera Pose Regression and Refinement	Assignment 3 Due; Project Proposals Due (Fri 03/17);
WEEK 10
16	Mon 03/20/2023	Neural Light Fields (slides)	Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering Learning Neural Light Fields with Ray-Space Embedding Generalizable Patch-Based Neural Rendering
17	Wed 03/22/2023	Point Cloud Processing I (slides)	PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation Vector Neurons: A General Framework for SO(3)-Equivariant Networks 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks	Assignment 5 Out Assignment 4 Due
WEEK 11
18	Mon 03/27/2023	Point Cloud Processing II (slides)	Point Transformer Frustum PointNets for 3D Object Detection from RGB-D Data Deep Hough Voting for 3D Object Detection in Point Clouds
19	Wed 03/29/2023	Guest Lecture by Neehar Peri on Outdoor Detection (slides)
WEEK 12
20	Mon 04/03/2023	Processing Meshes (slides)	SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation MeshCNN: A Network with an Edge
21	Wed 04/05/2023	Shape Abstraction (slides)	Learning Shape Abstractions by Assembling Volumetric Primitives Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks	Assignment 5 Due
WEEK 13
22	Mon 04/10/2023	Generative 3D Models (slides)	Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis DreamFusion: Text-to-3D using 2D Diffusion
23	Wed 04/12/2023	Modeling 3D in Time (slides)	Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes Nerfies: Deformable Neural Radiance Fields
WEEK 14
24	Mon 04/17/2023	Guest Lecture by Devendra Chaplot on 3D for Navigation (slides)
25	Wed 04/19/2023	Conclusion (slides)
WEEK 15
26	Mon 04/24/2023	Poster Presentations: I
27	Wed 04/26/2023	Poster Presentations: II
WEEK 16
	Wed 05/03/2023			Project Reports and Recorded Presentations Due