16-889: Learning for 3D Vision
Spring 2022
[
Home
|
Schedule
|
Assignments
|
Piazza
|
Canvas
|
Office Hour
]
This schedule is preliminary and subject to change as the term evolves.
Lecture
Date
Topic
Suggested Reading
Assignments
WEEK 1
1
Tue 01/18/2022
Introduction (
slides
)
2
Thu 01/20/2022
3D Representations (
slides
)
WEEK 2
3
Tue 01/25/2022
3D Representations: Conversions (
slides
)
4
Thu 01/27/2022
Image Formation (
slides
)
Assignment 1 Out
WEEK 3
5
Tue 02/01/2022
Single-view 3D: History + 2.5D (
slides
)
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
Office Hour
Wed 02/02, 4-5PM
PyTorch3D Tutorial
6
Thu 02/03/2022
Single-view 3D: Objects (
slides
)
Learning a predictable and generative vector representation for objects
Occupancy Networks: Learning 3D Reconstruction in Function Space
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
WEEK 4
7
Tue 02/08/2022
Single-view 3D: Objects and Scenes (
slides
)
Semantic Scene Completion from a Single Depth Image
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene
Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
8
Thu 02/10/2022
3D Prediction without 3D Supervision (
slides
)
Unsupervised Learning of Depth and Ego-Motion from Video
Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency
Learning Category-Specific Mesh Reconstruction from Image Collections
Assignment 1 Due
Assignment 2 Out
WEEK 5
9
Tue 02/15/2022
Single-view 3D: Humans and Animals (
slides
)
SMPL: A Skinned Multi-Person Linear Model
End-to-end Recovery of Human Shape and Pose
10
Thu 02/17/2022
Project Proposal: Selection and Writing (
slides - CMU access only
)
WEEK 6
11
Tue 02/22/2022
Volume Rendering (
slides
)
12
Thu 02/24/2022
Neural Radiance Fields (
slides
)
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Assignment 2 Due
Assignment 3 Out
WEEK 7
13
Tue 03/01/2022
Generalizable Neural Fields (
slides
)
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Learned Initializations for Optimizing Coordinate-Based Neural Representations
pixelNerf: Neural Radiance Fields from One or Few Images
14
Thu 03/03/2022
Rendering Signed Distance Fields (
slides
)
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
Volume Rendering of Neural Implicit Surfaces
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
WEEK 8 - Spring Break
WEEK 9
15
Tue 03/15/2022
Neural Light Fields (
slides
)(Ben)
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Learning Neural Light Fields with Ray-Space Embedding
Light Field Neural Rendering
16
Thu 03/17/2022
Neural Surface Rendering (
slides
) (Jason)
De-rendering the World's Revolutionary Artefacts
NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild
NeROIC: Neural Rendering of Objects from Online Image Collections
Assignment 3 Due
Assignment 4 Out
WEEK 10
17
Tue 03/22/2022
Point Clouds: Classification and Segmentation (
slides
)
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
18
Thu 03/24/2022
Point Clouds: Detection (
slides
)
Frustum PointNets for 3D Object Detection from RGB-D Data
PointPillars: Fast Encoders for Object Detection from Point Clouds
ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes
WEEK 11
19
Tue 03/29/2022
Processing Meshes (
slides
)
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation
MeshCNN: A Network with an Edge
20
Thu 03/31/2022
Shape Abstraction (
slides
)
Learning Shape Abstractions by Assembling Volumetric Primitives
Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids
Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks
Assignment 4 Due
Assignment 5 Out
WEEK 12
21
Tue 04/05/2022
Generative 3D Models (
slides
)
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
Thu 04/07/2022
No Class (Spring carnival)
WEEK 13
22
Tue 04/12/2022
Modeling 3D in Time (
slides
)
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Nerfies: Deformable Neural Radiance Fields
23
Thu 04/14/2022
Applications: Graphics, Robotics, and Beyond (
slides
)
Cognitive Mapping and Planning for Visual Navigation
Where2Act: From Pixels to Actions for Articulated 3D Objects
Assignment 5 Due on 04/15/2022 (Fri), 11:59pm ET
WEEK 14
24
Tue 04/19/2022
Conclusion (
slides
)
25
Thu 04/21/2022
Project Presentations: I
WEEK 15
26
Tue 04/26/2022
Project Presentations: II
27
Thu 04/28/2022
Project Presentations: III
WEEK 16
Fri 05/06/2022
Project Reports Due