This schedule is preliminary and subject to change as the term evolves.
Lecture
Date
Topic
Suggested Reading
Assignments
WEEK 1
1
Wed 01/17/2024
Introduction
Assignment 0 Out
WEEK 2
2
Mon 01/22/2024
3D Representations
3
Wed 01/24/2024
3D Representations: Conversions
Assignment 0 Due
Assignment 1 Out
3
Fri 01/26/2024
Pytorch3D Tutorial (NSH 3002, 4-5pm)
colab notebook
WEEK 3
4
Mon 01/29/2024
Image Formation
5
Wed 01/31/2024
Single-view 3D: History + Depth Prediction
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
WEEK 4
6
Mon 02/05/2024
Single-view 3D: Objects
Learning a predictable and generative vector representation for objects
Occupancy Networks: Learning 3D Reconstruction in Function Space
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
7
Wed 02/07/2024
Single-view 3D: Objects (II)
Projects: Selection and Timeline
Learning Category-Specific Mesh Reconstruction from Image Collections
Accelerating 3D Deep Learning with PyTorch3D
Assignment 1 Due
Assignment 2 Out
WEEK 5
8
Mon 02/12/2024
Single-view 3D: Humans, Hands, and Animals
SMPL: A Skinned Multi-Person Linear Model
Embodied Hands: Modeling and Capturing Hands and Bodies Together
End-to-end Recovery of Human Shape and Pose
DOVE: Learning Deformable 3D Objects by Watching Videos
9
Wed 02/14/2024
Volume Rendering
Volume Rendering Digest
WEEK 6
10
Mon 02/19/2024
Neural Radiance Fields
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Plenoxels: Radiance Fields without Neural Networks
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
11
Wed 02/21/2024
(Neural) SDF Rendering
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
Volume Rendering of Neural Implicit Surfaces
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
Assignment 2 Due
Assignment 3 Out
WEEK 7
12
Mon 02/26/2024
Neural Surface Rendering
NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild
PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting
13
Wed 02/28/2024
Differentiable Primitive (e.g. Gaussians, Triangles) Rendering
Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Project Proposals Due
WEEK 8 - Spring Break
WEEK 9
14
Mon 03/11/2024
Generalizable Neural Fields
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Learned Initializations for Optimizing Coordinate-Based Neural Representations
pixelNerf: Neural Radiance Fields from One or Few Images
15
Wed 03/13/2024
Sparse View 3D and View Synthesis
ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers
SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction
ReconFusion: 3D Reconstruction with Diffusion Priors
Assignment 3 Due
Assignment 4 Out
WEEK 10
16
Mon 03/18/2024
Distilling 3D from 2D Diffusion
DreamFusion: Text-to-3D using 2D Diffusion
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
Zero-1-to-3: Zero-shot One Image to 3D Object
17
Wed 03/20/2024
Generative 3D Modeling
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
WEEK 11
18
Mon 03/25/2024
Point Cloud Processing: Classification & Segmentation
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Point Transformer
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
19
Wed 03/27/2024
Point Cloud Processing: Detection
Frustum PointNets for 3D Object Detection from RGB-D Data
Deep Hough Voting for 3D Object Detection in Point Clouds
Assignment 5 Out
WEEK 12
20
Mon 04/01/2024
Guest Lecture by
Jason Zhang
(
Learning-based Pose Estimation
)
Assignment 4 Due
21
Wed 04/03/2024
Processing Meshes
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation
MeshCNN: A Network with an Edge
WEEK 13
22
Mon 04/08/2024
Modeling 3D in Time
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Nerfies: Deformable Neural Radiance Fields
23
Wed 04/10/2024
Guest Lecture by
Neehar Peri
(
Outdoor 3D Detection
)
Assignment 5 Due (Fri)
WEEK 14
24
Mon 04/15/2024
Conclusion
25
Wed 04/17/2024
Guest Lecture by
Vincent Sitzmann
(Neural Fields for Robotics)
WEEK 15
26
Mon 04/22/2024
Poster Presentations: I
27
Wed 04/24/2024
Poster Presentations: II
WEEK 16
Wed 05/01/2024
Project Reports and Recorded Presentations Due