Image-Based Modeling, Rendering, and Lighting

SIGGRAPH 99 Course #39

Tuesday, August 10
Room 152, Los Angeles Convention Center
8:30am - 5:00pm

Christoph Bregler
Michael F. Cohen
Paul Debevec
UC Berkeley
Leonard McMillan
François X. Sillion
Richard Szeliski

Official SIGGRAPH web site for this course

See also Monday's SIGGRAPH 99 course on 3D Photography, organized by Brian Curless and Steve Seitz

Course Abstract

Image-based modeling and rendering differs from traditional graphics in that both the geometry and appearance of the scene are derived from real photographs. The techniques often allow for shorter modeling times, faster rendering speeds, and unprecedented levels of photorealism. This course explains and demonstrates a variety of ways of turning images into models and then back into renderings, including movie maps, panoramas, image warping, photogrammetry, light fields, and 3D scanning.


Topics include: computer vision and how it relates to image-based rendering techniques, and how to apply the techniques to animation and 3D navigation. The course also shows how global illumination techniques allow photorealistic modifications of image-based models and illustrates results from recent research and creative applications.

Table of Contents

1. Introduction and Overview (8:30am, Debevec)
Notes: Abstract, Speaker Bios, and Schedule
Notes: What is Image-based Modeling and Rendering? (Debevec)
Presentation: Introduction to Image-Based Modeling, Rendering, and Lighting (Debevec)

2. Fundamentals of Image Formation and Re-use (8:50, Sillion)
Notes: Fundamentals of image formation and re-use (Sillion)
Presentation: Fundamentals of image formation and re-use (Sillion)
Paper: Rendering with Coherent Layers
Jed Lengyel and John Snyder, Proc. SIGGRAPH 97
Paper: Multi-layered imposters for accelerated rendering
Xavier Decoret, Gernot Schaufler, Francois Sillion, and Julie Dorsey, Proc. Eurographics
Paper: A Three Dimensional Image Cache for Virtual Reality
Gernot Schaufler and Wolfgang Sturzlinger, Proc. Eurographics 1996

3. Determining Geometry from Images (10:15am, Szeliski)
Presentation: Determining Geometry from Images (Szeliski)
Paper: From images to models (and beyond): a personal retrospective
Richard Szeliski, Proc. Vision Interface 1997
Paper: Modeling and Rendering Architecture from Photographs
Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik, Proc. SIGGRAPH 96

4. 2D and 3D Image Warping (11:00, McMillan)
Notes: Image-Based Rendering using Image Warping [Part 2] (McMillan)
Presentation: Image-Based Rendering using Image Warping
Paper: Plenoptic Modeling
Leonard McMillan and Gary Bishop, Proc. SIGGRAPH 95
Paper: View Morphing
Steven M. Seitz and Charles R. Dyer, Proc. SIGGRAPH 96
Lunch (noon - 1:30pm)

5. LDI and Lightfield/Lumigraph representations (1:30, Cohen)
Presentation: Image or Object?
Paper: Layered Depth Images
Jonathan Shade, Steven Gortler, Li-wei Hey, and Richard Szeliski, Proc. SIGGRAPH 97
Paper: Light Field Rendering
Marc Levoy and Pat Hanrahan, Proc. SIGGRAPH 96
Paper: The Lumigraph
S.J. Gortler, R. Grzeszczuk, R. Szeliski, an M.F. Cohen, Proc. SIGGRAPH 96
Paper: Efficient View-Dependent Image-Based REndering with Projective Texture-Mapping
Paul Debevec, George Borshukov, and Yizhou Yu, 9th Eurographics Rendering Workshop, 1998

6. Image-Based Lighting (2:20, Debevec)
Presentation: Image-Based Lighting (Debevec)
Paper: Recovering High-Dynamic Range Radiance Maps from Photographs
Paul E. Debevec and Jitendra Malik, Proc. SIGGRAPH 97
Paper: Rendering Synthetic Objects into Real Scenes
Paul Debevec, Proc. SIGGRAPH 98

7. Applications of IBMR in Human Animation (3:15, Bregler)
Notes: Video Based Animation Techniques for Human Motion (Bregler)
Presentation: IBMR Techniques for Animating People
Paper: Video Rewrite: Driving Visual Speech with Audio
Christoph Bregler, Michele Covell, Malcolm Slaney, Proc. SIGGRAPH 97
See the movies ELLEN*.MOV and JFK*.MOV in the MOVIES/ Directory
Paper: Synthesizing Realistic Facial Expressions from Photographs
Frederic Pighin, Jamie Hecker, Dani Lischinski, Richard Szeliski, and David H. Salesin, Proc. SIGGRAPH 98
Paper: Making Faces
Brian Guenter, Cindy Grimm, Daniel Wood, Henrique Malvar, and Fredrick Pighin, Proc. SIGGRAPH 98
Paper: Video Motion Capture
Christoph Bregler and Jitendra Malik

8. Applications of IBMR in Art and Cinema, (4:05, Debevec)
Presentation: Applications of IBMR in Art and Cinema (Debevec)
Notes: Rouen Revisited
Golan Levin and Paul Debevec
Web Page: Selected projects from Michael Naimark
9. Questions and Dialog, (4:40, All)

Other Resources:

Special thanks to all the course speakers and to Tim Hawkins, Stephen Spencer, and Barb Helfer for their help in preparing these notes.

Comments and Questions: Paul E. Debevec /