Dual Photography Lab
An interactive application implementing the dual photography technique — reconstructing how a scene looks from a projector's point of view using light transport matrices, SVD decomposition, and Helmholtz reciprocity.
Business Context
Dual photography is a computational imaging technique that reconstructs how a scene looks from a projector's point of view using only camera observations. The transport matrix T encodes every light interaction between projector and camera through scene geometry, reflectance, and occlusion. By measuring T with structured illumination patterns and applying Helmholtz reciprocity, we can computationally reconstruct viewpoints that were never physically occupied — seeing from where we illuminate.
Strategic Value
The application provides a complete interactive environment for exploring dual photography: 6 ray-cast scene types (flat, textured, concave, convex, multi-object, occluding), 10 illumination patterns, SVD decomposition with interactive singular value exploration revealing optical complexity, compressed sensing with Bernoulli/Hadamard matrices for efficient acquisition, and an optional webcam capture mode for real-world experiments. The SVD analysis T = UΣVᵀ connects imaging to linear algebra in a tangible way — the singular value spectrum reveals how many independent optical modes a scene supports.
The Challenge
Dual photography involves recovering the view from a projector's perspective using only camera observations. The transport matrix T encodes how every projector pixel contributes to every camera pixel through scene geometry and materials — but working with it requires efficient decomposition and reconstruction methods.
Our Approach
Ray-cast simulation with 6 scene types, SVD analysis (T = U·Σ·Vᵀ) to reveal optical complexity, 10 illumination patterns, optional webcam capture mode. Backend implements Lambertian BRDF models with NumPy/SciPy and Dash/Plotly visualization frontend.
Key Performance Indicators
| KPI | Baseline | Result | Impact |
|---|---|---|---|
| Scene Types | Static images | 6 interactive ray-cast scenes | Interactive optical exploration |
| Illumination Patterns | Single lighting | 10 structured patterns | Complete transport characterization |
Architecture
dual photography
Seeing from Where You Illuminate
Imagine reversing the roles of projector and camera. A projector sends structured light into a scene; a camera records what bounces back. The transport matrix T encodes every interaction — how each projector pixel contributes to each camera pixel through scene geometry, reflectance, and occlusion. If you can measure T, you can reconstruct what the scene looks like from the projector’s perspective — using Helmholtz reciprocity, the dual image is simply Tᵀ applied to a virtual illumination.
This is dual photography: computational reconstruction of a viewpoint you never physically occupied.
Why It’s Interesting
The SVD decomposition T = UΣVᵀ reveals the scene’s optical complexity. A flat white wall produces a simple singular value spectrum with rapid decay — one dominant mode. A complex textured scene with occlusions has many significant singular values — rich optical structure. This connects imaging to linear algebra in a way that makes abstract mathematical concepts tangible.
With structured illumination patterns (Bernoulli or Hadamard matrices), compressed sensing theory applies: the dual image can be recovered with far fewer measurements than the full transport matrix requires. And the condition number of T determines reconstruction quality — ill-conditioned matrices from complex scenes need regularization (Tikhonov or truncated SVD) to produce stable results.
The application implements 6 ray-cast scene types (flat, textured, concave, convex, multi-object, occluding), 10 illumination patterns, interactive SVD exploration, and an optional webcam capture mode for real-world experiments.
Technology Stack
Application Screenshots
Technical Diagrams
dual helmholtz