Ambient-robust Inverse Rendering using Active RGB-NIR Imaging

Overview

Active RGB-NIR imaging for stable inverse rendering

Existing inverse rendering methods can be sensitive to uncontrolled ambient lighting. This project uses imperceptible NIR flash illumination to obtain stable point-light shading, while RGB images preserve visible-spectrum appearance for downstream rendering.

The method combines multi-view RGB observations, active NIR flash images, and an RGB-NIR BRDF model to recover geometry, roughness, metallic parameters, diffuse albedo, and environment illumination.

Figure 1 teaser showing the mobile RGB-NIR imaging system, captured observations, reconstructed geometry and reflectance, and relighting results. — Figure 1: overview of acquisition, reconstruction, and relighting.

Pipeline

RGB-NIR inverse rendering pipeline

The pipeline proceeds in three stages: RGB-based geometry initialization, NIR flash inverse rendering for robust geometry and reflectance refinement, and RGB environment inverse rendering for visible diffuse albedo and illumination recovery.

Stage 1

RGB initialization

Initialize object geometry from multi-view RGB images under natural ambient lighting.

Stage 2

NIR flash rendering

Use flash-only NIR measurements to refine geometry and estimate NIR reflectance.

Stage 3

RGB environment rendering

Recover RGB diffuse albedo and ambient environment maps using the refined reconstruction.

System

Imaging system design

The acquisition platform combines a mobile base, robotic arm, pixel-aligned RGB-NIR camera, and synchronized NIR flash. The system scans objects from dense viewpoints while keeping NIR illumination largely invisible to human observers.

Dataset

Acquired RGB-NIR inverse rendering dataset

The dataset contains multi-view RGB-NIR image pairs with active NIR flash under multiple ambient illumination conditions, together with a synthetic counterpart rendered to match the acquisition setup.

Real-world RGB/NIR dataset gallery

Four objects and fourteen ambient scenes, shown as pixel-aligned RGB/NIR preview pairs.

14 assets

Synthetic RGB/NIR dataset gallery

Three synthetic objects and four ambient scenes, shown as RGB/NIR preview pairs.

12 assets

Results

Inverse rendering reconstruction results

The following synchronized clips show the recovered material components and environment-conditioned relighting for three synthetic objects and three real-world objects. Each scene is a 12-second result sequence that loops continuously like an animated figure.

Synthetic

Synthetic inverse rendering reconstruction

Controlled synthetic scenes expose the recovered reflectance decomposition and relighting behavior across different object geometries.

Real world

Real-world inverse rendering reconstruction

Real captures demonstrate reconstruction and relighting on objects with measured RGB-NIR observations under changing ambient environments.

Real world

Real-world point-light relighting

Reconstructed real-world objects rendered under changing point-light illumination to visualize the recovered appearance and material response.

Citation

BibTeX

@inproceedings{chung2026ambient,
  title     = {Ambient-robust Inverse Rendering using Active RGB-NIR Imaging},
  author    = {Chung, Hoon-Gyu and Kim, Jinnyeong and Kang, Hyunwoo and Baek, Seung-Hwan},
  booktitle = {SIGGRAPH Conference Papers},
  year      = {2026},
  doi       = {10.1145/3799902.3811078}
}