The JAX3D library provides tools for volumetric rendering, novel-view synthesis, and 3D reconstruction. The framework utilizes JAX and Flax to implement a hierarchical NeRF model that separates scene geometry from view-dependent appearance.
The implementation includes a synthetic dataset generator that creates scenes with spheres and a patterned floor. The system uses pinhole camera intrinsics and orbit-based placement to render consistent RGB, depth, and opacity values across multiple views.
The NeRF model employs sinusoidal positional encoding for spatial coordinates and viewing directions. A deep MLP with skip connections predicts volumetric density, while RGB values are conditioned on the viewing direction to capture specular effects.
The hierarchical rendering pipeline performs a coarse pass followed by importance sampling. It uses the coarse rendering weights to generate a piecewise-constant probability distribution, allowing the model to sample additional fine points in regions of high contribution.
