Illumination Browser: An intuitive representation
for radiance map databases

Abstract

---

Radiance maps (RM) are used for capturing the lighting properties of real-world environments. Databases of RMs are useful for various rendering applications such as look development, live action composition, mixed reality, and machine learning. Such databases are not useful if they cannot be organized in a meaningful way. To address this, we introduce the illumination space, a feature space that arranges RM databases based on illumination properties. Our method is motivated by how the RM illuminates the scene as opposed to describing the textural content of the RM. We avoid manual labeling by automatically extracting features from an RM that provides a concise and semantically meaningful representation of its typical lighting effects. We also introduce ‘Illumination Browser’, a user interface (UI) that visualizes the illumination space alongside a real-time preview renderer to enable intuitive browsing for artists. This is made possible with the following contributions: a method to automatically extract a small set of dominant and ambient lighting properties from RMs, a low- dimensional (5D) light feature vector summarizing these properties to form the illumination space, and a UI that effectively utilizes the illumination space.

Overview

---

To develop a low-dimensional representation, we follow an empirical approach of making careful design decisions and observing their effects in large RM databases. Numerous iterations of this design process lead us to five intuitive lighting properties of RMs, split into two categories, dominant and ambient lighting: the (1) size, (2) elevation angle, and (3) azimuth angle of the RM’s most dominant area light sources, as well as the diffuse (4) hue and (5) saturation of the RM. The dominant lighting property set (1, 2, 3) can occur multiple times for a given RM (one set for each bright light in the RM). To create a low-dimensional feature space, these properties are converted into a strictly 5D feature vector. This defines the ‘illumination space’ - a semantically meaningful, low-dimensional feature space. This feature space is embedded in our proposed ‘Illumination Browser’ interface to enable browsing RM databases.

Our contributions are summarized as follows:

• We define and extract a small set of dominant and ambient lighting properties from RMs using the DLM.
• From the extracted lighting properties, we introduce a 5D light feature vector that encodes RMs in a low-dimensional, semantically meaningful feature space — the ‘illumination space’.
• ‘Illumination Browser’ is presented, an interactive visualization for browsing an RM database. Alongside browsing, we also demonstrate searching the RM database by using an RM to obtain other similar RMs.

Methodology

---

Before introducing the 5D light feature vector, we first define how we obtain dominant lighting properties, which we then use to encode into the dominant light features. A dominant light in an RM is a local region of the lighting sphere Ω ⊂ S2 with the property that the region has a substantial impact on high-frequency lighting effects in a scene, including highlights and cast shadows. An RM may have multiple dominant lights, and we use Ωj to denote the jth dominant light. We assume light Ωj is given through a light detection scheme; we use statistics-based thresholding. The remainder of this section introduces the dominant light model (DLM) for summarizing the lighting effects of each Ωj, and the procedure of fitting a DLM to each Ωj (resulting in a DLM for each dominant light source in an RM).

We denote the elevation, azimuth, and size of a given dominant light as θ, φ, and σ respectively. θ and φ are the angular components of z, while σ is the average of σx and σy. An RM may contain multiple dominant lights, thus an RM may contain multiple sets of these properties. To obtain a low dimensional embedding, we describe a way of aggregating these dominant lighting properties into strictly three dominant light features: Fσ , Fθ , and Fc . The first two features describe the size and elevation, whereas the third takes into account the impact of multiple dominant lights, which describe as ‘‘complexity’’ or ‘‘angular spread’’. In addition to these dominant light features, we define the features Fs and Fh that describe the hue and saturation of the RM’s diffuse shading.

Results

---

Feature Fσ which shows the transition from hard to soft shadows. In the RM, this corresponds to the size of the dominant light source, transitioning from small to large. RMs with similar dominant light elevation Fθ are chosen.

Feature Fθ which shows the transition from long to short shadows. In the RM, this corresponds to the elevation of the dominant light source, transitioning from low to high. RMs with similar shadow softness Fσ are chosen.

Feature Fh and Fs are the color features. It is common to find RMs with either red (indoor, sunset), blue (clear sunny sky) or desaturated colors (indoor, overcast). This corresponds to a cool/warm color distribution. We visualize the transition from high saturation blue, desaturated, and high saturation red in one dimension (left to right).

Feature Fc which shows the transition from low to high azimuthal spread. The center row shows how it corresponds to the number of detected dominant lights (red dots have been added to the center of each detected dominant light source). The rendered images have been normalized to show the details.

Results

---

We had nine artists use ‘Illumination Browser’ to search for RMs. They were told to first familiarize themselves with its features. Following this, they were tasked to illuminate a synthetic object (which is composited into a photograph) with an RM obtained from the database (no time limit was given, but on average users took approximately 2 min per task). The artists did this for nine different photographs, covering a wide range of illumination conditions (see the supplementary materials). After this, the artists were asked to qualitatively evaluate each feature individually on how well it met their expectations on a 5-point Likert item (from −2, to 2). We found that the features visualized directly on the scatter plots (elevation, size, hue, and saturation) were very intuitive with a median score of 2 with little deviation. Users commented that the azimuthal spread was useful but not as intuitive due to few samples being returned during the browsing and difficulty seeing the clustering in the third dimension (median score of 1).

Presentation Video

---