NeuBTF: Neural Fields for BTF Encoding and Transfer

Teaser

teaser

On the left, renders generated with NeuBTF materials. On their right, a slice of the input training BTF and the guidance image onto which we propagate the BTF measurements.

Abstract

Neural material representations are becoming a popular way to represent materials for rendering. They are more expressive than analytic models and occupy less memory than tabulated BTFs. However, existing neural materials are immutable, meaning that their output for a certain query of UVs, camera, and light vector is fixed once they are trained. While this is practical when there is no need to edit the material, it can become very limiting when the fragment of the material used for training is too small or not tileable, which frequently happens when the material has been captured with a gonioreflectometer. In this paper, we propose a novel neural material representation which jointly tackles the problems of BTF compression, tiling, and extrapolation. At test time, our method uses a guidance image as input to condition the neural BTF to the structural features of this input image. Then, the neural BTF can be queried as a regular BTF using UVs, camera, and light vectors. Every component in our framework is purposefully designed to maximize BTF encoding quality at minimal parameter count and computational complexity, achieving competitive compression rates compared with previous work. We demonstrate the results of our method on a variety of synthetic and captured materials, showing its generality and capacity to learn to represent many optical properties.

Structural Editions

In this example, we propagate BTF measurements of a leather material to the structure represented in a different guidance image. In this video, we show a video where we change the light and camera angles of the propagated neural material. Please visit this link for a higher resolution video.
The propagation works with vector images too, as we show in this example. Please visit this link for a higher resolution video.
In this example, we propagate the BTF values of a metallic fabric to the structure of a floral fabric, using a high resolution guidance image. The propagated neural material has the same resolution as the guidance image. Please visit this link for a higher resolution video.

Citation 

@article{rodriguezpardo2023NeuBTF,
    title = {NeuBTF: Neural fields for BTF encoding and transfer},
    journal = {Computers & Graphics},
    volume = {114},
    pages = {239-246},
    year = {2023},
    issn = {0097-8493},
    doi = {https://doi.org/10.1016/j.cag.2023.06.018},
    url = {https://www.sciencedirect.com/science/article/pii/S0097849323001140},
    author = {Carlos Rodriguez-Pardo and Konstantinos Kazatzis and Jorge Lopez-Moreno and Elena Garces}
    }

Acknowledgements

Elena Garces was partially supported by a Juan de la Cierva - Incorporacion Fellowship (IJC2020-044192-I). This publication is part of the project TaiLOR, CPP2021-008842 funded by MCIN/AEI/10.13039/501100011033 and the NextGenerationEU/PRTR programs.