RealMat: Realistic Materials with Diffusion and Reinforcement Learning

Generative models for high-quality materials are particularly desirable to make 3D content authoring accessible. Alternatively, recent work used a small dataset of real flash photographs to guarantee realism, however such data is limited in scale and diversity. In RealMat, we first finetune a pretrained Stable Diffusion XL (SDXL) with synthetic material maps arranged in $2 \times 2$ grids.