AI

DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models

1 Mins read

Generating high-quality 3D content requires models capable of learning robust distributions of complex scenes and the real-world objects within them. Recent Gaussian-based 3D reconstruction techniques have achieved impressive results in recovering high-fidelity 3D assets from sparse input images by predicting 3D Gaussians in a feed-forward manner. However, these techniques often lack the extensive priors and expressiveness offered by Diffusion Models. On the other hand, 2D Diffusion Models, which have been successfully applied to denoise multiview images, show potential for generating a wide range of photorealistic 3D outputs but still fall short on explicit 3D priors and consistency. In this work, we aim to bridge these two approaches by introducing DSplats, a novel method that directly denoises multiview images using Gaussian Splat-based Reconstructors to produce a diverse array of realistic 3D assets. To harness the extensive priors of 2D Diffusion Models, we incorporate a pretrained Latent Diffusion Model into the reconstructor backbone to predict a set of 3D Gaussians. Additionally, the explicit 3D representation embedded in the denoising network provides a strong inductive bias, ensuring geometrically consistent novel view generation. Our qualitative and quantitative experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction. When evaluated on the Google Scanned Objects dataset, DSplats achieves a PSNR of 20.38, an SSIM of 0.842, and an LPIPS of 0.109.


Source link

Related posts
AI

Hugging Face Releases OlympicCoder: A Series of Open Reasoning AI Models that can Solve Olympiad-Level Programming Problems

3 Mins read
In the realm of competitive programming, both human participants and artificial intelligence systems encounter a set of unique challenges. Many existing code…
AI

From Genes to Genius: Evolving Large Language Models with Nature’s Blueprint

3 Mins read
Large language models (LLMs) have transformed artificial intelligence with their superior performance on various tasks, including natural language understanding and complex reasoning….
AI

Limbic AI's Generative AI–Enabled Therapy Support Tool Improves Cognitive Behavioral Therapy Outcomes

2 Mins read
Recent advancements in generative AI are creating exciting new possibilities in healthcare, especially within mental health services, where patient engagement is often…

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *