Self-Supervised Object Goal Navigation with In-Situ Finetuning

1 Mins read

A household robot should be able to navigate to target locations without requiring users to first annotate everything in their home. Current approaches to this object navigation challenge do not test on real robots and rely on expensive semantically labeled 3D meshes. In this work, our aim is an agent that builds self-supervised models of the world via exploration, the same as a child might. We propose an end-to-end self-supervised embodied agent that leverages exploration to train a semantic segmentation model of 3D objects, and uses those representations to learn an object navigation policy purely from self-labeled 3D meshes. The key insight is that embodied agents can leverage location consistency as a supervision signal — collecting images from different views/angles and applying contrastive learning to fine-tune a semantic segmentation model. In our experiments, we observe that our framework performs better than other self-supervised baselines and competitively with supervised baselines, in both simulation and when deployed in real houses.

Source link

Related posts

Re-imagining the opera of the future | MIT News

6 Mins read
In the mid-1980s, composer Tod Machover came across a copy of Philip K. Dick’s science fiction novel “VALIS” in a Parisian bookstore….

A generative AI-powered solution on Amazon SageMaker to help Amazon EU Design and Construction

11 Mins read
The Amazon EU Design and Construction (Amazon D&C) team is the engineering team designing and constructing Amazon Warehouses across Europe and the…

MDaudit uses AI to improve revenue outcomes for healthcare customers

3 Mins read
MDaudit provides a cloud-based billing compliance and revenue integrity software as a service (SaaS) platform to more than 70,000 healthcare providers and…



Leave a Reply

Your email address will not be published. Required fields are marked *