Self-Supervised Object Goal Navigation with In-Situ Finetuning

1 Mins read

A household robot should be able to navigate to target locations without requiring users to first annotate everything in their home. Current approaches to this object navigation challenge do not test on real robots and rely on expensive semantically labeled 3D meshes. In this work, our aim is an agent that builds self-supervised models of the world via exploration, the same as a child might. We propose an end-to-end self-supervised embodied agent that leverages exploration to train a semantic segmentation model of 3D objects, and uses those representations to learn an object navigation policy purely from self-labeled 3D meshes. The key insight is that embodied agents can leverage location consistency as a supervision signal — collecting images from different views/angles and applying contrastive learning to fine-tune a semantic segmentation model. In our experiments, we observe that our framework performs better than other self-supervised baselines and competitively with supervised baselines, in both simulation and when deployed in real houses.

Source link

Related posts

Researchers from the University of Maryland Introduce GenQA Instruction Dataset: Automating Large-Scale Instruction Dataset Generation for AI Model Finetuning and Diversity Enhancement

3 Mins read
Natural language processing has greatly improved language model finetuning. This process involves refining AI models to perform specific tasks more effectively by…

APEER: A Novel Automatic Prompt Engineering Algorithm for Passage Relevance Ranking

2 Mins read
A significant challenge in the field of Information Retrieval (IR) using Large Language Models (LLMs) is the heavy reliance on human-crafted prompts…

Cephalo: A Series of Open-Source Multimodal Vision Large Language Models (V-LLMs) Specifically in the Context of Bio-Inspired Design

3 Mins read
Materials science focuses on studying and developing materials with specific properties and applications. Researchers in this field aim to understand the structure,…



Leave a Reply

Your email address will not be published. Required fields are marked *