AI

Transfer Learning in Scalable Graph Neural Network for Improved Physical Simulation

1 Mins read

In recent years, graph neural network (GNN) based models showed promising results in simulating complex physical systems. However, training dedicated graph network simulator can be costly, as most models are confined to fully supervised training. Extensive data generated from traditional simulators is required to train the model. It remained unexplored how transfer learning could be applied to improve the model performance and training efficiency. In this work, we introduce a pretraining and transfer learning paradigm for graph network simulator.
First, We proposed the scalable graph U-net (SGUNet). By incorporating an innovative depth-first search (DFS) pooling, the SGUNet is configurable to adaptable different mesh size and resolutions for different simulation tasks. To enable the transfer learning between different configured SGUNet, we propose a set of mapping functions to align the parameters between pretrained model and target model. An extra normalization term is also added into loss to constrain the similarity between the pretrained weights and target model weights for better generalization performance. Then we created a dataset for pretraining the simulators. It includes 20,000 physical simulations with 3D shapes randomly selected from the open source A Big CAD (ABC) datasets. We demonstrate that with our proposed transfer learning methods, model fine-tuned with a small portion of the training data could reach even better performance compared with the one trained from scratch. On 2D Deformable Plate, our pretrained model fine-tuned on 1/16 of the training data could achieve 11.05% improvement compared to model trained from scratch.


Source link

Related posts
AI

Building a Legal AI Chatbot: A Step-by-Step Guide Using bigscience/T0pp LLM, Open-Source NLP Models, Streamlit, PyTorch, and Hugging Face Transformers

3 Mins read
In this tutorial, we will build an efficient Legal AI CHatbot using open-source tools. It provides a step-by-step guide to creating a…
AI

Optimizing Training Data Allocation Between Supervised and Preference Finetuning in Large Language Models

3 Mins read
Large Language Models (LLMs) face significant challenges in optimizing their post-training methods, particularly in balancing Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL)…
AI

This AI Paper from Weco AI Introduces AIDE: A Tree-Search-Based AI Agent for Automating Machine Learning Engineering

3 Mins read
The development of high-performing machine learning models remains a time-consuming and resource-intensive process. Engineers and researchers spend significant time fine-tuning models, optimizing…

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *