Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement

1 Mins read

We propose Dataset Reinforcement, a strategy to improve a dataset once such that the accuracy of any model architecture trained on the reinforced dataset is improved at no additional training cost for users. We propose a Dataset Reinforcement strategy based on data augmentation and knowledge distillation. Our generic strategy is designed based on extensive analysis across CNN- and transformer-based models and performing large-scale study of distillation with state-of-the-art models with various data augmentations. We create a reinforced version of the ImageNet training dataset, called ImageNet+, as well as reinforced datasets CIFAR-100+, Flowers-102+, and Food-101+. Models trained with ImageNet+ are more accurate, robust, and calibrated, and transfer well to downstream tasks (e.g., segmentation and detection). As an example, the accuracy of ResNet-50 improves by 1.7% on the ImageNet validation set, 3.5% on ImageNetV2, and 10.0% on ImageNet-R. Expected Calibration Error (ECE) on the ImageNet validation set is also reduced by 9.9%. Using this backbone with Mask-RCNN for object detection on MS-COCO, the mean average precision improves by 0.8%. We reach similar gains for MobileNets, ViTs, and Swin-Transformers. For MobileNetV3 and Swin-Tiny, we observe significant improvements on ImageNet-R/A/C of up to 20% improved robustness. Models pretrained on ImageNet+ and fine-tuned on CIFAR-100+, Flowers-102+, and Food-101+, reach up to 3.4% improved accuracy.

Source link

Related posts

Improving your LLMs with RLHF on Amazon SageMaker

10 Mins read
Reinforcement Learning from Human Feedback (RLHF) is recognized as the industry standard technique for ensuring large language models (LLMs) produce content that…

Researchers at the University of Tokyo Introduce a New Technique to Protect Sensitive Artificial Intelligence AI-Based Applications from Attackers

2 Mins read
In recent years, the rapid progress in Artificial Intelligence (AI) has led to its widespread application in various domains such as computer…

Do Machine Learning Models Produce Reliable Results with Limited Training Data? This New AI Research from Cambridge and Cornell University Finds it..

2 Mins read
Deep learning has developed into a potent and ground-breaking technique in artificial intelligence, with applications ranging from speech recognition to autonomous systems…



Leave a Reply

Your email address will not be published. Required fields are marked *