International Conference on Machine Learning (ICML) 2024

Apple is sponsoring the International Conference on Machine Learning (ICML) 2024, which is taking place in person from July 21 to 27 in the Messe Wien Exhibition and Congress Center, Vienna Austria. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. Below is the schedule of our sponsored workshops and events at ICML 2024.

Accepted Papers

Contrasting Multiple Representations with the Multi-Marginal Matching Gap
Zoe Piran, Michal Klein, James Thornton, Marco Cuturi Cameto

Data-free Bootstrapping Distillation
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu (University of Pennsylvania), Josh Susskind

On a Practical Implementation of Brenier’s Polar Factorization Theorem and Its Applications to Optimization and Sampling
Marco Cuturi Cameto, Nina Vesseron (ENSAE)

On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions
Denys Pushkin (EPFL), Raphael Berthier (EPFL), Emmanuel Abbe (Apple/EPFL)

Scalable Pre-training of Large Autoregressive Image Models
Alaaeldin Mohamed Elnouby Ali, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista Martin, Josh Susskind, Armand Joulin (Google Deepmind (Work done while at Apple))

Revealing the Utilized Rank of Subspaces of Learning in Neural Networks
Isha Garg, Eshan Verma, Daniel Ulbricht, Christian Koguchi

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
Thomas Merth, Qichen Fu, Mohammad Rastegari (Meta (Work done while at Apple)), Mahyar Najibi

Aligning Text-to-Image Diffusion as GFlowNets
Dinghuai Zhang (University of Montreal), Yizhe Zhang, Jiatao Gu, Ruixiang Zhang, Josh Susskind, Navdeep Jaitly, Shuangfei Zhai

Careful with that Scalpel: Improving Gradient Surgery with an EMA
Pierre Ablin, James Thornton, Eugene Ndiaye, Yu-Guan Hsieh, Michal Klein, Marco Cuturi Cameto

Executable Code Actions Elicit Better LLM Agents
Xingyao Wang (University of Illinois Urbana-Champaign), Yangyi Chen (University of Illinois Urbana-Champaign), Lifan Yuan (University of Illinois Urbana-Champaign), Yizhe Zhang, Hao Peng (University of Illinois Urbana-Champaign), Ji Heng (University of Illinois Urbana-Champaign)

How Smooth Is Attention?
Valérie Castin (ENS), Pierre Ablin, Gabriel Peyré (ENS)

Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomials
Jonny Scott (Institute of Science and Technology Austria), Aine Cahill

Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
Raviteja Vemulapalli, Hadi Pour Ansari, Fartash Faghri, Sachin Mehta, Mehrdad Farajtabar, Mohammad Rastegari (Meta (Work done while at Apple)), Oncel Tuzel

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
Minsik Cho, Mohammad Rastegari (Meta (Work done while at Apple)), Devang Naik

Optimization without Retraction on the Random Generalized Stiefel Manifold
Simon Vary (UC Louvain), Pierre Ablin, Bin Gao (Chinese Academy of Sciences), Pierre-Antoine Absil (UC Louvain)

Private Vector Mean Estimation in the Shuffle Model: Optimal Rates Require Many Messages
Hilal Asi, Vitaly Feldman, Jelani Nelson (University of California Berkeley), Kunal Talwar, Huy Nguyen (Northeastern University), Samson Zhou (Texas A&M University)

Projected Language Models: A Large Model Pre-Segmented Into Smaller Ones
David Grangier, Angelos Katharopoulos, Pierre Ablin, Awni Hannun

Swallowing the Bitter Pill: Simplified Scalable Conformer Generation
Yuyang Wang, Ahmed Elhag (University of Oxford), Navdeep Jaitly, Josh Susskind, Miguel Angel Bautista Martin

Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau Cuadros, Pieter Delobelle (KU Leuven), Rin Metcalf Susa, Armand Joulin (Google Deepmind (Work done while at Apple)), Nick Apostoloff, Luca Zapella, Pau Rodriguez Lopez

Demos

MLX

We are demonstrating large model inference and training on device using MLX. MLX is a flexible array framework that is optimized for Apple silicon, and brought to you by Apple Machine Learning Research. It enables training and inference of arbitrarily complex models on Apple silicon powered devices with great brevity and flexibility.

In this demo we showcase fine-tuning of a 7B parameter LLM on an iPhone, image generation using a large diffusion model on an iPad, and text generation using a number of large language models on a M2 Ultra Mac Studio and M3 Macbook Pro.

Private Federated Learning (PFL)

This demo showcases Apple’s Private Federated Learning (PFL) technology. PFL-research is the open-source framework enabling this technology for research simulations, open sourced in March 2024 at the Apple PPML Workshop. Here we are able to show how Siri can play Music and Podcasts on iPhones, which leverages several technologies such as Siri Signals and Siri Inference, Private Federated Learning (PFL) and Differential Privacy (DP). This highlights a user-facing feature in iOS that was shipped, thanks to this framework.

Acknowledgements

Ozan Sener and Pau Rodgriguez Lopez are Area Chairs for ICML 2024.

Aadirupa Saha is a Co-Organizer for the Models of Human Feedback for AI Alignment workshop.

Rin Metcalf Susa is a Panelist on the Models of Human Feedback for AI Alignment workshop.

Marco Cuturi, Samy Bengio, and Vladlen Kotlun are Senior Meta Reviewers for ICML 2024.

Arno Blaas, Bailin Wang, Gustaf Ahdritz, Junpei Zhou, Miguel Angel Bautista Martin, Miguel Sarabia del Castillo, Qichen Fu, and Ray Zhang are reviewers for ICML 2024.

Natalie Schluter is a Senior Workshop Chair for ICML 2024.