Tutorials

Organizer: Aaron Klein, ELLIS Institute Tübingen
Correspondence:  aaron.klein@tue.ellis.eu

Before training a large language model, we need to ensure that our training procedure (e.g. hyperparameter setting, dataset mixtures, model architecture, etc.) works at the target scale. However, since we can only train our model once at full scale, we need to test it on smaller scales first. Scaling laws offer a principled way to predict how a training procedure will behave at scale. By modeling loss as a function of parameters, data, and compute, we can extrapolate from many cheap small-scale runs to the single large model we actually intend to train. This tutorial introduces neural scaling laws from the ground up, starting with the foundational results of Kaplan et al. and the compute-optimal Chinchilla frontier, the different methodologies used to estimate them, and why early studies disagreed on the optimal trade-off between model size and tokens. I will then turn to scaling laws in practice, using OpenEuroLLM, which is a fully open effort to pre-train multilingual LLMs on European supercomputers, as a running case study. I close with an outlook on automating this process and its connections to AutoML.

Organizers: Andreas Kist, Sebastian Zimmermann, René Groh
Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) 
Correspondence:  andreas.kist@fau.de

Edge AI brings machine learning closer to sensors, users, and real-world decision points, but deploying models on microcontrollers and other highly constrained devices remains challenging. Accuracy alone is rarely sufficient: models must also satisfy strict constraints on latency, memory footprint, energy consumption, robustness, and deployability. This makes Edge AI a natural and timely application area for Automated Machine Learning. This tutorial will provide an accessible introduction to hardware-aware AutoML for Edge AI. We will first motivate why edge deployment is increasingly relevant for the AutoML community and explain the specific constraints that distinguish microcontroller-class hardware from cloud or GPU-based deployment. We will then introduce common strategies for bringing models to embedded devices, including architecture design, quantization, pruning, compilation, benchmarking, and hardware-in-the-loop evaluation. The second half of the tutorial will be hands-on: participants will use MIMaaS, a Microcontroller-as-a-Service platform, to evaluate candidate models on physical hardware and interpret latency, memory, and power measurements in a multi-objective AutoML setting. The tutorial will close with open research challenges and discussion. 

Organizers: Blaž Škrlj (Teads, Jožef Stefan Institute), Sebastian Koralewski (Teads)

Correspondence: blaz.skrlj@teads.com, sebastian.koralewski@teads.com

AutoML treats model design as a search over hyperparameters, architectures, and pipelines, a processusually driven by Bayesian optimization, evolutionary algorithms, or bandits over fixed encodings. Arecent line of work replaces that operator with a large language model. By processing naturallanguage task and past performance scores, the LLM directly proposes new candidates, mutatespopulations, or rewrites programs. This tutorial is a field overview of that idea. Part I: Introductionto LLM-Driven Search surveys the main patterns and the methods that instantiate them:hyperparameter tuning and Bayesian optimization (LLAMBO), neural architecture search (GENIUS),prompt optimization (OPRO, EvoPrompt), program and algorithm discovery (FunSearch,AlphaEvolve), reward design (Eureka), and feature engineering (CAAFE, PromptFE), relating each toevolutionary search and learned optimizers (OptFormer). It then examines what the publishedevidence does and does not establish: compute-matched comparison, evaluation noise, andbenchmark contamination (MLE-bench). Part II: A Case Study from Practice grounds the survey in avendor-neutral case study detailing a static pipeline turned into an LLM-driven loop. It examines howthe model proposes configurations, evaluates results, and iterates, alongside a discussion ofguardrails, cost controls, and failed attempts.