Research

We work on large language models, studying how to steer, adapt and control them, and on optimization, with learning-based and massively parallel methods for combinatorial problems.

Large Language Models

Prompting and in-context learning

LLMs are highly sensitive to how they are prompted, and good prompts are hard to find by hand. We construct prompts and in-context examples automatically, and study how far prompt sensitivity goes.

• PRL: Prompts from Reinforcement Learning
• GPS: General Per-Sample Prompter
• TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation
• PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data
• PLR: Plackett-Luce for Reordering In-Context Learning Examples
• Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?

Unlearning and model control

Deploying LLMs safely requires removing specific knowledge without retraining, and checking that it is really gone. Beyond unlearning, we change model behaviour at inference time and make generated text traceable.

• EvoMU: Evolutionary Machine Unlearning
• GROM: Gradient-Free Rapid One-Shot Machine Unlearning
• REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop
• ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion models
• MACRO: Markov Chain Routing of Transformer Layers
• CertMark: Distortion-Free Multi-Bit Watermarking with Certified Decoding

Reasoning

We study how neural models reason, both by looking inside them and by combining them with symbolic search.

• A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task
• NSA: Neuro-symbolic ARC Challenge

Optimization

Neural combinatorial optimization

We develop neural methods for combinatorial problems that can spend more computation on harder instances.

• Compute Time Scaling with Recursive Models for Combinatorial Optimization

Massively parallel solvers

Many machine learning problems contain large integer linear programs at their core. We build solvers that run massively in parallel on GPUs and can be trained end to end.

• FastDOG: Fast Discrete Optimization on GPU
• DOGE-Train: Discrete Optimization on GPU with End-to-end Training
• RAMA: A Rapid Multicut Algorithm on GPU
• Efficient Message Passing for 0-1 ILPs with Binary Decision Diagrams
• Structured Prediction Problem Archive

Earlier work

Earlier, we applied combinatorial optimization to computer vision: graph and shape matching, multi-object tracking and clustering.

• DiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape Matching
• Lifted Disjoint Paths with Application in Multiple Object Tracking
• ClusterFuG: Clustering Fully connected Graphs by Multicut

All publications: Google Scholar · DBLP