Tom Rainforth @tom_rainforth

Professor of Statistical Machine Learning at the University of Oxford, Head of RainML Research Lab (https://t.co/uuBwQ2WdMN) robots.ox.ac.uk/~twgr/ Oxford, England Joined November 2016

Tweets

374
Followers

7K
Following

293
Likes

482

Tom Rainforth @tom_rainforth

a month ago

I'm delighted to share that I have been made a full professor at Oxford (specifically, I am now Professor of Statistical Machine Learning)

316 394 7K 146K 239

View Details

1/n 🧵 Model evaluation is often bottlenecked by labelling cost. Active testing estimates a model's risk from far fewer labels, yet existing methods overlook a now-common resource: cheap predictions from powerful black-box models. We address this with PPAT.

3 7 26 4K 16

View Details

Freddie Bickford Smith @fbickfordsmith

10 months ago

Active testing enables label-efficient model evals but can be computationally expensive. We show how to reduce costs and scale up to LLMs. arxiv.org/abs/2508.09093 Work led by Gabrielle Berrada. Find her at EurIPS, or @janundnik and me at NeurIPS in San Diego.

3 7 15 8K 9

View Details

Jackson Atkins @JacksonAtkinsX

a year ago

Apple and Oxford just made AI 6.5x better at problem-solving. The secret: it teaches AI agents to ask perfect questions. This rockets success rates from 14% to 91%. No need for fine-tuning or retraining. It runs on current models. Here's how it works: It's a strategic loop designed for multi-turn conversations. At every step, the agent works to find the shortest path to the right answer. Hypothesize: The agent creates an internal list of all possible solutions to the problem. Score Questions: It simulates asking various questions and scores each one on "Expected Information Gain" (EIG). This number represents how much a question is mathematically likely to shrink the list of possibilities. Ask the Best Question: It asks the user only the single, highest-scoring question. Update & Repeat: Based on the answer, it filters its list of hypotheses, getting smarter with each interaction, and then begins the loop again for the next turn. Why this matters for your AI strategy: This marks a shift from building passive "oracles" to proactive, question-asking agents Business Leaders: A 6.5x multiplier on task success is a lever for efficiency. This translates to fewer failed customer interactions, faster diagnostics, and more accurate personalization, a clear ROI on smarter AI. Practitioners: This is a deployment-time framework, not a new model. You can build this agent on top of existing LLMs today. It provides a principled way to overcome common multi-turn issues like inconsistency and context loss without fine-tuning or retraining. Researchers: This paper is a victory for information theory. It proves that a full EIG calculation is superior to heuristics like predictive entropy. It sets a new standard for how to build intelligent information-seeking agents.

23 136 827 90K 1K

View Details

Tom Rainforth @tom_rainforth

a year ago

I have an opening for a 2-year postdoc in probabilistic machine learning and/or experimental design. The application deadline is the 3rd of September. See here for details and how to apply: tinyurl.com/rainmlpostdoc2…

0 11 51 4K 8

View Details

Tom Rainforth @tom_rainforth

2 years ago

@BlackHC @AdaptiveAgents @AndreyMalinin @kschweig_ @aichberger @HochreiterSepp @roydanroy @fbickfordsmith @WmLisa Given models are never really perfectly specified, this is very important in practice.

0 0 0 174 0

View Details

Tom Rainforth @tom_rainforth

2 years ago

@BlackHC @AdaptiveAgents @AndreyMalinin @kschweig_ @aichberger @HochreiterSepp @roydanroy @fbickfordsmith @WmLisa 2) it can be less stable compared to things like stacking (see Eg proceedings.mlr.press/v238/reichelt2…). Even if the model is well specified, you can get big variability in predictions with the exact data you happen to see

2 0 1 367 0

View Details

Tom Rainforth @tom_rainforth

2 years ago

@BlackHC @AdaptiveAgents @AndreyMalinin @kschweig_ @aichberger @HochreiterSepp @roydanroy @fbickfordsmith @WmLisa 3) it doesn't widen your model class. This is sort of the same as 1), but there also is a general principle that a combination of hypotheses is more powerful than one, and BMA will always collapse to one. There are methods of "Bayesian model combination" that do this.

0 0 2 174 0

View Details

Tom Rainforth @tom_rainforth

2 years ago

@BlackHC @AdaptiveAgents @AndreyMalinin @kschweig_ @aichberger @HochreiterSepp @roydanroy @fbickfordsmith @WmLisa 1) the optimality goes away when the model is misspecified (Eg Bayesian decision trees are usually significantly inferior to random forests)

0 0 2 193 0

View Details

Tom Rainforth @tom_rainforth

2 years ago

I have an opening for a 2.5-year postdoc position in the RainML lab as part of my ERC grant on probabilistic machine learning and intelligent data acquisition. Application deadline 10th July 2024. See here for details and to apply: tinyurl.com/rainmlpostdoc

1 21 56 12K 20

View Details

Tom Rainforth @tom_rainforth

2 years ago

I'm delighted to announce that from September I will officially be an Associate Professor (remaining at the Oxford stats department)

17 3 176 17K 3

View Details

Tom Rainforth @tom_rainforth

2 years ago

All credit goes to my fantastic coauthors @nmstatistics and @yeewhye

0 0 1 643 0

View Details

Tom Rainforth @tom_rainforth

2 years ago

Our new #ICLR2024 paper shows how LLMs can successfully check their own change of thought reasoning without any fine-tuning or even examples, using an approach we call SelfCheck. Join me at poster 125 this afternoon to learn more Paper: openreview.net/forum?id=pTHfA…

1 5 31 4K 8

View Details

Tom Rainforth @tom_rainforth

2 years ago

In-context learning can learn novel input-output relationships beyond what can be picked up from input context alone, but doesn't behave like conventional learning algorithm. Find out more at our ICLR poster #129 this afternoon. Paper: openreview.net/forum?id=YPIA7…, led by @janundnik

0 2 14 2K 3

View Details

Jannik Kossen @janundnik

2 years ago

Are you at ICLR? Have you heard that In-Context Learning in LLMs does not learn label relationships? Well that's not true. Visit our poster TODAY to find out how LLMs incorporate label information. Spoiler: it's not Bayesian inference. Poster #129, May 7, 4.30 pm

1 9 35 6K 20

View Details

Tim Reichelt @TimReichelt3

2 years ago

I will be presenting our work on "Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support" at AISTATS in Valencia tomorrow (details in thread below). If you are interested in probabilistic programming, come and say hi at poster session 1!

Tom Rainforth @tom_rainforth

3 years ago

Our new paper (arxiv.org/abs/2310.14888) shows that probabilistic programs with stochastic support are implicitly Bayesian model averages (BMA) which leads to issues if we assume our model is misspecified! w/ @TimReichelt3 and Luke Ong A thread (1/5)

1 10 53 7K 21

0 1 20 1K 0

View Details

Freddie Bickford Smith @fbickfordsmith

2 years ago

The current default recipe for Bayesian active learning doesn’t really work beyond MNIST scale. We suggest why that is and identify a simple fix. arxiv.org/abs/2404.17249 @aistats_conf with @AdamEFoster @tom_rainforth 1/5