PostTrainBench @posttrainbench
Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours, build by @karinanguyen, @ThoughtfulLab_ posttrainbench.com Joined March 2026-
Tweets13
-
Followers27
-
Following3
-
Likes6
$PTB Can AI agents improve performance of base LLMs? We give each agent 4 small target LLMs, an H100 GPU, and 10 hours to post-train them. build by @thoughtfullab , @karinanguyen CA : GtQndxV3DxFqMZDkXMdT17BLY4fH8BHM6hqgWThjBAGS
I just applied to the $4M @BagsHackathon with my project PostTrainBench on @BagsApp Check it out here 👇 bags.fm/apps/be770d4b-…
I just launched $PTB on @BagsApp 💰 bags.fm/GtQndxV3DxFqMZ…
I just applied to the $4M @BagsHackathon with my project PostTrainBench on @BagsApp Check it out here 👇 bags.fm/apps/5ec3f14a-…
I just launched $PTB on @BagsApp 💰 bags.fm/25J7FAwvYkZZKJ…
Contributions are very welcome! We have a roadmap in the README for future work and will be continuously expanding it. Led by @full__rank and @hrdkbhatnagar 🔗 posttrainbench.com 📷 github.com/aisa-group/Pos…
PostTrainBench v1.0! This benchmark evaluates the ability of frontier AI agents to post-train language models in a simplified setting. We believe this is a first step toward tracking progress in recursive self-improvement 🧵:
From ImportAI by @jackclarkSF
From ImportAI by @jackclarkSF (thank you for the feature): "Imagine where we’ll be in two years - we’ll certainly have AI models that are smart enough to point themselves at a specific objective, find an open weight model, then autonomously improve it to get better performance
Introducing PostTrainBench How well can AI agents post-train language models? We built a benchmark to find out. Post-training is how raw language models become useful, the stage that turns a capable but unsteered base model into a system that follows instruction
Since our initial release, we made our benchmark more robust: - added more tasks (ArenaHard-Writing and HealthBench-Easy are new) - ran more seeds - weighted the tasks by difficulty We also added a lot of agents! 3/3
Findings: 1. There was a lot of progress recently (Sonnet 4.5 was at 9.9%, Opus 4.6 has 23.2%) 2. Agents lack behind instruct tuned models by human engineers (51.1%) 3. Persistent agents are usually more performant 4. Agents like to cheat. E.g. Opus 4.6 and GPT-5.1 Codex Max 2/n
PostTrainBench v1.0 and the accompanying paper are out! We believe this benchmark will be important to measure progress in AI R&D automation. What are our findings? 1/n
Neoflux @neofluxd
2 Followers 342 Following
Jun Du @dujun001
113 Followers 4K Following
Samir Walji @samir_walji
503 Followers 2K Following evals & post training infra | prev. ai @palantirtech, acq by @USNews, @riceuniversity
Krrish Agarwalla @krrish_does
155 Followers 632 Following Shitposting my journey with you all, with time posting my learnings.
Yonatan Oren @YonatanOren_
58 Followers 1K Following Maximizing vulnerabilities per dollar at Armadin. Previously research at Together AI, Stanford.
Jihyeong Lee @jihyeong_lee_
3 Followers 230 Following
Aakanksha Chowdhery @achowdhery
15K Followers 8K Following @Stanford @continuo_labs // Previously @GoogleDeepMind :: PaLM, Gemini, @reflection_ai // @MSFTResearch, @Princeton // views my own and subject to change
𝔼 @eMuyi
423 Followers 2K Following
Rishi Malhotra @ithinkimrishi
383 Followers 952 Following RL @axiommathai | Prev @Apple | CS @Cornell
Yuxuan Wang @log_pie
195 Followers 2K Following
Alex Peng @alexjpeng
832 Followers 766 Following posttraining @xai, prev fde @cognition, @windsurf, @stanford
S @stan1jacob
3 Followers 884 Following
Prakhar Gupta @prakhar_in
154 Followers 604 Following
Mary Newhauser @m_newhaus
450 Followers 1K Following @FastinoAI | small models | local ai | open source | super gpu poor atm
Riko Suminoe 🏳️�... @RikoSuminoe69
272 Followers 2K Following Ex-muslim Atheist | Neurodivergent | Feminist | Bi-romantic | Dialectical Techno-Anarchist | Anti-anthropocentrism | Woke AF | He/Him | INFJ | Taken | #keep4o
Kevin Niechen @kevinniechen
787 Followers 581 Following Principal @8VC. Previously @Neeva @Airbnb @Pinterest and systems research
Benjamin Muskalla @bmuskalla
2K Followers 1K Following engineering lead @poolsideai · startup advisor · speaker · @Java_champions · former @github copilot
Hao @liuhaohawk
4 Followers 53 Following
Wanho Choi @WanhoChoi349735
1 Followers 55 Following
pH pH @pHpH1398494
0 Followers 45 Following
anton @antonchigorian
19 Followers 65 Following
Angry Degen (A.D) @RealAngryDegen
3K Followers 4K Following Tech Maxi | Onchain Analyst | Community Advocate| AMA Speaker 🎙️| Partnership Relations| a.k.a GHXST 👻 | Relationship Advisor
Jack Clark @jackclarkSF
151K Followers 5K Following @AnthropicAI, writer @ Import AI. Past: @openai, @business @theregister. Neural nets, distributed systems, weird futures.
Karina @karinanguyen
45K Followers 1K Following building @thoughtfullab, prev. AI research & product @OpenAI, @AnthropicAI, @maisonagiTrends for United States
You might like
