Fahim Arefin @FArefin

*Maximizing Happiness* Joined May 2011

Tweets

7
Followers

16
Following

129
Likes

32

Phoebe Yao @phoebeyao

3 months ago

Today we’re releasing LEAPBench, a benchmark for how efficiently models learn from feedback over many rounds. Most evals only score whether a model eventually reaches a good solution. LEAPBench scores how many tries it took to get there. Two systems can reach the same optimum while one needs 7 experiments and the other needs 27. Most benchmarks call that a tie, even though those 20 extra runs cost real time and materials. LEAPBench is 55 multi-turn optimization tasks, 45 in biology and 10 education RCTs, built from 2,719 real experimental observations. In each task, the model proposes parameters, sees the result, and iterates toward an optimum over as many as 30 rounds, e.g. one biology task involves maximizing antibiotic yield by tuning variables like pH, glucose concentration, temperature, and soy flour level.

11 9 108 28K 51

View Details

Phoebe Yao @phoebeyao

5 months ago

hiring spls and ems at @pareto_ai right now. dm me if you want to talk. happy to share more about our team, the work, and what it's actually like.

Phoebe Yao @phoebeyao

5 months ago

two of the highest-leverage roles for operators trying to break into ai right now: spl (strategic projects lead) and em (engagement manager) positions at data companies. day-to-day, they sit between a frontier lab's research team and the experts producing training data,

4 2 90 25K 85

5 3 46 8K 33

View Details

GREG ISENBERG @gregisenberg

7 months ago

you don't need to overthink this when you look at this goldman sachs chart long enough it becomes pretty obvious how people will build the next wave of $10m-$100m+ ARR vertical ai companies ill break it down so we all know every business function produces something tangible 1. a recruiting pipeline produces candidate summaries 2. a finance team produces monthly reporting packages 3. a real estate team produces market analyses and listing packages those outputs come from repeatable processes that pull information from a handful of systems and sources. builders who win in this environment start by understanding how those outputs get created today they collect real examples, reconstruct the process step by step, then design software that gathers the inputs and assembles the finished output automatically as adoption grows, the system expands into adjacent responsibilities until the product becomes the infrastructure that function runs on most people still think in terms of software categories. CRM. ATS. ERP. project management. that framing misses what is happening the next great vertical ai companies will be built around finished work. they will own the artifact the customer actually cares about, then expand outward until they own the function so the opportunity isnt really “build an ai tool for real estate” which is what i see a lot of on twitter the opportunity is much more specific: 1. build the ai employee that creates the broker opinion of value 2. build the ai employee that prepares the insurance renewal package 3. build the ai employee that drafts the first version of the investment memo 4. build the ai employee that assembles the lender reporting package every month that is how small software companies become very large ones in this market start with one painful output, automate it well, then expand until you own the workflow basically you go from automation to ai employees if you don't remember anything from this long post, remember that it's obvious that this is where its all going you dont need to overthink it you're in the robot business now

Idea Browser @ideabrowser

7 months ago

I look at this chart every day

5 16 95 189K 67

116 96 1K 193K 2K

View Details

POM @peterom

7 months ago

Deepseek got called out for scraping 150k Claude messages. So I'm releasing 155k of my personal Claude Code messages with Opus 4.5. I'm also open sourcing tooling to help you fetch your data, redact sensitive info & make it discoverable on HF - link below to liberate your data!

618 2K 15K 24.3M 9K

View Details

Phoebe Yao @phoebeyao

7 months ago

x.com/i/article/2027…

9 20 244 43K 276

View Details

sameel arif @endpoint

8 months ago

this research paper's website is so sick!!

43 241 3K 212K 3K

View Details

Fahim Arefin @FArefin

8 months ago

@phoebeyao This applies across high-context domains. Prompting is like telling a self-driving car "be careful." Real safety comes from sensor-based monitoring and automated braking - and those control layers only work when they're grounded in expert human judgment, not generic rules.