Albatross @albatrosscode

Open source agentic coding harness where local and frontier models live in harmony. albatross.sh Joined January 2009

Tweets

4K
Followers

4K
Following

341
Likes

7K

Morgan @morganlinton

2 months ago

I am going to start to share the leaderboard at VulcanBench more regularly. VulcanBench is the only benchmark I know that benchmarks across effort levels, on an eval suite of 100% real engineering tasks. Getting ready to sunset eval suite 3 and eval suite 4 is almost ready to go. Here's the current leaderboard for Eval Suite 3, Grok 4.5 High is the highest scoring model still with Fable 5 Low, yes Low, right behind it. What I've been able to uncover with VulcanBench is that way too many people are running models at Max effort, thinking that buys them more accuracy, when really it just costs more and uses more tokens so takes more time. If you're still in the mode of, tell my agent to do something then go get coffee, you're probably still living in the past. You can move faster, with higher accuracy, the key is not thinking you need Max effort all the time.

VulcanBench @VulcanBench

2 months ago

Time to share the leaderboard for VulcanBench, Eval Suite 3. As a reminder, these are 100% real engineering tasks, from real repos. No puzzles, no math problems, no exotic architecture. Our leaderboard takes the data we have from our full effort sweeps, every effort level, and

18 19 115 50K 41

72 45 516 5.1M 179

View Details

Albatross @albatrosscode

4 months ago

@jpschroeder Agreed.

0 0 0 49 0

View Details

Albatross @albatrosscode

4 months ago

@EXM7777 @orca_build Super interesting, thanks for sharing.

0 0 0 181 0

View Details

Albatross @albatrosscode

4 months ago

@kimmonismus Yeah, was very bummed to see this 😞

0 0 0 117 0

View Details

Albatross @albatrosscode

4 months ago

@bindureddy How do you know that 5.6 is cheaper, faster, and more pragmatic? Did you get early access?

0 0 0 38 0

View Details

Albatross @albatrosscode

4 months ago

Super interesting to see how strong the adoption curve has been with GLM 5.2

OpenRouter @OpenRouter

4 months ago

GLM 5.2 vs DeepSeek V4: share of tokens on OpenRouter GLM 5.2 has had a very fast uptake relative to most open-weight launches

6 months ago

One of the best small models out now.

Bindu Reddy @bindureddy

6 months ago

Gemma 4 is a very good small model that punches above it's weight class Gemma is a 31B model that is as good as other very large MoE models It's the best in the world for it's size 👏👏