Halvar Flake @halvarflake

Nancy Leveson drives this point home in her excellent book, Engineering a Safer World. "Safety" (or in today's parlance, "alignment") is not a property of the automation (the self-driving car, the AI agent). It is a dynamic property that must be maintained through active control.

So mathematicians are suddenly finding that "explanation" and "exposition" aren't arts of second-rate minds after all, eh? Hardy must be turning in his grave..

I dislike the recent trend of people who clearly continue to believe >80% of the core effective altruist beliefs disavow EA and say "oh we're definitely not that, we're [rebrand]" purely strategically, purely because they sense that dunking on EA is "cool" now. It's much more honorable to just come out and say "yes, that was our intellectual upbringing and those continue to be our core values, and also here are specific points where we disagree, and here's our argument for why we think for those specific points are the subset of EA that causes what you perceive to be the problem with the whole thing". And so with that I will say: Yes, effective altruism was a significant part of my intellectual upbringing and its core values of care for others, cosmopolitanism and rigor around effective resource allocation continue to be very important to me. And also here are specific points where I disagree with what I think is the subset of that style of thought that has backfired in the largest ways: vitalik.eth.limo/general/2025/1…

@halvarflake I've read your incident report. I can't help but notice that, despite saying the rabbit might kill us all, you've spent twice as much space on an interview with the rabbit about its rabbit-y feelings, than on the construction of the cage that you've put the little blighter in.

I know it's unfashionable to talk about limitations in the frontier models, but hear me out 130 billion (output) tokens for Navier-Stokes is like ~4000 person-years of effort (Dwarkesh's estimate) 4000 years is a long time: it's 100x more than a human gets to earn a Fields Medal Obviously it's crazy impressive that a machine was able to do this at all; and also impressive that it did so in such short wall-time But simultaneously, it's conspicuous that this level of effort didn't result in a stronger outcome: a new theory, a cleaner approach, etc (And before some rando accuses me of being an AI hater, I'll just add: lol, lmao even)

@HostileSpectrum Tumbling down the hill in a clown suit while managing to faceplant on every tumble is certainly a way to build novel theory?

@HostileSpectrum Also a language not historically known for intelligence successes.

Repost. short read. 9 pages, tinyurl.com/FacesIG/

US Government Misstated COVID Vaccine Risk The British Medical Journal (BMJ) reports that during the COVID vaccine rollout in 2021, US health officials used a safety-monitoring algorithm that they knew was inadequate but ignored the problem. At the time, the Food and Drug Administration (FDA) compared reports of adverse events after each vaccine with reports after other vaccines. But by 2021 more than 90 per cent of reports were about the (very similar) mRNA vaccines from Pfizer and Moderna. Because of this, adverse events which occured for both vaccines would not have statistically stood out. The BMJ extracted this information from internal emails it obtained under the Freedom of Information Act. These emails reveal that FDA employees Ana Szarfman and statistician William DuMouchel warned Peter Marks (then the FDA’s vaccine regulator) about this problem in early 2021. They proposed an improved method, but higher FDA administration ignored the warning. As a consequence, the algorithm failed to flag myocarditis (an inflammation of the heart muscle), which was later found to be a side-effect of mRNA vaccination. Report here: dx.doi.org/10.1136/bmj-20…

Given intense name collisions with artificial intelligence, the national security intelligence discipline probably just ought to shift to the French term “Renseignement” to distinguish what we are talking about

🥳 Excited to start revealing what we've been working on in the last few months. First, we decided to reinvent Kubernetes for agentic workloads with statefulness and fast resumption. Secondly, we are building an agentic orchestrator that will be Google's open agentic orchestrator and runtime. github.com/google/ax

This is an absolutely excellent read.

zooko🛡🦓🦓🦓 ⓩ @zooko

22 hours ago

Huh! This is an excellent essay by Venkatesh Rao. It is excellent in historical detail, analyses of the current social/business/intellectual/political structure, and in his clarion call for pluralism. contraptions.venkateshrao.com/p/ea-safety HT Tracy Harms

4 5 67 14K 50

In “Why Do We Need Human Mathematicians Anymore?”, Po-Shen Loh starts from an axiom: humans should help humanity flourish. As AI creates more “control points,” experts must oversee them -- and keep doing research to retain the expertise to steer AI. terrytao.wordpress.com/2026/09/19/why…

Things are heating up on Terry Tao’s blog. In “If Math Is More Than Proof, We Need to Better Celebrate the Rest of It,” Grant Sanderson of @3blue1brown proposes “open exposition problems” -- rewarding the work of making math genuinely understandable. terrytao.wordpress.com/2026/09/18/if-…

Here are my fav talks at the first @UnpromptedAU: @chompie1337 shared her 1-day exploit factory. Although she focused on Windows, many of the ideas also apply to other platforms and systems. She's a fantastic speaker who made a deeply technical topic accessible and easy to understand, even for noobs like me. @moyix's thesis is that AI has made reverse engineering much easier, and therefore security through obscurity is dead. I completely share this view. I won't say more because I don't want to spoil Brendan's talk, but man, he really loves what he does, and that love is contagious. Now I want to hack with a cat too! Ash Fox shared how his team at Google is building an AI red teamer. This one hit very close to home! I really enjoyed hearing how they teach their agents to maintain OPSEC. I also loved his idea of "living off the agents", leveraging existing agents to move laterally. @justdionysus shared the story of his first week at Calif, when he bypassed Apple's MIE with AI. I'm 100% biased, but Dion had the most beautiful slides at the conference. I'm fucken jealous -- how the hell he could make them look that good. It's a great conference, small enough to feel like a real community where everyone knows one another, but big enough to make new friends. It reminds me of Ekoparty in the early 2010s. Sydney is lovely, too. I'll definitely be back.

My pet rabbit chewed through the untreated wooden bar at the bottom of the cage and escaped. He's already superhuman at gnawing through wood, I strongly suspect that even a steel box can't contain him.

@zooko Glad you like them!!

@TalkingMusicz Oh, I understand the unanswerable part, and I answer with an axiomatic choice.

News at 12: if you spend a few hundred billion on emulating human language and inner monologue, you get a system that closely mimics human language and inner monologue.

Cameron Berg @camhberg

2 days ago

New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵

301 519 4K 1.3M 3K

@XenoKovah @aviramyh A classic

As the discourse moves further to what I personally deem a form of psychosis, I feel I have to state the following: I am unabashedly "carbonist", in the sense that I consider carbon-based organism to be life. There is no amount of language an LLM can generate that would make...