AI lets you try more ideas than you'll ever ship. Each one can teach you something. Combine what you learn and that's what ships. Ideally something of value to users.
the right way to use model capabilities is not to ship 10x more features to prod
it's to spend more time understanding your users, trying experiments, building prototypes, learning about things you don't understand so that you can ship things that actually work
the right way to use model capabilities is not to ship 10x more features to prod
it's to spend more time understanding your users, trying experiments, building prototypes, learning about things you don't understand so that you can ship things that actually work
a lot of students have a hard time with this so let me spell it out:
- class is for asking dumb ass questions to someone that knows the material
- learning happens on your own outside the classroom
- office hours is to ask adversarial questions to the main grader about exams
Nice write up. I also think we're at the beginning of software engineering completely changing forever. Not just "muh vibecoding yolo", but something more fundamental:
libraries are currently the cornerstone of swe. I think the future will be without libraries. You'll do ~everything in your codebase, tailored specifically to exactly what you do. The agents see the whole code. You can change the behavior of anything exactly the way you need it. No need to be backwards compatible. No need to worry about breaking others.
What i think might become more ubiquitous instead is something akin to "starter packs" which could really just be something like tutorials explaining a concept. Like if you make a game, get the "rpg(.)md from SquareEnix" and the "entity-component(.)md from Lucas" files into context and go.
Not "no libraries at all" but rather "only direct vendor libraries" like of course Vulkan or DirectX are still needed, but Unity or Unreal? Not so much. And i think this will happen in all domains. It will take a while, but i think e are at the very beginning of this future. I'm already starting to live in it.
I used to love coding for the sake of coding, and was a bit worried of it going away. However, it turns out it's not really the coding itself that i love, it's more the combined act of creating something and having "puzzles" to think about while doing so. And it turns out that in this new way of coding, these two parts are just as fun.
I think with Codex 5.3, the need for off-the-shelf deep learning libraries will fade away.
Reasoning models operate best at the boundary of exact verifiabilty, so ever venturing too far into "well this is kinda correct" is no longer the best strategy. Exact verification now
Humans try hard things, fail, learn, and get better through repetition.
AI models aren't as different as you might think.
Here's how models "learn" explained in simple terms. x.com/i/article/2080…
Based based based based based based based
Do not fall for Big Labs that make Frontier Intelligence look like a magic spell, it's all simple tech layered on top. Most of the big labs evals suck anyway - anyone not dumb can write better stuff.
Do you think big labs employees can write good code?
Do you think they can design anything well abstracted? No, not really. Their programs prove it, all of it sucks. Kudos to Kimi for making all this tech easily inspectable by everyone.
Releasing the model weights and technical report of Kimi K3.
Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.
New model architecture: 2.5x the intelligence per unit of compute, not just more params.
Alongside
Last month I wrote about how we can build a positive and safe future for everyone: meta.com/thefutureisfor…
Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.
The reality is:
- People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned.
There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.
- Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.
Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built.
- Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.
- Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well.
I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.
To have enough mental bandwidth to think deeply about what's going on in physics or any math-heavy field, you need to be very comfortable executing the underlying math.
And that's not going to happen if the math you need is close to the edge of your ability.
Sure, you can execute it... but not *comfortably,* and that makes all the difference.
Your high-level train of thought is going to get continually derailed by the low-level details of the math you have to do.
You're going to have a hard time seeing the forest for the trees.
Introducing CUDA Rust!
CUDA Rust lets you write GPU kernels natively in Rust, not just launch them from it.
Two paths: cuda-oxide for SIMT kernels compiled to PTX, and cutile-rs for Tile-based programming on stable Rust. Both can catch aliasing errors at compile time.
Technical blog: nvda.ws/4hm1bHS
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
992 Followers 3K Followingfde @modal
hosts london systems club
prev @snyk, @meta
prev founder @nordlyslabs
Opinions are my own.
https://t.co/YUS1SSEcF7
606K Followers 59K FollowingThe automated life | San Francisco/Silicon Valley AI, robotics, BCIs | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future.
28K Followers 8K FollowingComputer Use & Frontier Interfaces @OpenAI. Previously co-founder of https://t.co/EpbAUfAcoO, manager of Shortcuts at Apple, co-founder of Workflow.
6K Followers 8K Following🇪🇺 CTO. Builder.
Rusty AI agents. Category theory without the fog machine. If you build typed agents, tiny ML or evals, DM me your weirdest bug.
1K Followers 475 FollowingBuilding stuff with AI & code until I can pay bills / @destination_app - Bus tracker / @shortsfusion - AI marketing with videos
25K Followers 17K FollowingEx-builder at @fold_app. Libertarian/ancap working to spread the message of liberty, free markets, and sound money - #Bitcoin.
196 Followers 1K FollowingThe smartest way to learn, retain and apply what matters. Built for people who never stop learning.
˗ˏˋhttps://t.co/Xrz8FvbXwDˎˊ˗
10K Followers 466 FollowingCo-founder/CEO of @convex. Former Sr. Director/Principal Engineer of Storage & Databases at Dropbox. Shameless startup junkie.
146K Followers 94 FollowingSane + 🌶️ takes in an insane AI world... AI capabilities researcher: co-created RLHF/ChatGPT @ @openai now trying to right the wrong 🤭 (ceo @typesafeai)
77K Followers 3K FollowingWe're in a race. It's not USA vs China but humans and AGIs vs ape power centralization.
@deepseek_ai stan #1, 2023–Deep Time
«C’est la guerre.» ®1
22K Followers 1K FollowingThe sea is the sea
The old man is an old man
The boy is a boy and the fish is a fish
The sharks are all sharks no better and no worse
4K Followers 661 Following@NVIDIA Sr. Research Scientist | UIUC PhD
All opinions and tweets are personal.
Tweets about AI Inference, CUDA and GPU systems.