Lukas Fisch @codingfisch

Don't panic! Just build! https://t.co/FNuUXBB6HY Joined December 2013

Tweets

137
Followers

200
Following

108
Likes

327

Lukas Fisch @codingfisch

a month ago

@timodonnell Cool! Any idea what causes those drops in throughput?

0 0 0 180 0

View Details

Lukas Fisch @codingfisch

a month ago

@classiclarryd Makes these nice loss curves even more impressive 💪

0 0 1 144 0

View Details

Lukas Fisch @codingfisch

a month ago

@___Harald___ The Curse of Dimensionality: The Cure of Composability

0 0 1 138 0

View Details

@kjaved_ @chris_j_paxton Can you share specifics about the new OaK implementation? Options and Knowledge is clearly what we currently lack in RL! Instead of learning one blob of weights a composable set of learned things is needed. E.g. what OOM parameter count is it? Thousand, millions or billions?

0 0 0 55 0

View Details

Lukas Fisch @codingfisch

3 months ago

@kjaved_ @jsuarez "Atari Breakout in 4 seconds" and Physical Atari are two extremes. The first is too easy (not using pixels but [ball_x, ball_y...]). The second is too hard (takes too long to iterate on it). Currently working on something that tries to fill the gap...will take a few more weeks

1 0 0 185 0

View Details

Lukas Fisch @codingfisch

3 months ago

@kjaved_ @jsuarez The best part is...i have to wait for 6 hours. The pitfall of RL research in one sentence

1 0 0 217 0

View Details

Lukas Fisch @codingfisch

3 months ago

@mhutter42 The models show respect because they know that they are all slowly converging towards the AIXI model 😄

0 0 2 58 0

View Details

Lukas Fisch @codingfisch

3 months ago

@__tinygrad__ @bsilone Does the exabox fit in a bike shed? Would love that picture just for the lolz 😄

1 0 0 131 0

View Details

Lukas Fisch @codingfisch

3 months ago

@___Harald___ And this is the one true reward: ∃

0 0 1 130 0

View Details

Lukas Fisch @codingfisch

5 months ago

@OosGergVer @__tinygrad__ He should reconsider: Selling chips w solid firmware and OSS stack on top -> other people fixing the OSS stack -> possible alternative to CUDA moat CUDA is why nvidia is worth that much

1 0 3 282 1

View Details

Lukas Fisch @codingfisch

6 months ago

@__tinygrad__ These straighforward comparisons to LLMs are an insult to the brain. The brain runs so much more efficient software. LLM vs brain is like Windows 11 vs TempleOS. I liked the estimate in your blog post "Brain FLOPS": 162 GFLOPS.

0 0 2 426 0

View Details

Lukas Fisch @codingfisch

8 months ago

@gadevenyi Oops, this one should work: github.com/wwu-mmll/deepm…

1 0 2 82 0

View Details

Lukas Fisch @codingfisch

8 months ago

"deepmriprep: VBM preprocessing via deep neural networks" is published in Nature Computational Science 🧠💻 🔗 Paper: rdcu.be/e1t4N VBM preprocessing in ~10 seconds per #brain image 🚀 🔗 GitHub: github.com/wwu-mmll/deepm… Install via "pip install deepmriprep"

2 31 89 5K 61

View Details

TimHahn @TheRealTimHahn1

8 months ago

This really speeds up preprocessing and shows - yet again - that neural networks are eating software. Great work, @codingfisch !! Happy to see this finally published.

Nature Computational Science @NatComputSci

8 months ago

📢Out now! @codingfisch and colleagues present deepmriprep, a tool that leverages neural networks to enable 37x faster Voxel-based Morphometry preprocessing of MRI data than existing methods. nature.com/articles/s4358…

0 13 28 4K 22

0 1 2 183 0

View Details

Lukas Fisch @codingfisch

8 months ago

Not convinced? How about this: Christian Gaser (author of CAT12) is building his new SBM toolbox around deepmriprep: github.com/ChristianGaser…

0 1 6 302 1

View Details

Nature Computational Science @NatComputSci

8 months ago

0 13 28 4K 22

View Details

Lukas Fisch @codingfisch

10 months ago

@__tinygrad__ @ID_AA_Carmack RISC for array ops sounds too elegant to be impossible (as George said @clattner_llvm told him). I hope you will find the right abstractions to crush this problem soon!

0 0 0 147 0

View Details

the tiny corp @tinygrad

11 months ago

Our GPU stack for both NVIDIA and AMD, aside from minimal pieces of signed firmware, is 100% open source and pure Python except for the compiler. It's not using vendor drivers, frameworks, or libraries. That's why it's so easy to make it work on Mac. For compilers, on AMD, we use upstream LLVM, and on NVIDIA, we use the NAK compiler from the MESA project. We plan to replace the compiler with pure tinygrad in a year or two as well. With RANGEIFY merged, our lowering stuff now matches the state of the art, TVM style. We're studying ThunderKittens and TileLang for speed at that level, and should have all this stuff ready in 200 days for the due date of our AMD Llama 405B training contract. Due to tinygrad's small size and pure Python nature, it's the easiest ML library to make progress on, aka fastest slope of improvement. With Megakernel style for scheduling, MODeL_opt style for planning, and E-graph style for symbolic, we should blow past the state of the art in PyTorch and JAX speed. If we do that, NVIDIA's moat is over. It's 1000 lines at most to add a new accelerator to tinygrad. And I don't mean to add a new accelerator with help from a kernel driver, compiler, and libraries. Just 1000 lines of software for the *whole* accelerator speaking right on the PCIe BARs, like what tinygrad is doing with the NVIDIA and AMD GPUs now.