import python 🐍 @importpython

One way to ruin your product in the eyes of public is to bash others who are just starting out, especially when it is uncalled for.

Ankur Gupta @getpy

2 months ago

Classic Guardrail Tiers Visually Explained. Created for a talk. Last month learnt and did fine tuning of SLMs. Built for binary guardrails along with one for intent classification. The sheer landspace of types/classficiation of guardrails is wide, with passing time large

0 2 6 2K 8

Everyone is a Python Developer until ...

Ankur Gupta @getpy

4 weeks ago

With LLM everyone is a dev, until it breaks in prod.

1 2 16 2K 0

AGI is here, Autonomous GitHub Issues

Charlie Marsh @charliermarsh

a month ago

My agent is putting up PRs I don't understand on repos I haven't even heard of

43 69 2K 104K 86

Join Pauline van Nies on Sept 10 to see how she uses DSPy, knowledge distillation, and LangGraph agents to bridge this gap. 🎟️ Tickets: pretix.eu/pydata/amsterd…

@alxshp Cc @ankitml

Vibe coded a software that overlays map tiles for a large area from across years and then lets you play them and see how city changed/evolved. Tech Stack - Sentinel-2 L2A imagery (ESA/Copernicus) via the AWS Open Data mirror. - OpenStreetMap via the Geofabrik GCC-states extract, clipped to Dubai. - OSM processing: osmium-tool, pyrosm, geopandas/shapely - Viewer: MapLibre GL JS Almost *90 GB of data. You can forecast or answer with reasonable accuracy things like - How many years will it take before city runs out of land - You can make reasonable assumptions on which outskirt areas will grow and at what pace?. Or the empty land patch between the said outskirt area and the town before will be filled in how many years or decades. - You can find what population range a given area or city can support People in future will build a lot of throw away software aka for one time consumption.

I've been on a client consulting project since mid-March that uses LLMs to generate SQL. Over the last 6 months, the accuracy of LLM-generated advanced SQL (think 1000–1500 lines in one massive query, with multiple joins, window functions, aggregations, and recursive CTEs) has improved a lot in our internal benchmark. The spider2-sql.github.io leaderboard shows the same trend. Google hasn't given us access to Gemini-SQL2 yet, so I can't speak to that one. But overall Gemini 3.1 is simply the best, no matter the database and claude just sucks - at least from what we see. I'm from the pre-ORM generation. We wrote SQL by hand, and DBMS with a lab was a college subject, so almost everyone learned it. Still, writing highly optimised queries was always an acquired skill. Every org had a few senior devs who could write or debug those big complex queries, and it was a no-go zone for everyone else. You'd usually find them in the performance-critical parts of APIs. Looking at some of the SQL these LLMs generate gives me that same feeling of dread. You spend a few hours just understanding it, then test it, then wonder whether to break it into smaller queries so you can actually feel confident shipping it to prod.

Classic Guardrail Tiers Visually Explained. Created for a talk. Last month learnt and did fine tuning of SLMs. Built for binary guardrails along with one for intent classification. The sheer landspace of types/classficiation of guardrails is wide, with passing time large corporates will have an assortment of purpose built guardrail models and an API layer on top of it, they will enforce all prompts+context window to pass input/output just like we do WAF today. No code tool purpose made to create guardrail models easily will be norm.

In 2015 I formed a small group of engineers at Jane Street to rebuild the firm’s core trading system from the ground up, and we ended up cutting latency by two orders of magnitude. Some of the techniques we used, relevant for algorithmic trading systems and exchanges today: Zero allocation: Whenever a program allocates memory for an object on the heap, the runtime pays a steep penalty in latency. The simplest solution is to avoid memory allocation entirely. Jane Street famously uses OCaml, a strongly typed programming language that by default produces garbage collected by a dynamic collector. Most other firms use languages with manual memory management, but it was a strict part of Jane Street’s tech culture that all risk-sensitive code had to be written in OCaml. It took a collaborative effort across multiple groups within Jane Street’s technology org to create zero-allocation core libraries, combining the type safety of a functional programming language with the memory profile of a language like C. We built the new main trading loop in this hybrid OCaml/C-style, producing zero new allocations in the critical path from tick to trade. In modern languages like Rust, it is substantially easier to achieve precise memory management while still benefiting from type safety and compile-time guarantees. Kernel bypass: A primary goal of a low-latency trading system or exchange is to pull a network packet containing market data or order flow through the network card’s interface and into the program’s memory space as fast as possible. The standard Linux OS kernel uses slow abstractions to support a wide variety of network drivers, at the expense of the entire system’s end-to-end latency. When we started with an empty program that contained no business logic and only forwarded packets through when received, the end-to-end latency was already too slow. To fix this issue, we employed a standard practice in the HFT industry in which we bypassed the OS’s kernel stack entirely by leveraging our network card vendors’ proprietary APIs to DMA packets straight from the NIC into memory. This technique brought our empty-packet-forwarding baseline into the latency regime we needed in order to build out the rest of the trading, risk, and protocol code. Local IPC: Kernel bypass is necessary when reading routed packets off a network from a third party such as another exchange or client connection. When communicating between internal instead of external processes, the fastest transports avoid network stacks entirely. Processes within the same box can transfer messages using shared memory or Unix domain sockets. This allowed us to continue with our familiar process boundaries for separable components without sacrificing significant performance. We had to write custom logic to emulate many of the features of network- and transport-layer protocols, with the result of creating a reusable, zero-overhead IPC mechanism. Working on this problem was one of the most intellectually rewarding experiences of my early career. The above latency optimization techniques are fairly commonplace in the HFT trade but hard to learn outside the industry setting. Half of our team at Architect comes from Jane Street and other trading firms, and we value using our domain knowledge to build exchanges for the public rather than trading software that never leaves an HFT’s walls.

Under the Hood: The LLM Engineering Manual - the 2026 Version 2 update is live. 36 projects that build every layer of a large language model from scratch, from a scalar .backward() all the way to a production serving stack. One rule throughout: build it, break it, measure it. ~1,030 pages, and every number traces to a primary source. What's new in v2 A brand-new project: Diffusion & Non-Autoregressive Decoding. You build a decoder by hand, on a laptop, that writes a sentence the way you fill a crossword, committing the words it's surest of first instead of strictly left to right. Then you break it and watch coherence collapse when you rush the step count. 12 chapters got surgical 2026 updates, each fact-checked to primary sources: • KV cache quantized to ~3 bits + fp8 serving • speculative decoding: block-diffusion drafters & multi-token prediction • hybrid SSM/attention models (Jamba, Mamba-2): mostly-linear stacks with a thin spine of attention • inference-time compute, best-of-N, and RL with verifiable rewards I ran the new fp8-serving labs on real hardware (an NVIDIA GB10) to confirm the claims reproduce: fp8 KV holds ~2.07x the token budget of bf16, and a causal drafter composes with it for ~1.9x throughput at 71% draft acceptance. Charts + raw data are in the free code repo. In addition, many of the items called out by users in v1 have been rewritten/fixed. Already own it? It's a free update. Just re-download the PDF/EPUB. Book: leanpub.com/under-the-hood Code, all 36 projects: github.com/mechramc/Under… Next step: Amazon KDP (In progress) - drop a like, comment or retweet if you liked the updates

keyboard_arrow_left Previous keyboard_arrow_right Next

we recently rolled out new docs on fault tolerance for agents everything from handling transient errors to engaging a human in the loop when information is missing here's a guide on how to handle errors in agentic systems: docs.langchain.com/oss/python/dee…

At 38, I had 17 years of program management under my belt. PgMP, PMP, enterprise SaaS. Zero machine learning experience. At that time, as everyone does, I started doing some Coursera courses but everything was vague. There were two things that helped - @karpathy and @rasbt. Once the concepts were broken down - I could build, break, and eventually come up with 100s of ideas that failed. I decided to pivot into AI research. Not a side hobby. A real pivot. I spend to this day at least a few hours every week reading latest papers and trying to break the implementation of the paper. Even now, I am participating in the ICLR huggingface paper reproduction challenge. Two years later at 40: 6 papers on arXiv, an NVIDIA Inception membership, founded Murai Labs, and an open source Tamil language model that is coming out soon. The book grew directly out of what I needed to build TamilLM from scratch. If it helped me learn how to ship a real model with no ML background, it can help you too. It is such a beautiful coincidence that TamilLM finished pretraining and Under The Hood launched on Amazon on the same day - it was not planned. (Kadavul irukaan Kumaru moment) 100+ copies sold on Leanpub. #1 bestseller there. Now available worldwide on Amazon. 8 free chapters at murailabs.com/under-the-hood (Chapters 4, 5, 8, 13, 18, 23, 27, 32). India Kindle: amazon.in/dp/B0H9FYVK3W US Kindle: amazon.com/dp/B0H9FYVK3W US Paperback: amazon.com/dp/B0H9FWPCDV Leanpub: leanpub.com/under-the-hood I started late. But not done yet - I am glad I started. 🙏

Given how little people know their own code.

Ankur Gupta @getpy

2 months ago

<s>IN CLAUDE WE TRUST</s>

keyboard_arrow_left Previous keyboard_arrow_right Next

2 0 13 2K 0

Talking on @LangChain deepagents at @__bangpypers__ lets talk if you are around.

BangPypers @__bangpypers__

2 months ago

🎤 Final Speaker Announcement! We're excited to welcome @getpy to the July BangPypers Meetup! Learn how to build long-running LLM agents with task planning, long-term memory, file systems, and human-in-the-loop workflows.

1 3 11 2K 0

Phases of MCP Integrations for most developers in real life ( unless you never grew beyond 20-30 tools in context window ) - Ya, MCP will solve all my problems it's such a simple protocol - Wow, Why is MCP auth so awful?. - Too many tools are polluting my context and LLM is invoking the wrong tool - I have a routing layer now to only inject subset of tools that relevant in conjunction to the prompt - Why is my harness executing tools in wrong sequence before figuring out the right sequence and this is of material consequence with token consumption - I now have a plan mode where am asking LLM to share sequence of tools it will call to solve the problem and a bunch of heuristics and rule engine to verify sequence - The above is not working as model discovers a new detail that results in updating the plan and todo(s) and RLM just made a reactive discovery in last turn, - I am seeing better reliability now with off the shelf harness now ( claude agent sdk, google adk, etc etc ) - Wow, I just switched my model coz token cost is a crazy / older lite model has significant cost savings. Shit I am seeing a new set of behaviour ( this old model is so stupid ) I am again writing evals and updating some of the my prompts - I just learnt about preference tuning on ordering pairs and boy this should solve my troubles ... and it continues .... We will continue to have our job for more time.

Is there something postgres can't do

alex shapalov @alxshp

2 months ago

PostgreSQL 19 introduces graph-style queries. Instead of manually connecting table after table with joins, you can describe the path through your data: customer → bought → product ← bought ← similar customer → follows → brand Useful for recommendations, access control,

27 143 1K 131K 664

For domain specific agents that operate on a narrow knowledge base or are focused on one function only, the lowest hanging fruit via HITL is to nudge them to refine the first prompt with suggestions if prompt is missing basic information. Just this simple nudge is all you need to get the user to cultivate a habit of being detailed oriented ( cough cough not be lazy ) and save all the tokens downstream. One can accumulate top-N expensive traces from logs, their first prompt, their initial set of N tool executions and skills loaded post prompting. Then do a batch run getting LLM to be critical of the prompts based on the traces telemetry to mine what information could have been provided at get go. You start seeing common features emerge. You can then bundle it in a single LLM as a judge invoked only for the first prompt. Below screenshot is the simple example to drive the point home. Imagine the domain being subfields in medicine, telecom, insurance, banking etc. This works well for domain specific agents that aren't very broad based in their scope. User perceives the Agent to be smart to callout the obvious and in virtually all cases help new users get faster and intended results. The one LLM call for every prompt is definitely a fixed cost and lately what I have realised is that you disable per user once they hit N prompts in sequence above certain score on clarity. You can see it in subsequent user specific traces they get better at prompting as this HITL to refine the prompt has taught them to provide basic information.

x.com/i/article/2066…

`deno desktop` has landed in main. You can try it out by running `deno upgrade canary` - Mac, Linux, and Windows support - Can generate pkg and msi installers - Supports cross-compile (generate .exe from mac) - Supports chrome (CEF) or native Webview for smaller binaries