Sam Crowder @samecrowder

Head of Product, LangSmith at @LangChain 🚀 | prev: @Harvard MS/MBA, @RocksetCloud (acq. OpenAI), @BainCapVC, @ContraryCapital San Francisco, CA Joined September 2017

Tweets

538
Followers

1K
Following

636
Likes

5K

Sam Crowder @samecrowder

a month ago

Online evals are difficult to set up correctly. Even if you know what you want to look for, it's an optimization question of which model to choose, how to tune the prompt, and how to think about the cost trade off of adding more LLM spend on top of each agent run. LangSmith Tuned Evaluators makes this much easier!

LangChain @LangChain

a month ago

Introducing LangSmith Tuned Evaluators They automatically score agent behavior in production, starting with Perceived Error. Perceived Error is one of the clearest signals that your agent is giving users a helpful experience. In our benchmark, our specialized model

10 6 43 32K 26

2 2 15 3K 3

View Details

Harrison Chase @hwchase17

a month ago

some great new LangSmith docs on traces vs threads vs trajectories (new concept) observability data is no longer just for observability - its also for memory & learning having a really clear mental model of this data is incredibly helpful! docs.langchain.com/langsmith/obse…

10 17 119 23K 115

View Details

Sam Crowder @samecrowder

2 months ago

I couldn’t agree more!

Sam Altman @sama

2 months ago

also, a reason to favor open-source harnesses.

450 147 6K 1.3M 308

0 0 5 408 0

View Details

Julia Schottenstein @j_schottenstein

3 months ago

the most common FUD our competitors throw against LangSmith is that it only works if you use our open source. but actually as of this month, a MAJORITY of our active self-serve customers build without a LangChain open source framework / harness. LangSmith is model, framework, and cloud neutral, and we have the numbers to show it!

1 6 26 16K 1

View Details

David Duong @__dqbd

3 months ago

Trace AI SDK 7 operations with LangSmith!

AI SDK @aisdk

3 months ago

Trace AI SDK operations across your app by registering telemetry once and sending traces to your observability provider.

1 0 30 8K 5

1 6 17 4K 2

View Details

Sam Crowder @samecrowder

3 months ago

a great thread about online evals and why everyone should be running them!

Adam Łucek @AdamRLucek

3 months ago

Importantly, this lets us monitor the agent's live performance (akin to observability), see trends over time, and get alerted when a metric drops below a threshold. Combining heuristic functions and LLM-as-a-judge implementations lets us capture both hard and fuzzy metrics from

2 1 6 462 2

0 1 3 161 0

View Details

Sam Crowder @samecrowder

4 months ago

increasingly a company's ability to scale AI systems is limited by willingness to pay for tokens at the high end but you can use langsmith tracing and experiments to optimize costs and get the right balance. our team enjoyed collaborating with @harvey on this one!

LangChain @LangChain

4 months ago

x.com/i/article/2061…

8 20 109 60K 187

1 0 1 124 0

View Details

swyx @swyx

4 months ago

@bentannyhill @Zach_Kamran Langsmith Engine is the "Full Self-Driving" moment for AI Engineering

8 10 89 32K 67

View Details

Sam Crowder @samecrowder

4 months ago

LLMs clearly make for an amazing business today, but they are also the fastest depreciating asset in history. And open models have a lot to do with that. This is why the labs are moving more into the application and deployment layers. At LangChain, we love working with teams building on both closed and open models!

LangChain @LangChain

4 months ago

The latest finding in the LangSmith Signal: Open Models are having a moment. 1 in 3 AI teams ran an open-weights model in April 2026, up from 1 in 5 nine months ago. The overall number of teams using open weights grew 3x. We’re seeing newer users choose open models at a

17 33 124 69K 39

0 0 2 208 0

View Details

Adam Łucek @AdamRLucek

4 months ago

Trace data is literally worth its weight in gold these days, if you know what to do with it! As has been established, creating effective agents requires shipping early, observing behavior, and iterating quickly. At the core of this are your agent traces capturing exact inputs, outputs, steps, and metadata along the way. Analyzing traces helps surface inefficiencies and areas for improvement, but they can also be used in more sophisticated ways to set up robust evaluations. Here's two of the ways we use traces to build evals for production agents 👇

12 22 157 42K 268

View Details

Palash Shah @palashshah

4 months ago

everyone is talking about self-optimizing loops in software & agents. but what does that actually mean? in my mind, it's a system that observes it's own outputs, evaluates them, and uses that signal to improve itself in the future. the reason why it has become so popular now, is because the evaluation step is finally reliable with llms. this wasn't really the case a year ago. this is why i'm so bullish on langsmith engine. we've incorporated a ton of different concepts that allow developers to invest in this self-optimizing loop that makes the improvement flywheel spin faster & faster. some examples of this include > feedback you leave on traces are automatically triaged > every fix that we suggest has an online evaluator, so you never regress > we create offline evals that you can add to your test suite > we continually learn on your preferences, and tune our evaluation & fix step based on this and we're seeing crazy adoption, and lots of growth across our customers. it is truly something that just gets better the more time you spend on it.

12 13 87 52K 75

View Details

Arjun Natarajan @arjunnatarajan_

4 months ago

one of the best decisions i made was to start tracing my claude code sessions into LangSmith. Has been a game changer to be able to share my conversations, track usage patterns, monitor cost. And w new messages view its all suuuuppper easy to parse. def recommend trying it

6 7 44 9K 31

View Details

Sam Crowder @samecrowder

4 months ago

a few months back, it become clear to us that a large part of technical work would be driven by agents in the future. coding agents were becoming ubiquitous and highly capable. since we build a platform for technical users, we needed to update our beliefs and strategy accordingly! LangSmith Engine automates the improvement of agents by looking through recent traces and finding problems according to a taxonomy of common agent issues that we have defined. we launched the product at our annual conference last week and the reception so far has been very exciting. and we're just getting started 📈

Benjamin Tannyhill @bentannyhill

4 months ago

langsmith engine...

4 6 41 37K 4

3 2 17 3K 10

View Details

Viv @Vtrivedy10

4 months ago

We’re hiring for Labs! 🧪 If you’re interested in working with us to push forward Continual Learning, pls DM me with a blurb + link to the best Applied Research you’ve done (or even better shipped!) you’d be a good fit if you have some previous research background and are excited to build real experiments on: - understanding massive amounts of Agent Trace data - building + updating Environments over time - Harness Eng + Post-training over long time horizons

Harrison Chase @hwchase17

4 months ago

x.com/i/article/2054…

24 39 356 154K 293

11 28 314 50K 254

View Details

Benjamin Tannyhill @bentannyhill

4 months ago

Agent observability is a means to an end: making your agent better. But observability and evals tools have traditionally failed to connect traces to meaningful actions. Agent engineering teams are left combing through traces, guessing at root causes, and writing evals manually. We built Engine to close the agent improvement loop. Engine monitors your agent's traces, creates ready-to-merge fixes, and writes evals. Now every trace becomes a fix, an eval, and a better agent.

11 16 57 11K 52

View Details

Julia Schottenstein @j_schottenstein

5 months ago

All aboard!

2 5 30 7K 0

View Details

Sam Crowder @samecrowder

5 months ago

very excited for this one! a year ago, most of what was being traced to LangSmith was "LLM apps". now everything is becoming an agent. with that shift, it's getting harder to know what your software is doing from a UI, and more important to automatically assess quality/perf/security and other dimensions you care about we're launching evaluator libraries today!

LangChain @LangChain

5 months ago

New in LangSmith Evaluation: ✅ Evaluator template library ✅ Reusable evaluators Everything you need to know → langchain.com/blog/reusable-…

1 6 40 12K 24

0 2 15 4K 8

View Details

Julia Schottenstein @j_schottenstein

5 months ago

ty to SF transit for letting us put this on a bus

0 1 19 2K 2

View Details

LangChain @LangChain

5 months ago

🌉Join us in SF for a meetup on building better agents with the agent and code improvement loops. You'll hear talks by @samecrowder, Head of Product at @LangChain, and @nnennahacks, AI Developer Relations Lead at @QodoAI . Sam will walk through the agent improvement loop and how teams use traces as the foundation for continuous improvement. Nnenna will present how Qodo is building agents, their architecture, and how they’re enhancing them using LangSmith. 🗓️ Wed, April 29 | 🕕️ 6 PM | 📍 SF (SOMA) RSVP 👉️ luma.com/4nu6vpsh

1 4 16 3K 1

View Details

Sam Crowder @samecrowder

6 months ago

as always, it's an exciting time to be working at LangChain!

LangChain @LangChain

6 months ago

LangSmith 🤝 San Francisco You don't know what your agents will do until you actually run them. What works in demos can break in the real world. Without tracing and evals, you're just guessing at why. Track what your agent actually does. Optimize and fix your agents. Then