Arize's company blog covers LLM observability, evaluation, and ML engineering, including tracing for Claude Code and Arize Phoenix.
HN links it for its Claude Code observability and tracing post, which a commenter wants to connect to OpenTelemetry tools like Arize Phoenix.
> building LLMs is very costly, and that will probably remain so for quite some time.
Building LLMs is dropping in cost quickly. Back in mid 2023 training an 7B model had already dropped to $30K and it's even cheaper now.
> I think it is equally likely that LLMs will perform worse in the future, because of copyright reasons and increased privacy awareness, leading to less diverse data sets to train on.
I'll bet a lot of money this won't happen.
Firstly copyright isn't settled on this. Secondly people understand a lot more now about how to use less, higher quality data and how to use synthetic data (eg MS Phi series, Persona dataset etc and of course the upcoming OpenAI Strawberry and Orion models which use synthetic data heavily). Thirdly the knowledge about how to use multi-modal data in your LLM is much more widely spread, which means that video and code can both be used to improve LLM performance.
[1] https://arize.com/resource/mosaicml/
And yet it's a major part of the overall concept of being responsible with our use of AIs. Throwing our hands up in the air and prematurely declaring defeat is not an option long term.
It's a non-starter for no other reason than potential copyright infringement means the government becomes involved, and they will stomp on the AI mouse with the force of an elephant - the opinions of amateurs and the anti-copyright movement notwithstanding.
As such, AI Observability is a problem that's both under active research, and the basis for B2B companies.
https://censius.ai/wiki/ai-observability
https://towardsdatascience.com/what-is-ml-observability-29e85e701688
https://whylabs.ai/observability
https://arize.com/
We (Dylan, Alex, Adam) built this to answer “what actually happened?” in Claude Code runs to debug things like nested tool calls, prompt bloat, token/cost spikes... It’s an open proxy on LiteLLM that emits OTEL + OpenInference spans to Arize-Phoenix.Giving you access to Full traces with internal prompt content, tool calls, streaming chunks, costs, and latency. No signup; works with the Claude Code CLI unchanged; MIT-licensed.
Any feedback welcome, hopefully folks find it useful. Full blog: https://arize.com/blog/claude-code-observability-and-tracing-introducing-dev-agent-lens/
On reflection this thread is pretty clearly of general interest and my comment was more jargon that language, I hang out in ML zones too much.
For a broad introduction to the field Karpathy's YouTube series is about as good as it gets.
If you've got a pretty solid grasp of attention architectures and want a lively overview of stuff that's gone from secret to a huge deal recently I like this treatment as a light but pretty detailed podcast-type format: https://arize.com/blog/mistral-ai
That looks great! Any plans on allowing exports to OpenTelemetry apps like Arize Phoenix? I am looking for ways to connect my Claude Code using Max plan (no API) to it and the best I found was https://arize.com/blog/claude-code-observability-and-tracing-introducing-dev-agent-lens/, but it seems kinda overweight.
How do you compare to a platform like Arize? https://arize.com/