Company blog of Quesma, a database and AI tooling startup, covering AI agent benchmarks, LLM costs, and analytics infrastructure.
The founder comments on it directly, and HN readers question benchmark details such as the telecom agent policies. HN links it for AI benchmark and tool-use posts.
Thank you for the feedback!
In this (telecom) benchmark you can review agent policies and manuals here: 1) https://github.com/sierra-research/tau2-bench/blob/main/data/tau2/domains/telecom/main_policy.md 2) https://github.com/sierra-research/tau2-bench/blob/main/data/tau2/domains/telecom/tech_support_manual.md
Of course these are just parts of the prompt, you can inspect benchamark code to see how these are rendered to actual LLM calls.
In case someone is not familiar with framework methodology I've wrote a separate article covering that (with some of my thoughts) -> https://quesma.com/blog/tau2-from-llm-benchmark-to-blueprint-for-testing-ai-agents/
"A Visualization Language for the AI Era", as a tagline, sounds weird.
I had the best success with popular and versatile packages like matplotlib and ggplot2 - even 1.5 ago (vide https://quesma.com/blog/which-chart-would-you-swipe-right/).
Now, frankly, my go-to visualization package is React, sometimes with a pinch of D3.js (e.g. https://p.migdal.pl/tree-of-tree/).
Kibana is great, but managing Elasticsearch is pain. Slow ingestion, query performance, takes a lot of space, harder to tune up.
Modern columnar SQL such as ClickHouse are 10+ times more efficient in real-world use cases.
I'm a CEO and founder of Quesma, which, let's use Kibana with ClickHouse: https://quesma.com/
Forever free, source-available license.