Together AI Blog

K3 is highly competitive. They're essentially tied here:

> https://www.together.ai/blog/kimi-k3-vs-claude-fable-5-on-deepswe-cost-and-coding

And they're very very close here:

> https://artificialanalysis.ai/models

And that's if you don't take cost and guardrails into consideration. If you do, K3 beats Fable hands down.

That aside, I'm not an insider, and I can't really say when K3 completed training. What we do know is that it released to the public in mid-July, about a month ago. Fable 5's initial botched release was in mid-June, and it was re-released in early July.

> No one is going to release highly powerful near-future models for the same reason no one releases how to make physically compact but megaton yield fusion devices

So there's essentially an open Mythos-class model right now. Near-Mythos-class if you really want to quibble. Maybe past some threshold such models won't release to the public at all, or will release only in closed form, but we cannot predict when that threshold will be crossed.

Fortunately for the world, it remains politically advantageous for China to support open model development. It's a winning strategy, especially in the long-term. Besides, I don't think a Uyghur Underground would be able to acquire the hardware to run a K3-class model, to say nothing of a Skynet-class AI.

Looks awesome! Been doing a lot of OCR recently, and love the addition to the space. The reigning champion in the PDF -> Markdown space (AFAIK) is Facebook's Nougat[1], and I'm excited to hook this up to DSPy and see which works better for philosophy books. This repo links the Zerox[2] project by some startup, which also looks awesome, and certainly more smoothly advertised than Nougat. Would love corrections/advice from any actual experts passing by this comment section :)

That said, I have a few questions if OP/anyone knows the answers:

1. What is Together.ai, and is this model OSS? Their website sells them as a hosting service, and the "Custom Models" page[3] seems to be about custom finetuning, not, like, training new proprietary models in-house. They might have a HuggingFace profile but it's hard to tell if it's them https://huggingface.co/TogetherAI

2. The GitHub says "hosted demo", but the hosting part is just the tiny (clean!) WebGUI, yes? It's implied that this functionality is and will always be available only through API calls?

P.S. The header links are broken on my desktop browser -- no onClick triggered

[1] https://facebookresearch.github.io/nougat/

[2] https://github.com/getomni-ai/zerox

[3] https://www.together.ai/products#custom-models

"Together with architectural refinements, our Mamba-3 model achieves significant gains across retrieval, state-tracking, and downstream language modeling tasks. At the 1.5B scale, Mamba-3 improves average downstream accuracy by 0.6 percentage points compared to the next best model (Gated DeltaNet), with Mamba-3's MIMO variant further improving accuracy by another 1.2 points for a total 1.8 point gain. Across state-size experiments, Mamba-3 achieves comparable perplexity to Mamba-2 despite using half of its predecessor's state size. Our evaluations demonstrate Mamba-3's ability to advance the performance-efficiency Pareto frontier."

It's apparently optimised for inference efficiency

Additionally:
https://www.together.ai/blog/mamba-3

"Mamba-3 SISO beats Mamba-2, Gated DeltaNet, and even Llama-3.2-1B (Transformer) on prefill+decode latency across all sequence lengths at the 1.5B scale."

Yeah, the synchronous updates are a big deal. To get an idea of how much bandwidth this typically takes, Oracle has 1600gbps per node of interconnect with latency very low, guessing in the tens of microseconds. A really good home connection might have 1gbps of interconnect with latency in the tens of milliseconds. The big question is whether we really need all this interconnect -- GPT-JT[1] is a very promising step in this direction. The idea is that we just drop most of the gradient updates and it still works well. Unclear whether this will take off generally -- if it does it would be a huge deal, because 1600gbps of interconnect is very expensive.

[1]: https://www.together.xyz/blog/releasing-v1-of-gpt-jt-powered-by-open-source-ai

Memory is part of compute. I have no idea why you've separated them.

>How exactly are you going to find enough data to feed these monstrous petabyte scale models? You won't! Your models will stay small and become cheap over time!

People have this idea that we're stretching the limit of tokens to feed to models. Not even close.

This is a 30T token dataset https://www.together.ai/blog/redpajama-data-v2

It's just web scrapped which means all the writing largely inaccessible on the web (swaths of fiction, textbooks, papers etc) and in books are not a part of it. It doesn't contain content in some of the most written languages on earth (e.g Chinese. You can easily get trilions of tokens of Chinese alone)

Their models are much smaller: 1T vs 5T for the frontier models. 1T is Sonnet/Google Flash size, not Opus size.

The $0.87/M tokens price for Mimo Pro is probably subsidized.

Mimo models aren't widely available on western providers, but Kimi and Deepseek are similar sizes and cost about the same to run. They are priced $3-$4/M tokens (which is right were Google's very confused range of Flash models are priced at: between $0.40/M tokens and $9/M tokens depending on exactly which model - and you don't want the $9 one!).

Anthropic overprices Sonnet (probably because of their capacity issues). GPT 5.4 mini is $4.50/M tokens.

https://docs.fireworks.ai/serverless/pricing

https://www.together.ai/pricing