LlamaIndex's company blog covers its open-source framework and LlamaParse document parsing for building LLM and agent applications over private data.
HN commenters compare it with similar tools and point out a parsing error in the sample on its LlamaParse landing page.
https://www.llamaindex.ai/ is much better IMO, but it's definitely a case of boilerplate-y, well-supported incumbent vs smaller, better, less supported (e.g. Java vs Python in the 00s or something like that). Depends on your team and your needs.
Also Autogen seems popular and well-ish liked https://microsoft.github.io/autogen/
LangChain definitely has the most market-/mind- share. For example, GCP has a blog post on supporting it: https://cloud.google.com/blog/products/ai-machine-learning/deploy-langchain-on-cloud-run-with-langserve
I'm on:
Image
- Terminus Research Group, from bghira of SimpleTuner/diffusers https://discord.gg/cSmvcU9Me9
- AI Toolkit https://github.com/ostris/ai-toolkit https://discord.gg/VXmU2f5WEU
- Stable Diffusion https://discord.gg/stablediffusion
LLM
- LLamaIndex https://www.llamaindex.ai https://discord.com/invite/eN6D2HQ4aX
- Nous research https://discord.gg/nousresearch
- LangChain https://discord.gg/hMrfPpUk
Platforms
- Replicate https://discord.gg/replicate
- Fal https://discord.gg/fal-ai
Parsing docs using LVM is the way forward (also see OCR2 paper released last week, people are having ablot of success parsing with fine tunned Qwen2).
The hard part is to prevent the model ignoring some part of the page and halucinations (see some of the gpt4o sample here like the xanax notice:https://www.llamaindex.ai/blog/introducing-llamaparse-premium)
However this model will get better and we may soon have a good pdf to md model.
The very first (and probably hand-picked & checked) example on your website [0] suffers from the very problem people are talking about here - in "Fiscal 2024" row it contains an error for CEO CAP column. On the image it says "$234.1" but the parsed result says "$234.4". A small error, but error nonetheless. I wonder if we can ever fix these kind of errors with LLM parsing.
[0] https://www.llamaindex.ai/llamaparse
Llamaparse by LlamaIndex is probably SOTA at the moment and seems to have no problems with tables. Pricing is good at the moment too.
https://www.llamaindex.ai/enterprise
There's an error right on your landing page [1] with the parsed document...
It's supposed to say 234.1, not 234.4
https://www.llamaindex.ai/llamaparse