Tembo, a managed Postgres company, writes on Postgres extensions, message queues, vector search, and benchmarks.
HN commenters link its posts on PGMQ, managed Postgres in Rust, and pg_vectorize, and its CEO joins threads to discuss the product.
Reminds me of this interesting call out in Postgres 16: The exciting and the unnoticed:
> Moving towards Active Active replication: A feature was committed in Postgres 16 to allow logical replication to avoid replication loops, which is when a transaction gets replicated from source to target and back. Postgres 16 allows subscribers to process only changes which have no origin which allows you to prevent these loops. Bi-directional active-active replication is still very complicated and requires solving a lot of problems, but this feature tackles one of those important sub-problems.
https://tembo.io/blog/postgres-16/#laying-the-groundwork-for-an-exciting-postgres-17-and-beyond https://news.ycombinator.com/item?id=37613690 (18d ago, 1 comment)
The AWS submission here is quite short. There's a longer read available at https://aws.amazon.com/blogs/database/using-pgactive-active-active-replication-extension-for-postgresql-on-amazon-rds-for-postgresql/ that sheds some light on the basis for their technology:
> pgactive is based on the open-source BDR project and includes functions that let you set up an active-active PostgreSQL cluster, conflict detection and automatic resolution methods (for example, last-write-wins), conflict monitoring, and more.
They link to this old BDR project wiki page https://wiki.postgresql.org/wiki/BDR_Project , but it's quite stale. Efforts seems to have been ongoing as EnterpriseDB, https://github.com/EnterpriseDB .
That's many of the same folks…
We talk a little bit in https://tembo.io/blog/managed-postgres-rust about how we use PGMQ to run our SaaS at Tembo.io. We could have ran a Redis instance and used RSMQ, but it simplified our architecture to stick with Postgres rather than bringing in Redis.
As for scaling - normal Postgres scaling rules apply. max_connections will determine how many concurrent applications can connect. The queue workload (many insert, read, update, delete) is very OLTP-like IMO, and Postgres handles that very well. We wrote some about dealing with bloat in this blog: https://tembo.io/blog/optimizing-postgres-auto-vacuum
I did say the index, not the embeddings themselves. The index is a more compact representation of your embeddings collection, and that's what you need in memory. One approach for indexing is to calculate centroids of your embeddings.
You have multiple parameters to tweak, that affect retrieval performance as well as the memory footprint of your indexes. Here's a rundown on that: https://tembo.io/blog/vector-indexes-in-pgvector
Tembo CEO here - we are targeting feature parity for Tembo Cloud w/ RDS as soon as possible, would love to have you give Tembo a try sometime, give us feedback :)
Tembo Cloud is standard SaaS offering, and our new Tembo Self Hosted (https://tembo.io/docs/product/software/tembo-self-hosted/overview) allows you to run the same software that powers our SaaS, but in your own K8s cluster.
Like others have said, this is a control problem in prod (not using migrations in code and renaming instead of dropping). Anyways, as an fun exercise to get exactly what's asked, I wonder if this could be used https://tembo.io/blog/introducing-pg-later/ together with a trigger to trap the DROP, alert on Slack and run later if not aborted.
Few relevant blogs on using pg_vectorize:
* Doing vector search with just 2 commands https://tembo.io/blog/introducing-pg_vectorize
* Connecting Postgres to any huggingface sentence transformer https://tembo.io/blog/sentence-transformers
* Building a question answer chatbot natively on Postgres https://tembo.io/blog/tembo-rag-stack