The team building OrioleDB, a storage engine for PostgreSQL, writes about its design, benchmarks, beta releases and table engine API patches.
HN commenters link it when discussing Postgres storage engines, comparing it to Neon, and noting its patches to the table access method API.
I don't want to hijack Datadogs+Quickwit's post comment section with unrelated promotional-looking info. Quick summary below but if you have any other questions pls tag olirice in a Supabase GH discussion.
The OrioleDB storage engine for postgres is a drop-in replacement for the default heap method. Its takes advantage of modern hardware (e.g. SSDs) and cloud infrastructure. The most basic benefit is that throughput at scale is > 5x higher than heap [1], but it also is architected for a bunch of other cool stuff [2]. copy-on-write unblocks branching. row-level-WAL enables an S3 backend and scale-to-zero compute. The combination of those two makes it a suitable target for multi-master.
So yes, given that it could greatly improve performance on the platform, it is a goal to release in Supabase's primary image once everything is buttoned up. Note that an OrioleDB release doesn't take away any of your existing options. Its implemented as an extension so users would be able to optionally create all heap tables, all orioledb tables, or a mix of both.
[1] https://www.orioledb.com/blog/orioledb-beta7-benchmarks
[2] https://www.orioledb.com/docs
This is generally true, but there are some additional aspects.
1. With PostgreSQL heap, you need to access the heap page itself. And it's not for free. It goes all through the buffer manager and other related components.
2. In OrioleDB, we have a lightweight protocol to read from pages. In-memory pages are connected using direct links (https://www.orioledb.com/docs/architecture/overview#dual-pointers), and pages are read lock-less (https://www.orioledb.com/docs/architecture/overview#page-structure). Additionally, tree navigation for simple data types skips both copying and tuple deforming (https://www.orioledb.com/blog/orioledb-fastpath-search).
According to all of the above, I believe OrioleDB still wins in the case of secondary key lookup. I think this is indirectly confirmed by the results of the TPC-C benchmark, which contains quite a lot of log of secondary key lookups. However, this subject is worth dedicated benchmarking in the future.
As far as I can tell, Postgres is not designed with this inclination towards doing lighter work when clients are waiting and piling up maintenance work to do in background. I think the background work it does is mostly running vacuum on tables now and then.
Contrast that with ClickHouse, for example. It operates in a different niche than Postgres (OLAP instead of OLTP) – with their merge tree engine family [1] that does data deduplication in background.
There is one project of modernizing Postgres' storage engine called OrioleDB [2], but I think the company got acquihired by Supabase [3] and maybe the project has not been progressing very quickly since then.
[1] https://clickhouse.com/docs/engines/table-engines/mergetree-family
[2] https://www.orioledb.com/
[3] https://supabase.com/blog/supabase-acquires-oriole
Serverless in the context of Postgres means to decouple storage and compute, so you could scale compute "infinitely" without setting up replica servers. This is what Neon offers, where you can just keep hitting their endpoints with your pg client and it should just take whatever load (in principle) and bill you per request.
Supabase gives you a server that runs classic Postgres in a process. Scaling in this scenario means you increase your server's capacity, with a potential downtime while the upgrade is happening.
You are confusing _managed_ Postgres for _serverless_.
Others in the serverless Postgres space:
- https://www.orioledb.com/ (pg extension)
- https://www.thenile.dev/ (pg "distribution")
- https://www.yugabyte.com/ (not emphasizing serverless but their architecture would allow for it)
We are tracking the patches here:
https://www.orioledb.com/docs#patch-set
The actual storage engine is written as an extension - these patches are mostly to improve the TAM API. If these are accepted by the community then it should be simpler for anyone to write their own Storage extensions.
I think (correctly) it will take a lot longer to upstream the extension - the PG community should take a “default no” attitude with big changes like this. Over time we will prove its worthiness (hopefully beyond just supabase - it would be good to collaborate with other Postgres providers)
OrioleDB is new to me.
According to the docs, it “uses Postgres Table Access Method (TAM) to provide a pluggable storage engine for PostgreSQL. […] Pluggable Storage gives developers the ability to use different storage engines for different tables within the same database. Developers will be able to choose a storage method that is optimized for their specific needs: some tables could be configured for high transactional loads, others for analytics workloads, and still others for archiving.”
https://www.orioledb.com/docs