Google Open Source Blog

Congrats on the milestone! I've been following for a while - awesome to see the progress you've made.

Q: What are your plans for sharding / scale-out?

At Notion, we run ~480 logical schemas spread across ~32 Postgres databases [1]. Our data model [2] (sorry for the blog post spam) has a recursive / graph-like structures, and we make use of columns or jsonb attributes like `
{ table: Table, id: UUID }` which sounds like your polymorphic links feature. It seems like EdgeDB's model lines up well with how we already use our database.

I did a quick cmd-f here, on the linked announcement, and in your docs looking for "shard" and "scale" but didn't find any relevant results. Postgres needs a Vitess!

Q: Do you have plans to support EdgeQL embedding or SQLite?

I am always looking for a way to compile "better than SQL" languages down to SQL. I like Datalog in this area because it's a composable way to define relationships/facts/derivations but no one is putting serious business effort to this idea (honorable mention to logica [3]). EdgeQL also fits the bill -- queries aren't logical, but they are composable -- plus looks easier to teach than most Datalog variants.

I took a peek in the repo and saw that a few EdgeQL components are written in Rust [4]; are you considering porting more logic to Rust? That would make EdgeQL much more embeddable - it could run in WASM or linked into an Android/iOS app. My pie in the sky dream is to use a single composable query/logic language to…

For maybe a single weeks worth of investment you could pick up Dart (the learning curve is tiny) and have that exact same value proposition of write once and run anywhere but with many other substantial advantages on top of it.

Here’s a completely arbitrary one from just yesterday that focuses on supply chain security (https://opensource.googleblog.com/2022/06/Dart-and-Flutter-enable-Allstar-and-Security-Scorecards.html) but this kind of thing is happening on an almost weekly basis in the background around Dart and Flutter. I don’t think people understand what’s happening behind the scenes on these projects but they are developing some very solid fundamentals to build on top of into the future.

Seriously take a look at this basic language tour, it’s incredibly familiar for anyone with an OO background out of the box https://dart.dev/guides/language/language-tour

> New Relic did this for the Rails revolution, Datadog did it for the rise of AWS, and Honeycomb led the way for OpenTelemetry.

I find this reading of history of OTel highly biased. OpenTelemetry was born as the Merge of OpenCensus (initiated by Google) and OpenTracing (initiated by LightStep):

https://opensource.googleblog.com/2019/05/opentelemetry-merger-of-opencensus-and.html?utm_source=chatgpt.com

> The seed governance committee is composed of representatives from Google, Lightstep, Microsoft, and Uber, and more organizations are getting involved every day.

Honeycomb has for sure had valuable code & community contributions and championed the technology adoption, but they are very far from "leading the way".

For what it's worth, I reached out to the Open Source Security Foundation (OpenSSF) after their press release on "Securing Critical Open Source Projects" [0], because my open source project is on their "critical list" (Python top #200) [1].

After some clicking around, I was directed to the OpenSSF mailing list [2].

…where my request was left to rot, without any response.

Made me wonder whether this is just a PR stunt / corporate power grab.

[0] https://opensource.googleblog.com/2020/12/finding-critical-open-source-projects.html

[1] https://news.ycombinator.com/item?id=25381397

[2] https://groups.google.com/g/wg-securing-critical-projects

>they should have two formats anyway

Good thing JXL has the best progressive support of any image format available! [0, 1] Something which you can only find partially with incremental decoding in WebP and doesn't really exist in AVIF (just one of the many limitations of being bolted onto a video codec). You might often reconsider having both versions of a file with this capability. (which wasn't exploited much or at all with "dumber" and heavier progressive JPEGs)

0. https://opensource.googleblog.com/2021/09/using-saliency-in-progressive-jpeg-xl-images.html

1. https://jpegxl.io/articles/faq/#doesjxlsupportprogressivedecoding?

Not exactly, although CUDA is a huge topic. See https://gist.github.com/shawwn/0e524d4a7a5d8fb152a86616559cc02a for some description of the process. Basically, jax or other program is converted to XLA, https://opensource.googleblog.com/2023/05/pjrt-simplifying-ml-hardware-and-framework-integration.html then lowered to the specific architecture (which coudl be CPU, GPU, or TPU). Last time I looked, it was a horribly complicated stack with many parts changing rapidly, although with the switch to jax, things got cleaned up a bit. My personal favorite bits are the lower levels of jax, xla, and pjrt.