RisingWave Blog

As others have mentioned, this acquisition leaves many Rockset customers in a tough spot with a short timeline to migrate. I'd like to bring attention to a potential alternative: RisingWave(https://risingwave.com/). RisingWave is an open-source streaming database designed for real-time analytics and data processing. Like Rockset, it offers PostgreSQL compatibility and impressive ability to handle both streaming and batch data.

What sets RisingWave apart is its focus on stream processing while maintaining SQL compatibility. This could be particularly valuable for users leveraging Rockset's real-time capabilities. RisingWave offers several features that may appeal to Rockset users. It's built to scale in cloud environments and can ingest data from a large variety of sources. The database supports materialized views for efficient query processing and ensures data consistency with ACID transactions. For those concerned about vendor lock-in after this experience, RisingWave's open-source nature (Apache 2.0 license) provides an extra layer of assurance. There's also a managed cloud offering for those who prefer a hands-off approach.

I encourage impacted Rockset users to explore RisingWave as part of their evaluation process. The project has a welcoming community(join at risingwave.com/slack) and extensive documentation to help with the transition. [Disclosure: I'm associated with RisingWave. Happy to answer any questions or provide more details about how it compares to Rockset…

I am the founder of RisingWave (http://risingwave.com/), an open-source SQL streaming database. I am happy to see the launch of Warpstream! I just reviewed the project and here's my personal opinion:

* Apache Kafka is undoubtedly the leading product in the streaming platform space. It offers a simple yet effective API that has become the golden standard. All streaming/messaging vendors need to adhere to Kafka protocol.

* The original Kafka only used local storage to store data, which can be extremely expensive if the data volume is large. That's why many people are advocating for the development of Kafka Tiered Storage (KIP-405: https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+Storage). To my best knowledge, there are at least five vendors selling Kafka or Kafka-compatible products with tiered storage support:

-- Confluent, which builds Kora, the 10X Kafka engine: https://www.confluent.io/10x-apache-kafka/;

-- Aiven, the open-source tiered storage Kafka (source code: https://github.com/Aiven-Open/tiered-storage-for-apache-kafka);

-- Redpanda Data, which cuts your TCO by 6X (https://redpanda.com/platform-tco);

-- DataStax, which commercializes Apache Pulsar (https://pulsar.apache.org/);

-- StreamNative, which commercializes Apache Pulsar (https://pulsar.apache.org/).

* WarpStream claims to be "built directly on top of S3," which I believe is a very aggressive approach that has the potential to drastically reduce costs, even compared to tiered…

Sure! I mentioned a few orthogonal concepts that go well together, and each of the following examples has a different combination that they employ:

- the company that developed Madsim (RisingWave) [0] [1] is tries hardest to eliminate non-determinism with the broadest scope (stubbing out syscalls, etc.)

- sled [2] itself has an interesting combo of deterministic tests combined with quickcheck+failpoints test case auto-discovery

- Dropbox [3] uses a similar approach but they talk about it a bit more abstractly.

Sans-IO is more documented in Python [4], but str0m [5] and quinn-proto [6] are the best examples in Rust I’m aware of. Note that sans-IO is orthogonal to deterministic test frameworks, but it composes well with them.

With the disclaimer that anything I comment on this site is my opinion alone, and does not reflect the company I work at —— I do work at a rust shop that has utilized these techniques on some projects.

TigerBeetle is an amazing example and I’ve looked at it before! They are really the best example of this approach outside of FoundationDB I think.

[0]: https://risingwave.com/blog/deterministic-simulation-a-new-era-of-distributed-system-testing/

[1]: https://risingwave.com/blog/applying-deterministic-simulation-the-risingwave-story-part-2-of-2/

[2]: https://dropbox.tech/infrastructure/-testing-our-new-sync-engine

[3]: https://github.com/spacejam/sled

[4]: https://fractalideas.com/blog/sans-io-when-rubber-meets-road/

[5]:…

It's very useful any time the input to some system is a stream of events, potentially from a whole bunch of different sources, but you want the output to be a unified relational data model.

I used to work in insurance, and we had a whole bunch of systems of record for different functions of the business -- CRM, policy management, billing, claims, etc. Some were our own tech, many were SaaS. It's great to be able to keep these systems decoupled operationally. That way, you can replace pieces and have your business areas have fairly independent IT stacks.

But many backoffice tasks, like finance, accounting, and servicing need a holistic view of what's going on. It's helpful to ingest all the data into a centralized warehouse, and build up a unified model of the state of the business. A lot of analysts like to write these data transformations in SQL.

Insurance is not a fast-paced business, so we largely ingested the data in structured form. But you can imagine that for faster businesses, like advertising, monitoring, IoT, or trading, the data from the systems of record might be an event stream, rather than a data model. These stream processing databases are designed for this type of situation, where you may want real-time ETL, event-by-event.

EDIT: Also, their website has a use cases section: https://risingwave.com/use-cases/