SoftwareMill's engineering team writes about Scala, Java, message queues, and distributed systems, with benchmarks and code examples.
HN links it for its message queue comparisons, Loom and Scala posts, and self-hosting write-ups, and one commenter calls the queue evaluation a great writeup.
Cassandra won't try and load a partition into memory. It doesn't work that way. The only way you would get behavior like that is by setting "allow filtering" to on. Allow filtering is a dedicated keyword for "I know I shouldn't do this but I'm going to anyway". If you're trying to run those types of queries, use a different database. If someone is making you use a transactional database without joins for analytical load, get a different job because that's a nightmare.
Also, your partitions should never get that large. If you're designing your tables in such away that the partitions grow unbounded, there's an issue. There are lots of ways to ensure that the cardinality of partitions grows as the dataset grows. And you actually control this behavior by managing the partitioning. It's really easy to grok the distribution of data in on disk if you think about how it's keyed.
You've basically listed a bunch of examples of what happens when you don't use a wide columnar store correctly. If you're constantly fixing downed nodes, you're probably running the cluster on hardware from Goodwill.
This is a pretty good list of what not to do with Cassandra, or any similar database.
https://blog.softwaremill.com/7-mistakes-when-using-apache-cassandra-51d2cf6df519
Could you elaborate?
It's a great writeup. But I'm personally curious about them hitting the limits of vertical scalability. They said they used the biggest node 'available to them' without clarifying how big that was.
If the issue was RabbitMQ hitting resource limits then upgrading its hardware (or swapping it out for some other similarly featureful broker but which is faster) might have fixed some of the problems too, albeit not things like insufficient observability.
I also wonder to what extent RabbitMQ suffers from being written in Erlang. Switching to a featureful broker with sharding and replication that isn't Kafka e.g. Artemis might have allowed them to avoid rewriting code to not use helpful features.
This performance blog suggests RabbitMQ may top out at ~4000 messages/sec when making things durable, which isn't especially good performance:
https://softwaremill.com/mqperf/
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
Escape analysis in OpenJDK will stack allocate values where it can show it is safe to do so. Project Valhalla is also reducing the memory footprint of objects.
As for thread safety, that is more of a language concern than a runtime one. Amongst JVM languages Scala is leading here AFAIK. Its "capture checking"[1] provides thread safety (e.g. [2]) and actually covers escape analysis as well. On Scala Native (the native code backend for Scala) capture checking can be used for safe stack allocation and safe arena allocation.
[1]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html
[2]: https://softwaremill.com/understanding-capture-checking-in-scala/
> Git sucks. Reason? It is entirely imperative.
A git repository is a persistent data structure [1]. Operations are typically append-only, requiring --force flags to do any in-place mutation.
> And, well, just like in C, there's no safety pillow - if you screw up a call, something simply goes pear-shaped.
Git is the safety pillow. When you screw up your code, you restore it with git. Article doesn't mention the reflog, which is the second safety pillow hiding under the first.
[1] https://softwaremill.com/persistent-data-structures-in-functional-programming/
It seems that you are mainly interested in how to build preview environments for your app, these posts describe an approach to get there:
- https://softwaremill.com/preview-environment-for-code-reviews-part-1-concept/
- https://softwaremill.com/preview-environment-for-code-reviews-part-2-ci-cd-pipelines-for-the-cloud/
- https://softwaremill.com/preview-environment-for-code-reviews-part-3-reverse-proxy-configuration/
While the examples use Gitlab, it shouldn't be very hard to port the same idea to a Bitbucket.
Yes indeed. Attaching below a couple of references from my exploration:
Transactional outbox pattern:
- https://microservices.io/patterns/data/transactional-outbox.html
- https://softwaremill.com/microservices-101
- https://brandur.org/job-drain
A pattern similar to the one I mention using cache:
- https://masstransit-project.com/articles/outbox.html