The VictoriaMetrics team, writing about its monitoring and logging database along with Go internals and benchmarks.
Commenters recommend VictoriaMetrics as a lower-memory alternative to Prometheus, and team members point to its logging and benchmark posts.
I'm not sure where you saw that Victoria Metrics uses object storage. It doesn't - it uses block storage and it runs completely fine on HDD, you don't even need SSD/NVMe.
There are multiple ways to deal with ingestion floods. Kafka/distributed log is one of them, but it's not the only one. In cluster mode VM is a distributed set of services that scale out independently and buffer at different levels.
Resource usage for ingestion/storage is much lower than other solutions, and you get more for your money. At $PREVIOUS_JOB, we migrated from a very expensive Thanos to a VM cluster backed by HDDs, and saved a lot. Performance was much better as well. It was a while ago, and I don't remember the exact number of time series, but it was meant to handle 10k+ VMs (and a lot of other resources, multiple k8s clusters) and did it with ease (also for everybody involved).
I don't think you have really looked into VM - you might get pleasantly surprised by what you find :) Check out this benchmark with Mimir[1] (it is a few years old though), and some case studies [2]. Some of the companies in the case studies run at significantly higher volume than your requirements.
[1] https://victoriametrics.com/blog/mimir-benchmark/
[2] https://docs.victoriametrics.com/victoriametrics/casestudies/
> I don't see why did you label Aliaksandr Valialkin, the author, an "expert". I mean, he's no dummy but what exactly makes him an expert on optimizing Go code?
https://victoriametrics.com/team/ - Let's see. Author of multiple performance-optimized libraries with a masters degree in computer software engineering and a background in highly scalable systems (as needed for adtech). Sounds pretty much like an expert for optimizing code to me.
> And I've seen this trick before so it's not some secret, expert-only knowledge.
So, if you know it it's not export knowledge or what's your argument here?
So, I decided on three locking strategies: No-Lock Optimistic locking Pessimistic locking As a default, the no-lock behavior does exactly what the name implies. Nothing. This is the default because my research shows that for 99% all of this is not an issue and every interaction at this level will slow down the whole application. Aren't the mutexes in the more modern implementations (like Cosmo [0]) & runtimes (like Go [1]) already optimized so applications can use mutexes fearlessly?
[0] https://justine.lol/mutex/
[1] https://victoriametrics.com/blog/go-sync-mutex/
I really don't get why "scrape the prometheus endpoint" is a go-to now, push model seems to be way less PITA to manage at scale.
> If you get serious about Prometheus, eventually you will want longer data retention, checkout https://thanos.io/
Any idea how it compares with https://victoriametrics.com/ ?
We're slowly looking for a replacement for InfluxDB (as 1.8 is essentially on life support), the low disk footprint is pretty big advantage here.
> If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive.
While Go allows interior pointers, I don't think what you say is true. runtime.KeepAlive was added exactly to prevent GC from collecting the struct when only a field pointer is stored. Take a look at this blog post, for example: https://victoriametrics.com/blog/go-runtime-finalizer-keepalive/
I've found VictoriaMetrics all-in-one binary to be perfect size for home at the very least for metrics gathering.
Supports Prometheus querying and few other formats for ingesting so any knowledge bout "how to get data into prometheus" applies pretty much 1:1 + their own vmagent is pretty advanced. Not related to company in any way, just a happy user.
https://victoriametrics.com/