Exoscale, a Swiss cloud provider, blogs about hosting, object storage, microservices, Java, Clojure, and DevOps.
HN mentions Exoscale mainly as a European alternative to AWS, with a founder pitching it and a job post, rather than discussing the blog.
Instead of arguing back and forth, how about a test case instead?
Pretraining BERT takes 44 minutes on 1024 V100 GPUs [1]
This requires dedicated instances, since shared instances won't be able to get to peak performance if only because of the "noisy neighbour"-effect.
At GCP, a V100 costs $2.48/h [2], so Microsoft's experiment would've cost $2,539.52.
Smaller providers offer the same GPU at just $1.375/h [3], so a reasonable lower limit would be around $1,408.
For a single BERT pretraining, provided highly optimised workflows and distributed training scripts are already at hand, renting a GPU for single training tasks seems to be the way to go.
The cost of V100-equivalent end-user hardware (we don't need to run in a datacentre, dedicated workstations will do), is about $6,000 (e.g. a Quadro RTX 6000), provided you don't need double precision. The card will have equal FP32 performance, lower TGP and VRAM that sits between the 16 GB and 32 GB version of the V100.
Workstation hardware to go with such card will cost about $2,000, so $8,000 are a reasonable cost estimation.
The cost of electricity varies between regions, but in the EU the average non-household price is about 0.13€/kWh [4].
Pretraining BERT therefore costs an estimated 1024 h * 0.13€/kWh * 0.5 kW ≈ 57€ in electricity (power consumption estimated from TGP + typical power consumptions of an Intel Xeon workstation from my own measurements when training models).
In order to get the break-even point we can…
> It's not just AWS. I think there are only two types of cloud providers: The ones like AWS and DigitalOcean that shift the risk to the customer and the ones that offer shady "unlimited" and "unmetered" plans.
Actually there is a third category, those who care. I will grant you it is a rare category but it is there.
One example name: Exoscale[1]
Swiss cloud provider, they offer:
(a) hard spend limits via account pre-pay balances (or you can also have post-pay if you want the "usual" cloud "surprises included" payment model). (b) good customer service that doesn't hide behind "community forums" Sure they don't offer all the bells and whisles range of services of the big names, but what they do do, they do well.
No I am not an Exoscale shill, and no I don't work for Exoscale. I just know some of their happy customers. :)
[1]https://www.exoscale.com/
Exoscale | Cloud provider | Fully Remote (EU) | Full-Time | https://www.exoscale.com/
Exoscale is a Swiss/European cloud service provider. We offer not only virtual machines but also storage (Object & Block), managed k8s and databases etc...
We're looking for multiple SWEs & SREs either in Switzerland or remotely across Europe!
Join a dynamic working environment with a cutting-edge distributed team. Exoscale strives to create an environment with great working conditions and welcomes diverse applicants.
My team in particular is looking for a SWE to work on Software Defined Network with Rust, ebpf & Go; but we have a lot of other job openings.
If interested, please apply at https://www.exoscale.com/jobs/
I do understand that, my point was that the pieces needed to provide it as a managed service are much easier to come up with in comparison to what AWS had to do with Fargate.
- https://www.scaleway.com/en/kubernetes-kapsule/
- https://www.exoscale.com/syslog/introducing-scalable-kubernetes-service/
If you are connecting from Europe, maybe you could consider exoscale.com 's S3 storage offering?
It's 100% european (with Swiss and German regions), and priced pretty aggressively. https://www.exoscale.com/object-storage/
Disclaimer: I now work for exoscale, but was a happy user before that.
Indeed. I also fail to see how the existence of the CLOUD act, and thus use of any US company, is compatible with GDPR.
See https://www.exoscale.com/blog/cloudact-vs-gdpr/
( Though note exoscale, as a European provider has skin in the game here ).