Cyphertite, from Conformal Systems, was an encrypted online backup service with a command-line client and a cryptographic deduplication method.
HN commenters suggest it as an encrypted backup option similar to tarsnap, with one praising its deduplication of already encrypted data.
> Apart from Google (who uses it for Youtube), Go hasn't really been used in large projects.
I don't know how large each of these projects is, but a growing number of companies are using it for serious projects:
Iron.io: How We Went from 30 Servers to 2 with Go
http://blog.iron.io/2013/03/how-we-went-from-30-servers-to-2-go.html
Behind the Scenes: Big Data at Torbit
http://torbit.com/blog/2013/02/19/big-data-at-torbit/
pool.ntp.org DNS server in Go
http://news.ntppool.org/2012/10/new-dns-server.html
Juju at Canonical
http://www.reddit.com/r/programming/comments/18atce/juju_canonical_109k_lines_of_go_code/
Go at bitly
http://word.bitly.com/post/29550171827/go-go-gadget
CloudFlare blows hole in laws of Web physics with Go and Railgun
http://arstechnica.com/information-technology/2013/02/cloudflare-blows-hole-in-laws-of-web-physics-with-go-and-railgun/
dl.google.com now served by Go
https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/BNUNbKSypE0
Go At Conformal
https://www.cyphertite.com/blog.php?/archives/7-Go-at-Conformal..html
Go at Novartis
https://plus.google.com/114945221884326152379/posts/d1SVaqkRyTL
Go at the BBC
http://www.quora.com/Go-programming-language/Is-Google-Go-ready-for-production-use/answer/Kunal-Anand
Go at SoundCloud
http://backstage.soundcloud.com/2012/07/go-at-soundcloud/
Go at CloudFlare
http://blog.cloudflare.com/go-at-cloudflare
Go at Heroku
http://blog.golang.org/2011/04/go-at-heroku.html
Building StatHat with…
cyphertite uses a private (per-user) deduplication pool, not a global deduplication pool like dropbox and many other online storage providers.
Example 1 - user1 uploads file1, then at a later time user1 tries to upload file1 again. the 2nd occurrence of the data in file1 is recognized as a duplicate of what has already been stored, so the service does not re-upload the bulk data of file1. this saves bandwidth, time, cpu cycles etc.
Example 2 - user1 uploads file1, then at a later time user2 uploads file1. in a private (per-user) deduplication pool, both user1 and user2 would need to separately upload a copy of file1, but in a global deduplication pool, user2 would not need to actually upload the bulk file data since it can reference data that user1 uploaded earlier.
the upshot here is that each user has their own separate crypto keys that are used client-side, similar to spideroak, tarsnap, etc.
i could get into the details here, but it's pretty well summarized in our infographic and crypto whitepaper
https://www.cyphertite.com/papers/CT_InfoGraphic.pdf
https://www.cyphertite.com/papers/WP_Crypto.pdf
There are a few reasons why many people would not (or should not) use Cyphertite:
* It's been such a long time in development and there's still no client for OS X. Building from source should be an option, not the only way, if this is ever meant for mass adoption.
* There's no information about accessing backup files from mobile devices.
* Most importantly, unless you have more than 100GB of data to backup, you're better off with the premium plan (enterprise segment) than the expensive personal plan where you would get a lot lesser in return.[1] I had pointed this out to them a year or two ago, but there's been no change and this structure does not make sense for the section of home users who may have only a few tens of GBs to backup.
[1]: https://www.cyphertite.com/plans.php
Casting doubts and aspersions when you don't know anything and haven't
bothered to read up on what they do is extremely unfair.
They do some very impressive cryptographic deduplication. Essentially
it's a method to deduplicate already encrypted data reliably. If you
don't realize how amazingly cool that is, you need to do some reading.
It's a really tough problem, and it solves a major pain point for anyone
with large volumes of encrypted data (i.e. governments).
EDIT: Links
https://www.conformal.com/
https://www.cyphertite.com/
You can do deduplication on chunks of the file.
AFAIK, this is what cyphertite[1] does. Split each file in 256KB chunks, store their checksum and match those against a db do avoid resending / copying the same data over and over again.
I haven't tested cyphertite, but I've been meaning too. I mean, Ryan McBride is involved in the project as well as other OpenBSD devs. I'm hoping it has the same level of polish as OpenBSD.
[1] https://www.cyphertite.com/
Cyphertite may also be of interest: https://www.cyphertite.com/
Client-side encryption and deduplication, with source code. 8GB free, $10/mo for personal unlimited use, 10c/GB/month for business/enterprise. My main reservations are they seem to be based in one datacenter, and don't seem to have support for multiple keyfiles with separate read/write/delete/machine restrictions. Also not in FreeBSD ports :P