Aldo Cortesi

Agreed re: the quality of Frank McSherry's posts - they're all great fun and push the technical envelope!

I'm glad you brought up Hilbert ordering. It's fascinating but I rarely see it. I usually see Z-order curves in their place - simpler and more intuitive but also less effective. Does anyone know of large scale databases with Hilbert ordering out of the box?

For anyone who hasn't seen them before, or why they help in ordering, I highly recommend "Portrait of the Hilbert curve"[1] and "Spatial indexing with Quadtrees and Hilbert Curves"[2]. The tldr is that they enable efficient multi-dimensional indexing. Given a request like "X < 40 and Y > 80", most databases need to run "X < 40" and then filter over the results for "Y > 80" - Hilbert curves allow you to find the query's intersection and only then perform minor filtering.

As far as large scale use of Hilbert ordering, Google use it internally for much of their geographical work. They even open sourced a Java library specialised in multi-dimensional indexing based on Hilbert curves called Uzaygezen[3].

[1]: http://corte.si/%2Fposts/code/hilbert/portrait/index.html

[2]: http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves

[3]: http://google-opensource.blogspot.com/2008/08/uzaygezen-multi-dimensional-indexing.html

It's not really true that (due to the data being encrypted client side) your data is safe even if someone were to gain control of the server. It's something often claimed by these "Host Proof" style services.

As long as you are downloading the client side code from the server, someone just needs to make a small change to the javascript and they get access to your data.

Only if you can trust the code, and then make certain that the code does not change, is your data truly safe from the host.

Cortesi (http://corte.si) has done some interesting writing on the subject. http://corte.si/posts/security/hostproof.html

He implemented a similar service (http://cryp.sr + http://corte.si/posts/security/crypsr.html
), and also worked on creating a browser addon that would verify a webpage against a known hash (https://github.com/cortesi/apphash).

Here are a couple of HN submissions on cryp.sr which have some discussions on the "Host Proof" concept:

https://news.ycombinator.com/item?id=1438552

https://news.ycombinator.com/item?id=1226277

If someone managed to compromise LastPass' servers, they would be able to modify the content that was sent to your browser, and thereby do anything they wanted with your master password after you typed it in.

That's the weakness of services that are supposedly "Host-proof". If the host is sending you the webapp/javascript/webpage in which your data is decrypted, each and every time you visit their website, you have to trust that they wont (or someone else wont) modify it in a malicious way in order to gain access to your data.

The only way to be safe from this kind of attack is to use the same trusted copy of the code, and not use any new versions until you trust that it is also safe, or to somehow verify that the webpage sent to your browser is exactly what you expect it to be.

Cortesi has a very good write up on this issue: http://corte.si/posts/security/hostproof.html

As a side note, you can do something similar (albeit a thousand times safer, as you (= your computer) issues the certificates, not some unknown hacker) to fool Game Center.

I get maybe 20 Game Center invites from 8 year old kids on a daily basis... The reason is I faked my score on the popular game Jetpack Joyride (I'm #1 out of 16 million players), and also beat Minesweeper's hard map in a few milliseconds (second guy is I think over a minute)...

I did it just for fun and experiment, but apparently there are kids who really take Game Center seriously and I stopped doing it, not wanting to hurt their feelings. The instructions are here: http://corte.si/posts/code/mitmproxy/tute-gamecenter/index.html , but please don't go crazy, and don't submit ridiculous scores like I did. You may get your account banned (which is not a great thing, if you're a developer).

You might find this useful: http://corte.si/posts/visualisation/binvis/index.html

and related: http://corte.si/posts/visualisation/entropy/index.html (visualizing entropy in binary files)

I used his software on an unknown format from the 1988 game Circuit's Edge to find out if/where the file was encrypted or compressed. It works well and was able to give me a (very) general idea of where in the file I needed to look further.

Edit: Nope, I remembered incorrectly. I used it on the game's .exe to tell me if I was dealing with a compressed executable. It turned out that it was compressed and so I went on a search for an ancient unpacker.

See before and after:

edge.exe packed: http://imgur.com/PlHGl

edge.exe unpacked: http://imgur.com/jeFLb

Humans are awesome at recognizing visual patterns. It's difficult to get a sense of what a file contains from a hex editor, but if you apply the right transformations on the data, distinct features stand out in the visualization.

This allows people to recognize similarities between different files as the resulting patterns are revealed. Take this previous blog post by the same author:

http://corte.si/posts/code/sortvis-fruitsalad/index.html

This shows various sort methods operating on a set of unsorted data. If all you had visibility into was how the data was manipulated, by generating an image based on the data transformations, your pattern matching brain will quickly be able to see which sorting algorithm was used.