Loup Vaillant

The common advice of, "You'll never know enough to use them safely so don't bother trying, just trust us." that has been going around has been proven itself to weaken encryption.

Just recently, an amateur programmer with very little background in cryptography discovered a flaw in libsodium in the Argon2 implementation and also in the reference implementation that everyone in the world was trusting without question. My advice is if you're an engineer, don't be afraid to write your own implementation of tried and trusted ciphers. This is how we find bugs and improve. This isn't the only trusted library or algorithm that has been shown flawed in recent times.

The strength of your cipher implementation can be tested and proven. We need to stop telling everyone these algorithms are absolutely trustworthy so don't try understanding them or implementing them. Nothing ever advances or improves that way. Buck the trends, create competing libraries, try new things.

The ancient ones were not all knowing, they were doing everything wrong. Their designs are full of flaws. Deny them. We need to code ourselves out of the coming cryptographic apocalypse. Do not hide your heads in the sand and hope the world doesn't come crashing down around you.

Edit: I found the blog/website of the man I mentioned in this comment who discovered the Argon2 flaw.

http://loup-vaillant.fr/articles/implemented-my-own-crypto

In his own words, "There's something worrying about this bug: I was the first to…

LR parsers like yacc are obsoleted by Earley parsers, which Cox apparently didn't know about in 2010. Quoting <http://loup-vaillant.fr/tutorials/earley-parsing/what-and-why#Why>:

The biggest advantage of Earley Parsing is its accessibility.
Most other tools such as parser generators, parsing
expression grammars, or combinator libraries feature
restrictions that often make them hard to use. Use the wrong
kind of grammar, and your PEG will enter an infinite loop.
Use another wrong kind of grammar, and most parser
generators will fail. To a beginner, these restrictions feel
most arbitrary: it looks like it should work, but it doesn't.
There are workarounds of course, but they make these tools
more complex.

Earley parsing Just Works™.

On the flip side, to get this generality we must sacrifice
some speed. Earley parsing cannot compete with speed demons
such as Flex/Bison in terms of raw speed. It's not that bad,
however:
• Earley parsing is cubic in the worst cases, which is the
state of the art (and possibly the best we can do). The speed
demons often don't work at all for those worst cases. Other
parsers are prone to exponential combinatorial explosion.
• Most simple grammars can be parsed in linear time.
• Even the worst unambiguous grammars can be parsed in
quadratic time.

My advice would be to use Earley parsing by default, and only
revert to more specific methods…

What you're describing is how net neutrality works, not how the Internet works.

> First, bandwidth is limited. It's whatever download and upload you sign for.

I said that I don't encourage bandwidth limits, not that it wasn't limited. The point is you usually sign a flat fee, you don't have universal access and then pay for traffic, more like in a country like Andorra.

> Second, there is no such thin as "full speed access to Netflix" or whatever. It's limited by your own bandwidth at least.

It's only limited because the ISP has limited it, it's entirely possible to give someone with a slower overall internet connection a higher speed to certain services.
> They don't have to update their infrastructure every time a new web site comes up. As such, access to Netflix, Youtube, or http://loup-vaillant.fr are not separate services.

I don't see you point. ISP do peer directly with services, but I never said they had to. You just have to have a separate exchange for full speed services.

> ISPs that treat access to particular web sites as separate services are just plain lying. The real "separate service", for which they actually spend money, is the throttling/filtering/censoring infrastructure they have to put in place to charge different web sites differently. Text book malicious features, not unlike DRM.

I don't agree with throttling, but "throttling" (or congestion) does occur naturally on the Internet.

I think you really don't get my point, so I'll try and explain in…

This project seems to be the culmination of a rhetorial question that the author posed a while ago: if the advice is 'don't roll your own crypto' [1], then how does any crypto ever come into being? [2]

So perhaps the marketing is off: I don't view this as a "small, secure, auditable, easy to use crypto library"; I view this as a practical and ongoing study in investigating how to make small, secure, auditable, easy-to-use crypto libraries. To be fair, the page mentions that it's "not ready for production" and needing external review a quarter of the way down, but if the point is to prove that a sufficiently dedicated individual can learn the material, endure some criticism, and produce quality software following the example of well-acclaimed experts, then own this point, bare your disclaimers in the beginning, and make it more prominent.

And once you phrase it that way -- which is arguably consistent with the author's previous posts (and HN comments) on the topic, it's suddenly a lot more palatable, and regardless of its ultimate outcome, I can only applaud.

[1] I use an interpretation of "don't roll your own crypto" that's slightly less circular than the author's. In my opinion, it's shorthand for 'defer to the opinion of experts, use ready-made constructs when possible, and if not, then exercise caution when hooking crypto primitives together in unproven ways'. Originally posted here: [3]

[2] http://loup-vaillant.fr/articles/rolling-your-own-crypto

[3]…

I agree that placing sequentially executed code in order of execution often improves readability over abstracted code (especially dynamic dispatch and static/dynamic traits). A similar article is at http://number-none.com/blow/john_carmack_on_inlined_code.html, but linear code has its own failure modes, if code is not factored into blocks with identifiable functionality and constrained/documented side effects (for example 500-line functions twiddling hardware registers and reading/writing global variables). Carmack later wrote an article in support of small-f functional programming and avoiding side effects and global state when practical (https://web.archive.org/web/20190123060017/http://gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php, the article lost all line breaks during migration to gamedeveloper.com)

Another article that touches on this idea (among others) is https://loup-vaillant.fr/articles/source-of-readability which advocates that "code that is read together should be written together" (reading it made me confused until I realized it meant "placed together"), specifically "Consider inlining functions that are used only once".

Tries ain't easy. It's no surprise that a naive trie is bested by a naive hashtable, especially where the number of entries is dwarfed by the capacity of the hashtable. The gold standard for a trie is probably the Judy array[1], which I've heard has some patents on it, but I can't confirm.

Judy arrays obtain their performance by using a myriad range of possible leaf and branch nodes. The goal is to fit as much data that could be used for decision making and branching in the first cache line that would be read. And in the case of very sparse tries this pays off handsomely.

That said, hash tables are really hard to beat. They are susceptible to denial of service attacks though.

[1] http://judy.sourceforge.net/application/shop_interm.pdf

[2] http://loup-vaillant.fr/projects/string-interning/intern_trie.c