Reproducible Builds

The hard part -- sometimes really hard -- is the "bit-for-bit identical" requirement.

Lots of builds are recreatable but not reproducible (there is probably a better term of art here). You can go back to a point in time and build the version of the software as it was, but you are not guaranteed to get a bit-for-bit clone. (See https://reproducible-builds.org for a thorough discussion)

The problem is that there are lots of uncontrolled inputs to a build that are due to sourcecode or compiler changes. Most famously there are timestamps and random numbers, which mess up all sorts of hashing-based approaches.

These can even be non-obvious. Just the other day I and a colleague were investigating the (small but unsettling) possibility that an old buildpack had been replaced maliciously. We compared the historical hash to the file: different. We rebuilt the historical buildpack with trusted inputs: still different.

Then we unzipped both versions and diff'd the directories: identical.

What had thrown our hashes off was that zipfiles, by default, include timestamps. We have a build that is recreatable but not reproducible.

Speaking of builds, we are able to reproducibly build some binaries but not others. Off the top of my head our most high-profile non-reproducible build is NodeJS. Some other binaries (Ruby and Python, in my not-at-all-complete recollection) are fully reproducible.

This difficulty with fully reproducing makes it hard to provide a fully trustworthy chain of…

- Even assuming all this, a hybrid approach of HTTP + IPFS (or DAT) is still better than what we have now, since IPFS is essentially a worldwide CDN for static files.(Sorry: an inter-planetary one.)

- The content-addressing aspect makes it perfect for distributing commonly used libraries.

- We already cache all this content locally. What a waste! Why do I have to fetch jQuery from fricking California when it's sitting on my girlfriend's phone in the other room?

- This extends beyond the web: think about the benefits (both in security, practicality, and performance) of content addressing introduced into package managers (take it one step further even: combine this idea with the new move towards reproducible builds (https://reproducible-builds.org) and package managers like guix and nix and things get really interesting).

- It's actually easier to use for the average person. If you don't think this is the case I propose a simple experiment: download the beaker browser and set up a simple static site. I recently did this. It really is one-click hosting! Considering how complicated web hosting is to the average person (ever try to walk a friend through setting up a website? not. fun.) -- people would love to be able to set up personal websites this easily... and for free?

- As others have mentioned, there are many solutions being worked on for the mirroring of data (Filecoin etc).

- For websites that are visited regularly, this is not an issue -- all content is cached…

I disagree that rust, go or python inherently do not work with linux package managers, instead there's a culture within subsections of those communities who do not value stability (I'm not sure about nodejs community though), which cause these disagreements. C isn't immune to this (and never has been)—science codes are infamous around their lack of care around stability (and IMHO one of the reasons for scientific python's success over alternatives has been this stability).

I'd also disagree with the idea that the alternative is reliable—try building and modifing a project that hasn't been touched in 6 months, and see how reliable that is. With your tree of dependencies (absent specific projects which abstract over a set of unstable dependencies and hence implicitly stabilise them, e.g. SDL) the stability of your project (whether it is a library or application or framework) is set by the least stable dependency you have. Increasing you dependencies increases the risk, but if you have an ecosystem which values stability, larger dependency trees should not see a significant increase in risk (personally, I'd love to see the scientific rust ecosystem achieve similar stability to python).

That's not to say linux distros are perfect (change can be slow ;)), but there's lots of little things they do get right (how many projects handle updating configuration files correctly—that kind of thing is built into distro tooling), and they enable ecosystem-wide changes more that the…

Congrats on the launch.

There is both a large need for improvements in npm supply chain security and a market willing to pay for them.

Concerns:

1) npm Open-Source Terms condition four[1] states.

You may access and use data about the security of Packages, such as vulnerability reports, audit status reports, and supplementary security documentation, only for your own personal or internal business purposes. You may not provide others access to, copies of, or use of npm data about the security of Packages, directly or as part of other products or services.

This statement seems vague enough to potentially include your use case. It also seems to include what snyk, jfrog xray, sonatype, and white-source do, so maybe this is not an issue.

2) It appears that this will be an open-core business. What capabilities are you willing to provide in the free/community edition and under which licenses?

3) The website doesn't show pricing. Can you provide details on this?

Questions:

1) What are your thoughts on using reproducible builds[2] plus Diverse Double-Compiling (DDC)[3] on the dependency graph to ensure build artifacts originate from known git repositories? Disclosure, I've been working on this for a few months now.

2) Where do you run your analysis? AWS and DigitalOcean have terms that prevent running high risk code.

3) Do you have examples of previous attacks and how your tooling would handle them?

Best of luck.

[1]…

It's probably exactly your concern about the ability for apps with network access to upload user data that led the Android team to introduce the locations permissions for BLE identifier scanning.

Without a location requirement, an app could claim to use only Wi-Fi and BLE permissions, yet it could combine beacon scanning and data access to de-anonymize user locations.

This permission requirement was introduced in Android 6.0 (see the release notes[1]) in October 2015, so it's been around for a while.

If you have some suggestions around how to improve the tracking of permission usage (static analysis? run-time requests (preferably avoiding times when users are under duress and likely to click 'OK' by default)?) then you may wish to file some requests with them and/or contribute to other projects that you feel are taking a better approach.

Your concerns around trust in app developers -- regardless of whether they are a government or any other entity -- are best handled by two means:

- Open sourcing the code (which NHSX have done)

- Enabling reproducible builds[2] so that users can confirm they have an authentic binary build of the source code

[1] - https://developer.android.com/about/versions/marshmallow/android-6.0-changes#behavior-hardware-id

[2] - https://reproducible-builds.org/

Great article.

I see this trend too: the systems powering the modern world are increasingly featureful, complicated, centralised into a few hands, and this will likely continue.

I'm a keen developer and user in this world, and recognize the vast users this world provides for.

At the same time, I appreciate a back-to-basics approach that emphasizes systems that can be understood and controlled by individuals and small communities:

* Hardware that has open specs: Pinephone, Pinebook, Raspberry Pi, ...

* Open source OSs: Linux desktop, Linux mobile, Lineage OS.

* https://reproducible-builds.org/ and https://www.bootstrappable.org/, allowing trust to be distributed.

* Monorepo based OSs: NixOS, Guix.

* Gemini, rather than the web: web browsers are now beyond reasonable understanding/control for small communities.

* Mastodon rather than Twitter.

* Projects that are community driven.

* XMPP/Matrix over Signal/Discord/FB.

I choose to live within this world for my personal world where possible. It's not as featureful, and that's fine.