Lockdown Sceptics

But you're overlooking the most likely outcome - that nobody is qualified in this field. However software developers can admit this and epidemiologists can't.

Bearing in mind for a moment that epidemiologists are amateur programmers, their work output is literally papers+code, they have a major handicap that they can't stop and say "maybe this doesn't work". What else would they do?

I've found the comments by Caswell Bligh very insightful on this topic. Bligh is one of the awful "men in tech" programmers the article laments, who has been writing his own models based on epidemiological papers. Perhaps even from an armchair. Everything he's concluded is worth reading but I'll quote some extracts:

https://lockdownsceptics.org/2020/04/26/latest-news-12/#comment-935

Do ‘R0’ or the other ‘R’ values even mean anything when ‘infection’ versus ‘exposure’ can’t even be defined? If social distancing reduces the level of exposures but not the number, does this mean the ‘R’ number is the same, or reducing? Maybe the lockdown means that the population is being quietly, in the background, inoculated to a certain level of resistance (which is what I think is probably happening) but that it doesn’t show up in antibody tests.

Many of these things will never be known. They are probably not even predictable with viruses that are not ‘novel’. Virologists and immunologists may know some of it, and they also know what they don’t know. The modellers don’t know any of it.

As such, models will…

I am referring to this blog post:

https://lockdownsceptics.org/code-review-of-fergusons-model/

It says, word-by-word:

> Clearly, the documentation wants us to think that, given a starting seed, the model will always produce the same results.

>

>Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.

> I’ll illustrate with a few bugs. In issue 116 a UK “red team” at Edinburgh University reports that they tried to use a mode that stores data tables in a more efficient format for faster loading, and discovered – to their surprise – that the resulting predictions varied by around 80,000 deaths after 80 days: ...

The bugs which the blog post implies here are such ones as described by Jens Regehr: https://blog.regehr.org/archives/213

Not that I do not endorse these statements in the blog - I am rather skeptical whether they are true at all.

What the authors of the blob post mean is clearly "undefined behaviour" in the sense of non-deterministic program execution of a program that is not well-formed. It is clear that many non-experts could confuse that with the pseudo-randomness implicit in Monte-Carlo simulations, but this is a very different thing. The first is basically a broken, invalid, and untrustworthy program. The second is the established method to produce a computational result by introducing stochastic behavior, which is for example how modern weather models work.

These are wildly…

>And idiots will continue parroting this exact line as billions of people are dying. Exactly the same way idiots are protesting social distancing, lockdowns, and mask wearing in US right now

There is no purpose to have a discussion when your thinking is so narrowly black and white. Your worldview doesn't seem to allow for the shades of grey necessitated by uncertainty.

If all skepticism was met with the same belittling as climate science, we'd still be stuck in the dark ages. Don't think it's possible for an institution of sciences to be very wrong for a very long time about a very important topic? Look at the forces that caused us to fruitlessly pursue the amyloid plaque hypothesis for decades.

Consensus has a strong normalizing effect, but that does not necessarily mean that a given consensus is correct.

Anyway I'm being rate limited by sanctimonious downvoters so this conversation is effectively over.

Edit: by the way, with respect to the lockdown controversy, did you hear about the atrocious academic modeling code that informed the lockdown policy? A single 15k line C file which, among other things, does not seed random number generation and cannot be reproduced. So bad that I just found there's an entire website dedicated to a teardown[0]! But I'm an "idiot" for showing skepticism. Do you think climate modeling code is any better? Have you worked with academic code? I have. It's universally bad. But, again, I'm an "idiot" for expressing any degree of…

This article brings up scientific code from 10 years ago, but how about code from .. right now? Scientists really need to publish their code artifacts, and we can no longer just say "Well they're scientists or mathematicians" and allow that as an excuse for terrible code with no testing specs. Take this for example:

https://github.com/mrc-ide/covid-sim/blob/e8f7864ad150f4002279cff6cf8673208b5cd387/src/SetupModel.cpp

This was used by the Imperial College for COVID-19 predictions. It has race conditions, seeds the model multiple times, and therefore has totally non-deterministic results[0]. Also, this is the cleaned up repo. The original is not available[1].

A lot of my homework from over 10 years ago still runs (Some require the right Docker container: https://github.com/sumdog/assignments/). If journals really care about the reproducibility crisis, artifact reviews need to be part of the editorial process. Scientific code needs to have tests, a minimal amount of test coverage, and code/data used really need to be published and run by volunteers/editors in the same way papers are reviewed, even for non-computer science journals.

[0] https://lockdownsceptics.org/code-review-of-fergusons-model/

[1] https://github.com/mrc-ide/covid-sim/issues/179

John Carmack replies https://twitter.com/ID_AA_Carmack/status/1258192136744456199:

> It is disappointing, but not really unexpected, to see this take on the epidemic simulation code release: https://lockdownsceptics.org/code-review-of-fergusons-model/ If we accept it at face value, we have a retired software engineer making the case that non-determinism in a simulation shows incompetence. I am all for deterministic-everything, but issues with random seeds and parallel floating point reductions are extremely widespread, and a great deal of science gets done with non-deterministic simulations. Heck, professional software engineering struggles mightily with just making completely reproducable builds. I take no position on the mechanics of the simulation or the conclusions drawn from the runs, but implicitly encouraging scientists to keep their code secret if it isn't "perfect" is damaging.

Also later:

> There was a continuous integration regression test running a small model in single threaded mode on every check in, comparing for exact accuracy, at least when I was working on it -- I haven't pulled the public release.

For background, he was pulled in to do a bit of work on the simulation code a month or two ago.

Really?

> Imperial are trying to have their cake and eat it. Reports of random results are dismissed with responses like “that’s not a problem, just run it a lot of times and take the average”, but at the same time, they’re fixing such bugs when they find them. They know their code can’t withstand scrutiny, so they hid it until professionals had a chance to fix it, but the damage from over a decade of amateur hobby programming is so extensive that even Microsoft were unable to make it run right.

That's from the first analysis[1]. There's a follow up[2]:

> Sadly it shows that Imperial have been making some false statements.

and

> It’s clear that the changes made over the past month and a half have radically altered the predictions of the model. It will probably never be possible to replicate the numbers in Report 9.

[1] https://lockdownsceptics.org/code-review-of-fergusons-model/

[2] https://lockdownsceptics.org/second-analysis-of-fergusons-model/