Moritz Hardt, a machine learning researcher, writes about differential privacy, fairness, optimization, and how research and peer review are evaluated.
HN commenters cite his posts on the NIPS review experiment and gradient descent, and one praises the Chebyshev polynomial material as a basis for numerical analysis tricks.
Background:
The technique is actually pretty fascinating. This is something that's been well understood by the cryptography community for decades, but is somehow just recently being fully appreciated by the ML community. See here:
http://blog.mrtz.org/2015/03/09/competition.html
https://www.kaggle.com/c/restaurant-revenue-prediction/forums/t/13950/our-perfect-submission
Summary -
Submitting guesses to a system that gives you back scores for your guesses, will quickly leak out enough information that you can reverse engineer a huge number of hidden numbers/labels in surprisingly few iterations, e.g. 700 iterations to covertly extract 10,000+ real numbers with high precision. This surprisingly rapid convergence is a bit reminiscent of the birthday paradox.
Further, this not only lets you win the against the "test" dataset, as apposed to the final "validation" set, but this allows you to significantly increase the data available to you to train your model on, since now you can train your model against both the "test" and "training" datasets.
Layman summary -
ML breaks datasets into 3 partitions "test", "train", and "validation". In cases where they're evenly split, this technique can double the training data you have access to, which is a massive advantage in ML competitions where scores differ by tiny amounts.
Moral judgement -
My opinion, this moral argument is misdirecting the attention from where it needs to be. Yes, it's bad what occurred here. But at this point,…
The PSU Grievance Studies Affair [1] had the same objective as the 1996 Sokal Affair: to test peer review by submitting obviously bad papers that should be rejected, and seeing how many of them are accepted.
The motivation behind this is entirely reasonable. Bad "true negative" as well as known-good "true positive" articles ought to be regularly submitted to every peer review process as a test of accuracy. A great example of a similar idea was the 2014 NIPS consistency experiment [2].
However, there is a problem with the "dog rape" hoax article [3] as a test of the peer review system. Specifically, the author in that paper claims to have spent over 1000 hours carefully cataloguing the behavior of ten thousand people and dogs over the course of one year. The paper then proceeds to produce silly but surprising statistics: female dogs are 70% more likely to be leashed than male dogs, 100% of dogs with shock collars are male, 847 instances of dogs fighting were observed, and so on.
The problem here is that the data was falsified: the observations and experiments claimed never happened, the numbers were all made up. The article was likely accepted because of its data, but the data was a lie. If this data had been real, it might have been a minor but useful study relevant to economists, sociologists, or urban planners, regardless of how silly or made-up the conclusions at the end were.
Peer review works based on the assumption that the author is telling the truth about what…
I think this is largely a case of 'moloch' as described by Scott Alexander: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/
In that long (but excellent essay) Scott points out that self organizing systems like human society can sometimes arrange themselves in horrible local minimas which can be very difficult to escape.
I think the way modern science is organized is decidedly suboptimal. Note that I'm talking purely sociologically: science, as a method, is still by far the best thing we've developed to understand the natural world. However, the incentives around publishing / grants / hiring work are broken in a myriad of ways, from small to massive.
For example:
- High prestige journals/conferences are basically a crapshoot: http://blog.mrtz.org/2014/12/15/the-nips-experiment.html - consider that getting a paper accepted in NIPS/published in Nature might completely change the way your career shapes up.
- Grant funding is incredibly competitive, and the way they are awarded is also a crapshot: https://psmag.com/why-the-national-institutes-of-health-should-replace-peer-review-with-a-lottery-41d09bcce5d8
- The pressure to publish/obtain grants drives people to either make up data to obtain fancy publications, or even to kill themselves: http://www.dcscience.net/2014/12/01/publish-and-perish-at-imperial-college-london-the-death-of-stefan-grimm/
- We reward researchers for brilliant clean discoveries, not brilliant methodologies. However, the outcome of a…
> paper reviewers are usually not supposed to look at the actual source code of the papers
Wait what? I haven't reviewed for ACL but most conferences don't say "don't look at the source code." They will say that reviewers are not required to look at it (as well as the appendix). But generally it just isn't uploaded. I do always look at the main method when it is there but talking to my peers and advisor, this is very uncommon[0]. My experience is that most reviewers do not spend more than an hour on a work and make an opinion within 15 minutes.
> Not sure what the best solution is, other than having the most "hyped" papers double verified by researchers on Twitter.
I'd say (as a start):
1) Get rid of the conference system. A zero-shot (maybe 1-shot if "rebuttal" is allowed) zero-sum system is just disastrous, especially at scale. There's high incentives to actually reject works you review for. A conference system has a binary outcome and the purpose is to reject 80% of papers based on a rather noisy metric of "top tier." A journal system is a back and forth where reviewers are trying to improve the paper. The purpose of the reviewers here is to determine if the idea is indeed good, and then if the paper meets the requirements or not and must explicitly state what needs to be changed for acceptance.
1.5) An actual rebuttal system could help alleviate some of these issues. Using OpenReview for a conversation between authors and reviewers is critical. A singular 1 page…
> It's very difficult to pass lackluster work through these places
Not really that hard. It's difficult to pass outright bad work through a top-tier conference, but above a quality threshold, it's basically a crapshoot; the reviewing process doesn't reliably separate middling from great papers. The very low acceptance rates (~10-15%) and very high inter-reviewer variance [1] combine to make it pretty random whether a given paper, of sufficiently "ok" quality, gets accepted or not. So the main successful strategy to accumulate a lot of papers in top-tier conferences is simply to submit a lot of decent papers to top-tier conferences. How many such papers a given professor can churn out depends largely on how many students, post-docs, and research scientists they can hire, which depends on funding.
[1] NIPS, one of the top-tier machine learning conferences, did a very interesting experimental study of this. http://blog.mrtz.org/2014/12/15/the-nips-experiment.html
Great post! The author touches on Chebychev polynomials, which are the basis for quite a few numerical analysis tricks, including conjugate gradient [1], accelerated gradient descent [2], and Chebychev semi-iterative methods (which find the best combination of past iterates to use during an optimization procedure; sadly I can't find a good reference).
There are a number of facts/folklore about Chebychev polynomials that seem to go stated without proof in a lot of papers, so a few years ago I wrote up some notes [3,4] with the most common claims. Maybe someone will find them useful!
[1] (p35) http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf
[2] http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent.html
[3] https://benrbray.com/static/notes/chebychev-polynomials_dec16.pdf
[4] https://benrbray.com/static/notes/minimax-approx_nov16.pdf