Reddit Blog

"But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten, and not only because these languages are better suited to their task performance-wise, but because they have other, more important things to think about. If your challenge is not translating a business model into code, but solving a real algorithmic or technical problem, then the programming language really doesn't matter, and you're better off sticking with the one(s) you know best."

This is so true. They often use something like MATLAB or R, but that doesn't affect your essential point, which is a great one and worth thinking about. How much of what you encode is original/cutting edge thought vs routine (CRUD apps) say.

That said, that particular blade cuts both ways. There is nothing (other than inertia) stopping the "one(language) you know best" being something reasonably advanced, with decent, if not ubiquitous libraries and tooling, like Scala or Clojure or OCaml, or even Common Lisp. Why would you want to use an 'inferior' language if you know a better one? So in the end this argument may just devolve to "Most engineers in charge of implementing systems are more comfortable with Java/C++ than Clojure/OCaml".

Something of medium complexity, between the basic CRUD web app and pioneering projects like Watson, like the (Twitter)Storm library for…

>There is an optimal tradeoff somehow.

You are basically talking about Bandit Algorithms, which are a family of techniques to solve the problem of trading of exploration ('discovering new good content') and exploitation ('just promoting what has already proven itself').

http://en.wikipedia.org/wiki/Multi-armed_bandit

There are several bayesian approaches to bandit problems, which calculate the the uncertainty in a bayesian way, and then use this uncertainty to make the tradeoff between explore and exploit (e.g. http://en.wikipedia.org/wiki/Thompson_sampling).

So that's a well understood problem.

For my money, the 'correct' way to solve the HN problem is as a bandit arm for each article, which you use to decide how much to display low vote articles.

I think this bandit process should be run on top of a bayesian model that features not just the estimated quality of the article from its votes, but also a prior which it gets from the submitter's 'quality' (which is updated depending on how their articles do - this would also serve as a brake on spammers).

Then you just need a heuristic to age out articles over time.

I'm sure PG knows enough ML/stats people to do this sort of thing properly; the fact that its not done means admins are probably happy enough with the status quo.

It's also worth looking at this article by the author of XKCD on voting on reddit, for an easy discussion of simple bayesian…

> Soon after launch reddit had performance issues and was rewritten in Python.

The reason was explained here:

http://blog.reddit.com/2005/12/on-lisp.html

I quote:

Emacs and SLIME are a killer combination, but I develop on a Mac, and reddit.com is a FreeBSD box. On my Mac, my choices of threaded Lisp implementations was limited to OpenMCL, and in FreeBSD it's CMUCL. Because of the low-level socket and threading code we had to write, reddit would not run on my Mac, and I was always tethered to our FreeBSD development server. Not being able to program offline is a pain.

If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand.

Back in 2005 Common Lisp had a fragmentation problem: not all libraries would work on all implementation/os pairs and not all implementations would work on all operating systems (in fact iirc no open source implementation would work on windows, freebsd, linux and mac)

"Steve Newhouse, the chairman of Advance.net, decided very early on that his company would not be the blob that ate Reddit, and for the most part, left well enough alone. “We had some ideas about what would be good, but it might not have worked,” Mr. Newhouse said. “We paid attention to the community instead.”"

I may be remembering stuff wrong, but wasn't there more tension + conflict during the period right after the acquisition? Didn't Reddit struggle to pay for basic infrastructure stuff due to their new parent company's skepticism about its viability and perception of its value?

http://blog.reddit.com/2010/07/reddit-needs-help.html

And it was a long while until it got spun out. (http://venturebeat.com/2011/09/06/reddit-break-conde-nast/)

Not that there's anything crazy in any of this. Just that the NYT story paints it like right away they 1) knew Reddit's worth and 2) knew to have a hands-off approach.

I'm far from a Reddit power user, happy to hear other perspectives.

How would the HN community react if any of the ideas suggested in this thread were implemented by PG?

Non-redditors need to understand the overwhelming revolt against any sort of advertisement in the reddit community. Yishan Wong (CEO) had to respond to one of the "top" comments when this article was posted in r/technology[1].

While reddit "fans" will support the company through "Reddit Gold"[2], advertisements in the form of "sponsored links" or "feed ads" doesn't seem to work under the beloved format [ex: 3]. If you're interested in learning about Condé Nast and other investors , I found a "myth busters" blog post [4].

[1]http://www.reddit.com/r/technology/comments/1tvfze/reddit_is_going_for_profitability_next_year/cec2256

[2]http://blog.reddit.com/2013/10/thanks-for-gold.html

[3]http://www.reddit.com/comments/1o6f63/now_you_can_fly_historys_greatest_warplanes_in/

[4]http://blog.reddit.com/2013/08/reddit-myth-busters_6.html

If a comment has one upvote and zero downvotes, it has a 100% upvote rate, but since there's not very much data, the system will keep it near the bottom. But if it has 10 upvotes and only 1 downvote, the system might have enough confidence to place it above something with 40 upvotes and 20 downvotes -- figuring that by the time it's also gotten 40 upvotes, it's almost certain it will have fewer than 20 downvotes. And the best part is that if it's wrong (which it is 5% of the time), it will quickly get more data, since the comment with less data is near the top -- and when it gets that data, it will quickly correct the comment's position. The bottom line is that this system means good comments will jump quickly to the top and stay there, and bad comments will hover near the bottom.

[1] http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html