Hacking and Gonzo is a Medium publication by a developer writing on ranking algorithms, Redis analytics and front-end design patterns.
HN commenters mainly link its explainer on how the Hacker News ranking algorithm works, including when discussing moderator interventions and how HN-style ranking is reused.
I was learning authentication with graphql and decided to put my knowledge into practice by cloning one of my favorite sites!
First of all, I am sorry for not creating a demo server for this, I don't really know how to put full stack projects into production yet. That being said, I have dockerized the project for easy setup, if you have time you can check it out, I would appreciate that.
What I would appreciate more is, giving me ideas about some hurdles I came across while implementing this.
1. It was somewhat difficult to create your own authentication system from scratch(although I walkthrough a youtube tutorial). Are you implementing your own system, or using a 3rd party? I heard `hasura` and `firebase` is also good alternative for authentication.
2. While using Prisma and GraphQL, I felt like, I was duplicating the schemas for both of them. Is there a way to combine them into a single place?
3. Implementing home page was difficult for me. I have assigned a `ranking` field for each `post` in database. This ranking field calculated as described in this link : https://medium.com/hacking-and-gonzo/how-hacker-news-ranking-algorithm-works-1d9b0cf2c08d.
I've only take into account submissions that created less than 2 days ago and updated rankings every 10 minute by a cron process. How would you architect this?
Just out of curiosity, how do you crush a site with just posting maybe 100 text based comments on the top 20 items or whatever?
I mean what exactly is the load that causes this?
If you are executing some algorithm for each request - stop. Just run it every minute to rebuild a cache. HN discourages rapid fire replies anyway.
How hard is it to simply build an html fragment from a tree and cache it? You just see what nodes need other nodes to be appended.
OK, I can imagine MAYBE the most intensive thing is the votes and re-ordering of replies.
First of all I hope you are using this algorithm: https://medium.com/hacking-and-gonzo/how-reddit-ranking-algorithms-work-ef111e33d0d9?source=linkShare-a7703696fe94-1489002238
Secondly, seriously once again you can batch the upvotes by thread and then process them during the recalculation phase. You only need to recalculate weight totals for the threads where there were upvotes - and this can be parallelized.
I recognise the huge amounts of effort involved in this and I applaud the moderators for keeping HN an interesting place to be.
That said, I think it's reasonable for us to have visibility on their manual interventions, and this could be easily surfaced via the Hacker News API (https://github.com/HackerNews/API), if the Story JSON included the values of "contro," "bury," and "gag" fields, which are currently opaque to users of the API
See https://medium.com/hacking-and-gonzo/how-hacker-news-ranking-algorithm-works-1d9b0cf2c08d for more discussion on terminology
One other positive thing to mention about HN: the cost of running HN is non-zero, yet it remains ad-free. I'm grateful for this. So many websites have become unusable due to monetisation dark patterns.
Smaller subreddits are beings dead is partially due to reddit's Hot ranking algorithmn. It is designed to have fresh posts up top on the daily [1]. Unless you switch to an alternate rating scheme (monthly top is helpful), any subreddit without hundreds of posts per week is basically dead.
Large subreddits with thousands of weekly posts have the opposite problem, where low effort posts will dominate the top if unmoderated.
[1] https://medium.com/hacking-and-gonzo/how-reddit-ranking-algorithms-work-ef111e33d0d9
See:
How Hacker News ranking algorithm works
https://medium.com/hacking-and-gonzo/how-hacker-news-ranking-algorithm-works-1d9b0cf2c08d
Posts get to the front page by being upvoted by users. The posts then decay off the front page as time goes by. So the mix that is there is based upon what users upvote.
Therefore, if you want differing content on the front page, then start upvoting stuff sitting on the new page that you'd like to see on the front page to help it get to the front page.
It seems like they're doing something similar to how Reddit's "hot" algorithm reportedly works. As of 2015, that was supposedly like:
log(upVotes - downVotes) - (hoursOld / 12.5),
according to https://medium.com/hacking-and-gonzo/how-reddit-ranking-algorithms-work-ef111e33d0d9.
Fair Trending basically differs just in the log() part, substituting (fractionUpVotes * views) for (upVotes - downVotes). This could change the ranking if a lot of people view without voting either way.