I downloaded 35 million posts from HackerNews, here's what I've known
Hey everyone! Today, we're taking a deep dive into HackerNews data.
I've downloaded information on all posts, comments, job listings, and user profiles.
You can find both datasets on our HuggingFace, so there's no need to scrape data by yourself.
Just a heads up — all the times mentioned in this article are in GMT. In this analysis, we're focusing on a subset of posts that include both a URL and title.
Posts – 14.6 GbUsers – 349 MbQuestions that got answered by this article
• What's the best day and time to post on HackerNews?
• How does title length affect engagement?
• Does being associated with Y Combinator affect engagement?
• a16z vs Sequoia vs Kleiner Perkins
• Elon vs Musk vs Elon Musk?
• AI vs Web3?
• Who are the Top-10 and Bottom-10 users on HackerNews?
User Activity
Weekly activity peaks at 4 PM GMT with most posts created.
Product launches typically occur around 4 PM GMT as well.
Posts mentioning "YC" appear to follow a strict schedule, likely due to YC coordinating these postings.
It appears that, on average, post frequency decreases after December 10th.
Content
The most popular domains on HackerNews include GitHub, Medium, YouTube, and NYTimes, with GitHub surprisingly taking the top spot.
It seems crypto has seen better days.
Elon Musk's mentions reached a record high in 2022, driven by the Twitter buyout news.
a16z is definitely riding a wave of hype.
Algorithm
It appears the optimal time to post on HackerNews is at 1 PM. Despite most product launches and YC posts happening at 4 PM, it's better to aim for 1 PM to maximize engagement.
Interestingly, posts on weekends get more engagement, likely due to fewer postings during that time.
Clearly, shorter titles tend to receive higher engagement on posts.
Posts mentioning YC enjoy twice the average engagement.
Audience
Here's the Top-10 list of HackerNews users.
Curious about the Bottom-10 list of HackerNews users by karma? Here it is — and yes, karma can be negative
Over the past decade, signups on HackerNews have remained stable.
Conclusion
• The best time to post is between 1 PM and 3 PM GMT
• Weekend launch can be advantageous
• Post frequency declines towards the end of the year
• Musk outperforms Elon in engagement
• AI generates more buzz than web3
• a16z is hype king
• Better get accepted to YC
To be continued...
If you enjoyed this article and want to see a follow-up, please like and share our Twitter Announcement
In second part we will look at the impact of HackerNews articles on other websites (including GitHub repositories) and provide much more audience analysis.
If you need design, code, machine learning, or data scraping, feel free to reach out to us at hi@wearefastflash.com
Good luck, and have fun!
