Hacker News archive (open-index)
Data
dataset archive
SHOW HN2026
Legend
Hacker News archive (open-index)
Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
huggingface.coHow HN took it
9 positive, 34 neutral or asking, 0 mixed, 10 critical, out of 53 comments about the project. Thread score -0.03.
Docs: 0 for, 4 againstIdea: 4 for, 0 againstPrivacy: 1 for, 3 againstExecution: 0 for, 2 againstDesign: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful dataset for projects and classroom use
From 3 comments
As someone who had made a project analysing hackernews who had used clickhouse, I really feel like this is a project made for me (especially the updated every 5 minute aspect…
Imustaskforhelp on HNIs is possible to only download a subset? e.g. Show HNs or HN Whoishiring. The Show HNs and HN Whoishiring are very useful for classroom data science i.e. a very useful set of…
Onavo on HNYay! So much knowledge in just 11GB. Adding to my end of the World hoarding stash!
imhoguy on HN
Frequent updates enable new ideas
From 2 comments
As someone who had made a project analysing hackernews who had used clickhouse, I really feel like this is a project made for me (especially the updated every 5 minute aspect…
Imustaskforhelp on HNThis is great. I've soured on this site over the past few years due to the heavy partisanship that wasn't as present in the early days (eternal September), but there are still…
politician on HN
Objections
Data licensing and terms of use unclear
From 3 comments
what’s the license? “do whatever the fuck you want with the data as long as you don’t get caught”? or does that only work for massive corporations
bstsb on HNwhat's the license for HN content?
tonymet on HNis the legal page a placeholder, do words have no meaning? https://www.ycombinator.com/legal/ Mods, enforce your license terms, you're playing fast and loose with the law…
GeoAtreides on HN
No way to remove personal comments; privacy and GDPR concerns
From 3 comments
Where’s the opt out ?
robotswantdata on HNHello. I didn’t consent to any of my HN comments being used in this way. Please kindly remove them.
trwhite on HNis the legal page a placeholder, do words have no meaning? https://www.ycombinator.com/legal/ Mods, enforce your license terms, you're playing fast and loose with the law…
GeoAtreides on HN
Replacing the full file every 5 minutes is wasteful
From 1 comment
Replacing an 11.6GB Parquet file every 5 minutes strikes me as a bit wasteful. I would probably use Apache Iceberg here.
vovavili on HN
Requests
Release the update system source code
From 1 comment
Could you also release the source code behind the automatic update system?
maxloh on HN
Upload to academic torrents
From 1 comment
Please upload to https://academictorrents.com/ as well if possible
lyu07282 on HN
Subset download such as Show HN or Who is hiring
From 1 comment
Is is possible to only download a subset? e.g. Show HNs or HN Whoishiring. The Show HNs and HN Whoishiring are very useful for classroom data science i.e. a very useful set of…
Onavo on HN
Questions
Why update every 5 minutes and is the update current
From 2 comments
Good for demo but every 5 minutes? Why?
kshacker on HNStatic web content and dynamic data? > The archive currently spans from 2006-10 to 2026-03-16 23:55 UTC, with 47,358,772 items committed. That’s more than 5 minutes ago by a day…
mlhpdx on HN
Why are deleted and dead stored as integers
From 1 comment
Under the Known Limitations section ``` deleted and dead are integers. They are stored as 0/1 rather than booleans. ``` Is there a technical reason to do this? You have the type…
0cf8612b2e1e on HN
Is the data license clear
From 1 comment
what's the license for HN content?
tonymet on HN
Alternatives named
ClickHouse HN playground
From 1 comment
The best source for this data used to be Clickhouse (https://play.clickhouse.com/play?user=play#U0VMRUNUIG1heCh0aW1lKSBGUk9NIGhhY2tlcm5ld3NfaGlzdG9yeQ==), but it hasn't updated…
xnx on HN