Till
Dev tools
web scraping proxy
SHOW HN2021
209points
58comments
#215of 2,533 in 2021
Top 9%of 2021 by points
How HN took it
RoastedHalf or more of the comments are critical
2 positive, 3 neutral or asking, 0 mixed, 6 critical, out of 11 comments about the project. Thread score -0.20.
Idea: 2 for, 5 againstDocs: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful for scraping
From 1 comment
Looks extremely useful. Starred for if I ever need to do scraping
awestroke on HN
Objections
Circumvents robots.txt and site defenses, ethically dubious
From 3 comments
> Till helps you circumvent detected as a web scraper by identifying your scraper as a real web browser. It does this by generating random user-agent headers and randomizing…
cube00 on HNBut what when there are separate interfaces specifically designed for real users and machines? Case in point: OpenStreetMap (run by volunteer sysadmins) provides completely open,…
Doctor_Fegg on HN> Till was architected to follow best practices that DataHen has accumulated over the years of scraping at a massive scale. The best practice is not to build a business that's…
JeremyBanks on HN
Source code not public, unclear if runnable locally
From 1 comment
Why is this on Github if I still need an API and cannot run it locally since the actual code is not public? Or is this some sort of „source available“ content marketing nonsense?
axiosgunnar on HN
Sign-up required
From 1 comment
This sounds technically interesting but ethically dubious; also requires sign-up - even though free - which (I think?) is discouraged for Show HN. If you are able to show how…
mellosouls on HN
Requests
Clarify difference between repo and hosted service
From 1 comment
What's the difference between the linked repo and the hosted service at https://till.datahen.com/api/v1 ? Why do I need to sign up for a token if I am hosting my own server? Or…
aftbit on HN
Questions
Can it bypass Cloudflare?
From 1 comment
Can this bypass cloudflare?
dannyw on HN
Is there a decent source of proxies?
From 1 comment
This looks really interesting, I'm going to bookmark it! > Proxy IP address rotation I wonder where you would get a decent proxy list nowadays. In the late 90s, you could find…
captainmuon on HN
Alternatives named
microlink.io
From 1 comment
These guys bypass cloudflare https://microlink.io it's open source but it's just more convenient to pay and forget about it.
new_guy on HN