Apify Page Analyzer
Web and SaaS
web scraping tool
SHOW HN2018
How HN took it
6 positive, 11 neutral or asking, 1 mixed, 6 critical, out of 24 comments about the project. Thread score 0.00.
Execution: 0 for, 4 againstIdea: 2 for, 1 againstDesign: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful, handy tool for scraping and analysis
From 3 comments
We use this for some stuff at my office. It's handy.
chadlavi on HNJust wanted to say great tool!
nickthemagicman on HNLove this.
alexroan on HN
Clever JavaScript DSL idea
From 1 comment
why ? and how would you envisage authentication support to work ? (p.s. awesome tool guy...the clever part is to allow user the ability to describe their process via JavaScript.…
lozzo on HN
Objections
Cannot scrape authenticated sites or POST-based forms
From 2 comments
Aha! I see, it shows data based on POST request from FORM on this page http://www.dsden93.ac-creteil.fr/spip/spip.php?page=annu1d so if you provide just a link to the results…
jardah on HNSadly not for now. Our company has a solution for that (for some websites), but currently this tool does not have this functionality, since I wanted it be as simple as possible.…
jardah on HN
Fails on some sites such as Amazon ratings and certain tables
From 2 comments
hmm, no joy getting rating information from an amazon.com product page
petagonoral on HNNot giving me anything useful on this pretty straightforward table: http://www.dsden93.ac-creteil.fr/spip/spip.php?page=ecoles
guilamu on HN
UX: Analyze button hidden below the input
From 1 comment
I entered a URL and pressed enter, wondering why nothing happened. Only then did I scroll down to find the 'Analyze' button. I wasn't looking for specific attributes, and the…
tcmb on HN
Requests
Authentication support
From 2 comments
Authentication support on this would make it an instant purchase for me.
lanewinfield on HNIs there a way to access an authenticated web site?
BrandoElFollito on HN
RDFa support
From 1 comment
Nice work! It seems that it only supports microdata and not RDFa at the moment?
oevi on HN
Listing all page assets by type
From 1 comment
Suppose I wanted to extract an image that gets loaded async via Javascript (For example, a Pinterest page). How would that work? Looking at your documentation, it looks like I…
JustARandomGuy on HN
Recording user actions like clicks and form inputs
From 2 comments
Awesome tool! How do you handle scraping data that's hiding behind layers of ~fancy~ JS libraries? Is it as simple as triggering click events, pausing for loading, and then…
at_smith on HNCould be an interesting enhancement. Sounds a little bit like what Capybara, a test framework for Ruby apps can do[1], things like ``` click_link('Link Text') fill_in('Password',…
cseelus on HN
Questions
How it handles JavaScript-heavy pages
From 1 comment
Awesome tool! How do you handle scraping data that's hiding behind layers of ~fancy~ JS libraries? Is it as simple as triggering click events, pausing for loading, and then…
at_smith on HN
Whether it works behind WAFs
From 1 comment
Looks awesome! Does the tool work when trying to access websites behind web application firewalls? eg. F5 WAF [1] https://f5.com/glossary/web-application-firewall
cstrat on HN
Whether it handles digital certificates
From 1 comment
Is it able to deal with digital certificates?
rmateus on HN
Alternatives named
Capybara
From 1 comment
Could be an interesting enhancement. Sounds a little bit like what Capybara, a test framework for Ruby apps can do[1], things like ``` click_link('Link Text') fill_in('Password',…
cseelus on HN
Horseman/PhantomJS
From 1 comment
I'd have money on using Horseman with phantomjs in node.
razki on HN
Microlink
From 1 comment
Similar but just for getting normalized metadata: https://microlink.io
Kikobeats on HN
Feedity
From 1 comment
Cool tool! * shameless plug * Our little startup, Feedity - https://feedity.com, helps create custom RSS feeds for any webpage, utilizing Chrome for full-rendering and many other…
nreece on HN