Hivemind
Infrastructure
distributed job processing
SHOW HN2017
66points
35comments
#657of 2,242 in 2017
Top 30%of 2017 by points
How HN took it
MixedNo clear lean
1 positive, 2 neutral or asking, 0 mixed, 2 critical, out of 5 comments about the project. Thread score -0.13.
Idea: 0 for, 1 againstPerformance: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful for processing large data quickly
From 2 comments
>How is this working for you? Where do you see it going? It actually works quite well. I use it for processing large amounts of data quickly. I originally built this to process…
madamelic on HNI definitely appreciate that, and I didn’t mean to take anything away from your work. Cheers.
dvdhnt on HN
Objections
Tied to a single vendor (AWS)
From 1 comment
I’m sure this is great, but for some of us, being tied to a single vendor is a non-starter.
dvdhnt on HN
Lambda retry and timeout limits are restrictive
From 2 comments
Thanks for sharing! I ended up building a similar featureset in Python but using invocation type Event instead of RequestResponse. This ended up being a pain in the ass because…
teej on HNFYI, I believe you can disable retries and replace them with a DLQ (which you can then ignore, if you don't care about failures). I would be interested, though, if anyone has…
ak217 on HN
Requests
Support for chaining and orchestration of Lambdas
From 2 comments
Most of the things I've added are for logging and orchestration. The most important unlock for us was the ability for us to enable chaining of Lambdas that worked seamlessly with…
teej on HNAmazon doesnt really give you any direct access to the underlying "worker" so it doesnt work like that. It's really as simple as: - the active lambda decides to stop performing…
teej on HN
Questions
How is it working in practice and where is it going
From 1 comment
Thanks for sharing! I ended up building a similar featureset in Python but using invocation type Event instead of RequestResponse. This ended up being a pain in the ass because…
teej on HN
Alternatives named
PyWren
From 1 comment
More mature python based solution. http://pywren.io/
posnet on HN
AWS Step Functions
From 1 comment
Have you tried Step Functions? I've used them to orchestrate Lambdas with good success when I needed something that will run longer than Lambdas can individually.
joseph on HN
SQS with Lambda
From 1 comment
There's an SQS/Lamba integration. Push 10k messages in a queue as jobs. The lambdas pull them off and delete them once done.…
tyingq on HN