kafka-worker
Infrastructure
message broker
SHOW HN2022
How HN took it
5 positive, 11 neutral or asking, 1 mixed, 1 critical, out of 18 comments about the project. Thread score +0.15.
Idea: 2 for, 1 againstDesign: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Cool demonstration of distributed systems built on Durable Objects
From 2 comments
This is really cool as a demonstration of how distributed systems can be built on top of Durable Objects. In particular I was really happy to see this: > What about replication?…
kentonv on HNThis is so neat, great demonstration... good work Max!
kabircf on HN
Neat proof of concept
From 2 comments
Ah yep, so it's effectively a proxy in front of the broker(s) - like Pulsar's proxy (that Kafka really needs to copy IMO). And kudos mate, it's a very cool way to prove the…
EdwardDiego on HNthis is cool. edge and decentralized applications are going to be a topic of focus (IMO) in the next 5 years.
teknopurge on HN
Objections
Not practical for serious use; throughput may be too low
From 1 comment
Producers (and all clients) can only see and interact with the gateway worker. DOs aren’t accessible from the outside world, and all communication with them happens through the…
maxwellpeterson on HN
Durable Objects limit transactions per object and cross-object transactions are hard
From 2 comments
The concurrency with Durable Objects when it comes to mutating state is effectively 1: https://developers.cloudflare.com/workers/learning/using-durable-objects/ Parallel requests…
ec109685 on HNDOs are arguably the wrong primitive if you're looking to do transactions across more than one of them. They're essentially partitions, which is how this project uses them. If…
bastawhiz on HN
Requests
Support newer Kafka protocol versions (0.10+) for client compatibility
From 1 comment
Yep, one of the challenges here is that KafkaJS requires Kafka 0.10+, which is true for several other popular clients. For this broker implementation to be somewhat practical, it…
maxwellpeterson on HN
Implement Elasticsearch-style search on Durable Objects
From 1 comment
Now, do Elasticsearch, please.
k__ on HN
Questions
How is replication and leader election handled?
From 2 comments
I feel like this is a dumb question. But how exactly does Durable Objects help you avoid replication and leadership election? I've read the high level overview of Durable…
cjdoc29 on HNSo every topic has a (Kafka) replication factor of 1, but how do producers find their partition leader if the DO fails? From my understanding, all the DOs are effectively the…
EdwardDiego on HN
What can this be used for?
From 2 comments
What can I do with this ?
m00dy on HNwhat are the advantages of having this design ?
m00dy on HN
Alternatives named
Pulsar proxy
From 1 comment
Ah yep, so it's effectively a proxy in front of the broker(s) - like Pulsar's proxy (that Kafka really needs to copy IMO). And kudos mate, it's a very cool way to prove the…
EdwardDiego on HN
BigQuery
From 1 comment
That's super cool ! Also, with D1 coming, I can't wait to see an alternative to BigQuery built on CF workers!
greatNespresso on HN