Sneakers

Infrastructure

background job processing

SHOW HN2014

114points
37comments
#246of 2,272 in 2014
Top 11%of 2014 by points

How HN took it

CuriousMostly questions and neutral comments

4 positive, 12 neutral or asking, 0 mixed, 2 critical, out of 18 comments about the project. Thread score +0.18.

Idea: 2 for, 0 againstDesign: 0 for, 1 againstPerformance: 0 for, 1 againstDocs: 1 for, 0 against

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Fast throughput compared to Sidekiq in microbenchmark

    From 1 comment
    • Author here - Hi! :) Sneakers is a high-performance background-job processing framework based on RabbitMQ. It uses a hybrid process-thread model where many processes are spawned…
      jondot on HN
  • Good documentation of design rationale

    From 1 comment
    • Kudos to the OP for a thorough Wiki/documentation As always, the first question on my mind is "Why X, instead of A,B,C?" (sidekiq in this case). The OP's page is here:…
      danso on HN

Objections

  • Redis-based Sidekiq is less reliable than RabbitMQ clustering

    From 1 comment
    • More or less the same. Plus it will use all cores on MRI, so you can keep using gems with C-Extensions (i.e. you don't have to run rbx or JRuby to max all cores). Discussing…
      jondot on HN
  • Redis clustering not production-ready

    From 1 comment
    • Still in beta/alpha as far as I last checked (we've been waiting for it). Additionally as far as clustering is concerned there still seem to be a lot of 'unknowns' for me with…
      druiid on HN

Requests

  • Pluggable backends such as PostgreSQL

    From 2 comments
    • Have you considered pluggable backends (with a Sidekiq/Resque/Rediss backend) so you can easily upgrade a Sidekiq stack to Sneakers?
      judofyr on HN
    • How hard would it be to use a PostgreSQL backend with sneakers?
      joevandyk on HN
  • Batch job support

    From 1 comment
    • Wow, just finished deploying a rails app that needs a lot of scrapers with sidekiq and it was a great process but sneakers looks very nice! Is batches of job on the roadmap?…
      knes on HN
  • Web UI like Sidekiq's

    From 2 comments
    • Look forward to checking this out. The biggest win regarding Sidekiq for me wasn't so much performance (which is was definitely better than Resque) but it's retry logic, error…
      bdcravens on HN
    • The thing about Sidekiq is the management is closer to my app, but I'd of course need to look into Rabbit's UI before making any decisions. I suspect that if this project takes…
      bdcravens on HN

Questions

  • How does it compare to Celery and support result return

    From 1 comment
    • I have more of a django background, so I've never used any of the libraries that you compare this to. How does Sneakers compare to something like Celery…
      nat on HN
  • How to deploy and monitor RabbitMQ and workers

    From 1 comment
    • I'm a big sidekiq fan but you've got my interest piqued ... aside from the nice approach to retries/failures, the best thing about sidekiq for me is ease of production…
      callmeed on HN
  • How to return job completion status to frontend

    From 1 comment
    • My biggest obstacle with Sidekiq is communicating back to the frontend client when jobs are done (i.e. credit card processed, FacebookGraph friends cached, etc). Right now I use…
      jeffblake on HN

Alternatives named

  • Sidekiq

    From 2 comments
    • I have been using Sidekiq in production for over a year now, running over 11B jobs over that year, peaking around 20m jobs per day. I've been wanting to migrate away from it to…
      avitzurel on HN
    • Look forward to checking this out. The biggest win regarding Sidekiq for me wasn't so much performance (which is was definitely better than Resque) but it's retry logic, error…
      bdcravens on HN
  • Resque

    From 1 comment
    • Sidekiq's retry and robust handling of jobs (at least with reliable queuing in pro) was the big win indeed. I had a app I tried switching over to it though and it did not go…
      chadcf on HN
  • Celluloid

    From 1 comment
    • Yes. I dislike reinventing the wheel, I actually bumped my head against making Sidekiq use a different backend. Three problems - one - it is too tightly coupled to Redis.…
      jondot on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/7017580)