Unify
AI
llm routing
SHOW HN2024
298points
126comments
#208of 3,515 in 2024
Top 6%of 2024 by points
How HN took it
CuriousMostly questions and neutral comments
9 positive, 26 neutral or asking, 2 mixed, 12 critical, out of 49 comments about the project. Thread score +0.07.
Idea: 5 for, 7 againstPricing: 0 for, 2 againstExecution: 0 for, 2 againstPrivacy: 1 for, 0 againstDocs: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful idea for scaling LLM apps and cost control
From 2 comments
I wonder, have you also done any work on actually improving performance by enhancing the prompt or something similar? I always thought a product like this that could empirically…
aleksiy123 on HNThis is interesting! Sort of a super mixture of experts model. What's the latency penalty paid with your router in the middle? The pattern I often see is companies prototyping on…
lovesdogsnsnow on HN
Well executed product
From 1 comment
Very well executed. Congrats. Signed up. The chatbot providers will enjoy seeing how they stack up against each other. Your data/analytics/alerts will be valuable to them.…
billylo on HN
Objections
Prompt changes across models make treating LLMs as interchangeable bogus
From 1 comment
Sorry but I hate stuff like this We spend $30k+ per month with OpenAI and Anthropic Even minor prompt changes between minor model versions can have huge differences in output We…
weird-eye-issue on HN
No clear business model; no margin taken yet
From 3 comments
> Pricing-wise, we charge the same rates as the backend providers we route to, without taking any margins. We also give $50 in free credits to all new signups. What's your plan…
jefftk on HNThis is interesting! Sort of a super mixture of experts model. What's the latency penalty paid with your router in the middle? The pattern I often see is companies prototyping on…
lovesdogsnsnow on HNSince you take no extra charges from the end user, the presumption is you would charge LLM's for giving them traffic?
thomasfromcdnjs on HN
Router adds latency to time-to-first-token
From 2 comments
This is interesting! Sort of a super mixture of experts model. What's the latency penalty paid with your router in the middle? The pattern I often see is companies prototyping on…
lovesdogsnsnow on HNSo the neural scoring introduces ~20ms latency, but this only impacts time-to-first-token (not inter-token-latency). When using our public endpoints there is an additional ~150ms…
danlenton on HN
Requests
Local or on-prem deployment
From 2 comments
It's a good options to test local models I guess.
Sontho on HNactually we don't support local deployment quite yet, it's all run in our server. However, fully local deployment is on the roadmap, stay tuned!
danlenton on HN
Support for bring-your-own API keys
From 1 comment
Another point here is that some users prefer to use their own API keys for the backend providers (a feature we're releasing soon). Any "discounts" would then be harder to…
danlenton on HN
Questions
How the company will make money long term
From 2 comments
> Pricing-wise, we charge the same rates as the backend providers we route to, without taking any margins. We also give $50 in free credits to all new signups. What's your plan…
jefftk on HNThis is interesting! Sort of a super mixture of experts model. What's the latency penalty paid with your router in the middle? The pattern I often see is companies prototyping on…
lovesdogsnsnow on HN
Whether using GPT4 as judge for training labels is allowed
From 1 comment
> It is trained in a supervised manner on several open LLM datasets, using GPT4 as a judge. Does this mean GPT4 predictions are used as labels? Is that allowed?
iknownthing on HN
Alternatives named
OpenRouter
From 1 comment
definitely similar! I'm a fan of Alex and his work on OpenRouter :) Some of the main differences would be: - we focus on performance based routing, optimizing speed, cost and…
danlenton on HN
Semantic router
From 1 comment
Thanks for sharing! These are useful toos, but they are a bit different, more based on similarity search in prompt space (a bit like semantic router:…
danlenton on HN
Mixture of experts models
From 2 comments
Isn't this what a MoE LLM does already?
3abiton on HNI wonder, have you also done any work on actually improving performance by enhancing the prompt or something similar? I always thought a product like this that could empirically…
aleksiy123 on HN