WhisperFusion
AI
voice assistant
SHOW HN2024
272points
102comments
#252of 3,515 in 2024
Top 8%of 2024 by points
How HN took it
CuriousMostly questions and neutral comments
13 positive, 46 neutral or asking, 4 mixed, 6 critical, out of 69 comments about the project. Thread score +0.12.
Idea: 7 for, 0 againstPerformance: 3 for, 4 againstExecution: 1 for, 2 againstDesign: 1 for, 0 againstPrivacy: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Impressive low-latency voice interaction
From 2 comments
Great to hear its seamless real-time ultra low-latency. Hopefully the next iteration is blazingly fast too!
ramon156 on HNVery neat capability. Need to see more of hyper-optimizing models to one specific use case, this is a great example of doing so.
asynchronous on HN
Excellent packaging and project
From 1 comment
This is an excellent project with excellent packaging. It is primarily a packaging problem. Why does every Python application on GitHub have its own ad-hoc, informally specified,…
doctorpangloss on HN
Objections
Whisper is not designed for streaming; latency and WER unclear
From 3 comments
This is using phi-2, so the first assumption would be that it's local. It's a tiny little model in the grand scheme of things. I've been toying around with something similar…
regularfry on HNSee also the blog post: https://www.collabora.com/news-and-blog/news-and-events/whisperfusion-ultra-low-latency-conversations-with-an-ai-chatbot.html WhisperFusion, WhisperLive,…
albertzeyer on HNAh, but that sounds like a very inefficient approach, which probably still has quite high latency, and probably also performs bad in terms of word-error-rate (WER). But I'm happy…
albertzeyer on HN
Lacks interruption support
From 2 comments
There are two things that I think are needed and that I'm not sure if anyone provides yet to make this scenario work well: 1. Interruption - I need to be able to say "hang on"…
localhost on HNIndeed a great point. Waiting for a specific cue, before responding, is an interesting idea. It would make the interaction more natural, especially in situations where the user…
stiffler01 on HN
Hard to install for ordinary users
From 1 comment
We see these tools such as this posted several times a week. Is there any expectation they will be installable by the common person? Where is the setup.exe, .deb, .rpm, .dmg?
abdullahkhalids on HN
Requests
Interruption support
From 2 comments
There are two things that I think are needed and that I'm not sure if anyone provides yet to make this scenario work well: 1. Interruption - I need to be able to say "hang on"…
localhost on HNThanks! Really looking forward to interruptions. I think about the cue as kind of being like "Hey Siri/Alexa/Cortana" but in reverse.
localhost on HN
Latency and WER benchmark numbers
From 2 comments
See also the blog post: https://www.collabora.com/news-and-blog/news-and-events/whisperfusion-ultra-low-latency-conversations-with-an-ai-chatbot.html WhisperFusion, WhisperLive,…
albertzeyer on HNYes, but when you change Whisper to make it live, to get WhisperLive, surely this has an effect on the WER, it will get worse. The question is, how much worse? And what is the…
albertzeyer on HN
Cue-based turn taking
From 2 comments
There are two things that I think are needed and that I'm not sure if anyone provides yet to make this scenario work well: 1. Interruption - I need to be able to say "hang on"…
localhost on HNIn order to feel like a human, cues should not be a pre-programmed phrase, the system should continuously listen to the conversation, and evaluate constantly if speaking is…
Valgrim on HN
Questions
Does it run locally or call remote APIs?
From 1 comment
I had a quick read through, maybe I missed something but is this all local run or does it need api access to OpenAI’s remote system? The reason I ask is that I’m building…
WhackyIdeas on HN
Which GPUs are supported, can it run on a Jetson?
From 1 comment
Seeing that this uses TensorRT (i.e. seems well optimized), what GPUs are supported? Could I run this on a Jetson?
codethief on HN
Can TensorRT be replaced with CoreML?
From 1 comment
Does anyone know if you can replace TensorRT with a similar call to Apple's CoreML for the same functionality?
jdkee on HN
Alternatives named
Vocode
From 1 comment
This post reminded me of Vocode: https://github.com/vocodedev/vocode-python Discussion on them here from 10 months ago: https://news.ycombinator.com/item?id=35358873 I tried the…
monkeydust on HN
Kaldi streaming ASR models
From 1 comment
whisper is simply not designed for this, in many ways, and it's impressive engineering to try and overcome its limitations, but I can't help but feel that it is easier to just…
huac on HN
Persona / SmarterChild voice demos
From 1 comment
> Interruption Well, today is your lucky day!: https://persona-webapp-beta.vercel.app/ and the demo https://smarterchild.chat/
lambdaba on HN