Tokencost
Dev tools
cost estimation library
SHOW HN2024
How HN took it
2 positive, 20 neutral or asking, 0 mixed, 2 critical, out of 24 comments about the project. Thread score +0.01.
Idea: 2 for, 1 againstPricing: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful for comparing model costs
From 2 comments
wow, this is really useful!! Just the price list alone has given me a lot of inspiration, thank you
jaredliu233 on HNGemini also only charges for output tokens, not sure if that’s considered. All in all this is something I was looking for or was roughly going to do to compare costs. Cool stuff.
neverokay on HN
Good to have a price list in one place
From 1 comment
wow, this is really useful!! Just the price list alone has given me a lot of inspiration, thank you
jaredliu233 on HN
Objections
Claude models listed without a real tokenizer, so counts may be inaccurate
From 3 comments
I don't understand how the Claude functionality works. As far as I know Anthropic haven't released the tokenizer for Claude - unlike OpenAI's tiktoken - but your tool lists the…
simonw on HN"This package can be used to count tokens for Anthropic's older models. As of the Claude 3 models, this algorithm is no longer accurate [...]" I've been bugging Anthropic about…
simonw on HNSome companies like Anthropic haven't publicly released the tokenizer used in their API, thereby making it impossible for this library to use the appropriate tokenizer in all…
spencerchubb on HN
Some models show zero cost due to precision or data issues
From 1 comment
A whole bunch of the costs are listed as zeroes, with multiple decimal points. I noticed y'all used the Decimal library and tried to hold onto precision so I'm not sure what's…
Karrot_Kream on HN
Per-token pricing is hard to compare across models
From 2 comments
Can you do a column and normalize them? Too many zeroes for my blind ass making it hard to compare.
yelnatz on HNYeah a Tokens per $1 column would vastly help the readability.
ryaneager on HN
Requests
Add Claude tokenizer support
From 2 comments
> # TODO: Add Claude support There are no cases for Claude models yet. I wonder if anyone has run a bunch of messages through Anthropic's API and used the returned token count to…
refibrillator on HNAnthropic actually has a Claude 3 tokenizer tucked away in one of their repos: https://github.com/anthropics/anthropic-tokenizer-typescript At this moment, Tokencost uses the…
Areibman on HN
Add image and vision cost calculation
From 2 comments
Nice. Any plans to add calculations for image input for the models that allow that?
ilaksh on HNNice. Does it add Vision costs too?
jacobglowbom on HN
Show tokens per dollar or normalized price column
From 2 comments
Can you do a column and normalize them? Too many zeroes for my blind ass making it hard to compare.
yelnatz on HNYeah a Tokens per $1 column would vastly help the readability.
ryaneager on HN
Add a Rust version
From 1 comment
Would anybody be interested in this for Rust? I already do everything this library does with the exception of returning the price in my LLM utils crate [1]. I do this just to…
J_Shelby_J on HN
Questions
How are Claude 3 token counts computed without a tokenizer
From 1 comment
I don't understand how the Claude functionality works. As far as I know Anthropic haven't released the tokenizer for Claude - unlike OpenAI's tiktoken - but your tool lists the…
simonw on HN
Does it account for function calls and images
From 1 comment
Are you also accounting for costs of sending images and function calls? I didn't see that when I looked through the code. I developed this package so that I could count those…
pamelafox on HN
Alternatives named
LiteLLM model price file
From 1 comment
Very cool! Is this cost directory you're using the best source for historical cost per 1M tokens? https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json
zackfield on HN
openai-messages-token-helper
From 2 comments
Are you also accounting for costs of sending images and function calls? I didn't see that when I looked through the code. I developed this package so that I could count those…
pamelafox on HNI have a package here that includes calculation for images for OpenAI: https://github.com/pamelafox/openai-messages-token-helper
pamelafox on HN
anthropic-tokenizer
From 1 comment
Sorta: https://github.com/javirandor/anthropic-tokenizer
GranPC on HN