LLM-aided OCR
Productivity
ocr document conversion
SHOW HN2024
How HN took it
6 positive, 29 neutral or asking, 2 mixed, 18 critical, out of 55 comments about the project. Thread score -0.20.
Execution: 1 for, 11 againstIdea: 4 for, 3 againstPerformance: 0 for, 2 againstPricing: 0 for, 1 againstPrivacy: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Multi-stage prompting with cheap fast models works well
From 1 comment
"real improvements came from adjusting the prompts to make things clearer for the model, and not asking the model to do too much in a single pass" This is spot on, and it's the…
kbyatnal on HN
Useful for turning old scanned books into readable markdown
From 1 comment
Agreed, this should not be used for anything mission critical unless you're going to sit there and carefully review the output by hand (although that is still going to be 100x…
eigenvalue on HN
Objections
LLM can silently alter numbers and names, risky for critical data
From 3 comments
Having tried this in the past, it can work pretty well 90% of the time. However, there are still some areas it will struggle. Imagine you are trying to read a lease contract. The…
janalsncm on HNThis is how you end up with "Xerox scanners/photocopiers randomly alter numbers in scanned documents"…
rasz on HNIt's a very interesting idea, but the potential for hallucinations reminds me of JBIG2, a compression format which would sometimes substitute digits in faxed documents:…
jonathanyc on HN
Hallucinations in OCR output are hard to detect
From 2 comments
It's a very interesting idea, but the potential for hallucinations reminds me of JBIG2, a compression format which would sometimes substitute digits in faxed documents:…
jonathanyc on HNThis is how you end up with "Xerox scanners/photocopiers randomly alter numbers in scanned documents"…
rasz on HN
Requests
Measure word and character error rates with and without the LLM
From 1 comment
I'd suggest measuring the word- and character error rates with and without the LLM. It'll let people quickly know how well it works.
esafak on HN
Handle two-column text and formulas better
From 2 comments
The iOS app would likely not handle two-column text very well. I really made the iOS app on a lark for personal use, the whole thing took like 2 hours, and I'd never even made a…
eigenvalue on HNI use Google lens for OCR 15th century Latin books — then paste to ChatGPT and ask to correct OCR errors. Spot checking, it is very reliable. Then translation can occur
dr_dshiv on HN
Questions
How does it handle bogus text in the scanned input
From 1 comment
This assumes that input text actually is well formed, right? If I scan a page containing bogus text / typos, this will actually correct those mistakes in the output, right?
echoangle on HN
Risk of safety filters or prompt injection in scanned documents
From 1 comment
Something that makes me nervous about this general approach is the risk of safety filters or accidental (or deliberate) instruction following interfering with the results. I want…
simonw on HN
Alternatives named
PaddleOCR (PPOCR)
From 1 comment
This is a wonderful idea, but while I appreciate the venerable Tesseract I also think it's time to move on. I personally use PaddlePaddle and have way better results to correct…
geraldog on HN
Nougat and Donut
From 1 comment
In my experience, this works well but doesn't scale to all kinds of documents. For scientific papers; it can't render formulas. meta's nougat is the best model to do that. For…
troysk on HN
Zerox (GPT-4o-mini OCR)
From 1 comment
Very recently we had Zerox [0] (Pdf -> Image -> GPT4o-mini based OCR) and I found it to work fantastically well) Would be curious about comparisons between these. [0]…
jdthedisciple on HN
Gemini Flash 1.5 for page transcription
From 1 comment
I think Gemini Flash 1.5 is the best closed-source model for this. Very cheap. Particularly compared to GPT4o-mini, which is priced the same as GPT4 for image input tokens.…
__jl__ on HN
Apple iOS built-in OCR
From 1 comment
Cool, I know there's a little Windows Power Toy for doing something similar: https://learn.microsoft.com/en-us/windows/powertoys/text-extractor But the built-in functionality for…
eigenvalue on HN