Boislisle Saint-Simon Memoirs OCR
Learning and science
ocr of historical texts
SHOW HN2024
126points
45comments
#684of 3,515 in 2024
Top 20%of 2024 by points
How HN took it
CuriousMostly questions and neutral comments
3 positive, 11 neutral or asking, 2 mixed, 0 critical, out of 16 comments about the project. Thread score +0.10.
Performance: 2 for, 0 againstExecution: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Interesting project and hard OCR problem
From 2 comments
"A very crude method would be to remove the last line every 16 pages but that would not be very robust if there were missing scans or inserts, etc. I prefer to check every last…
gregschlom on HNOCR to original structure is a really fun problem! I did something similar in an internship for newspapers pre-LLM Vision models, and it ended up being a bunch of interval…
lassenordahl on HN
Objections
Footnote and zone identification is hard for OCR and LLMs alike
From 3 comments
Getting the footnotes right is going to be really tricky. Sometimes I couldn't even read the superscript numbering on the original scans. And that was after zooming in to the…
complexworld on HNI just tested Gemini 1.5 Flash (interactively on Google AI Studio) and the results are far from acceptable. OCR seems good, on par with Google Vision. But the footnotes are not…
bambax on HNYes I periodically try to get scanned images of Medieval Latin and Hebrew books ocr’d and translated by Gemini and ChatGPT… sometimes the results are amazing, but you have to…
richard_todd on HN
Alternative OCR tools produced poor results on this book
From 1 comment
Thanks, but I'm sorry to say, the result is... bad. It invents words ("rusticas" in the second line of the title of the output, isn't anywhere in the source file -- it's not even…
bambax on HN
Requests
EPUB edition
From 1 comment
Getting the footnotes right is going to be really tricky. Sometimes I couldn't even read the superscript numbering on the original scans. And that was after zooming in to the…
complexworld on HN
Crowdsourced corrections interface
From 1 comment
For the human review part: maybe crowdsource it? Make the book available for reading online, with a UI to submit corrections (Wikipedia-style).
gregschlom on HN
Questions
How were missing scans and signature marks handled
From 1 comment
"A very crude method would be to remove the last line every 16 pages but that would not be very robust if there were missing scans or inserts, etc. I prefer to check every last…
gregschlom on HN
Alternatives named
Gemini or other vision LLMs
From 2 comments
Use a ~SoTA VLM like Gemini 2.0 Flash on the images. It’ll zero-shot de-hyphenated text in semantic HTML with linked footnotes.
wll on HN- There is no reason it should take 10-20 seconds per page. In my experience with complex documents of 20-30 pages it takes 3-4 seconds. - Please message me for more info if you…
ajcp on HN
Abbyy, Surya, Tesseract, LLMWhisperer
From 3 comments
You could upload the books to the Internet Archive and let their OCR pipeline take a try. It is (or at least was) written around Abbyy. Results weren't great but they were a…
throwaway81523 on HNI’ve used Surya (https://github.com/VikParuchuri/surya) before. It is very good (on par with Google Vision, potentially better layout analysis), but yours is a challenging use…
bondeau on HN> The "best" models just made stuff up to meet the requirements. They lied in three ways: > The main difficulty of the is project lies in correctly identifying page zones;…
constantinum on HN
ScanTailor
From 1 comment
Oh wow! I've worked on turning PAIP (Paradigms of Artificial Intelligence Programming) from a book into a bunch of Markdown files, but that's "only" about a thousand pages long,…
pronoiac on HN