Digitising a historical collection is mostly not a scanning problem. The images are the easy part; turning them into text a researcher can search is where projects stall, and the decisions that matter are made early — which engine, which segmentation approach, and how many pages someone is going to transcribe by hand before any of it works. The open-source tooling for this is genuinely good now and runs entirely on Linux, but the options are not interchangeable and the wrong choice is expensive to unwind. What follows is how the pieces fit together.
On this page
- What Separates OCR From HTR
- Why Tesseract Is Still the Default
- Where Kraken Outperforms It
- eScriptorium as the Interface Layer
- OCR4all for End-to-End Work
- Layout Analysis Decides Everything Downstream
- Ground Truth Is the Real Cost
- Measuring Accuracy Honestly
- Running the Pipeline on Linux
- What Changes When Transcriptions Exist
What Separates OCR From HTR
Optical character recognition reads printed type; handwritten text recognition reads script. The distinction is not pedantry — it decides which engine can touch your material at all, because the two problems have different failure modes and different training requirements.
Printed text has consistent glyph shapes, so a model trained on one typeface generalises reasonably to another. Handwriting does not: letterforms vary between writers, within a single writer, and across a page as the hand tires. This is why Tesseract, the most widely deployed open-source engine, does not do handwriting out of the box — its bundled models were never trained on handwritten data. Pointing it at a nineteenth-century letter book produces output that looks like text and means nothing.
Why Tesseract Is Still the Default
Tesseract remains the baseline in the cultural heritage community because it is mature, packaged in every major Linux distribution, and good enough for clean printed material. For a twentieth-century typed report or a modern printed book, it is frequently the correct answer and requires no training at all.
Its limits appear with age and layout complexity. Early printed books with irregular inking, ligatures and long-s forms defeat the stock models, and Tesseract's layout analysis struggles with multi-column pages, marginalia and tabular material — exactly the features that make historical documents interesting. Institutions typically keep Tesseract for the modern end of a collection and reach for something else at the point where the type stops behaving.
Where Kraken Outperforms It
Kraken is an Apache-licensed recognition engine built specifically for historical and non-Latin scripts, and on difficult material it has demonstrated better accuracy than Tesseract in many published comparisons. It was designed from the outset around the assumption that the page is awkward.
The practical advantages are trainability and script coverage. Kraken trains custom models from modest quantities of transcribed ground truth, which matters when a collection has its own scribal hand or an unusual typeface, and it handles right-to-left and bidirectional scripts that Latin-first engines treat as an afterthought. A modular pipeline pairing Kraken with a sequence model for post-correction has been applied to early printed books with strong results — the recognition and the correction are separate stages, each replaceable.
eScriptorium as the Interface Layer
eScriptorium is an open-source platform for historical document analysis whose underlying recognition is Kraken. It supplies what a bare engine does not: a browser interface for segmenting pages, correcting transcriptions and training models without writing code.
The comparison people reach for is Transkribus, which remains popular in handwritten document analysis despite being closed-source. eScriptorium covers much of the same workflow while leaving the data and the models on infrastructure you control — a consideration for archives that cannot export material to a third-party service, whether for licensing, privacy or institutional policy reasons. Running it means running a server, a database and a queue, which is a real operational commitment rather than an install.
OCR4all for End-to-End Work
OCR4all packages layout analysis, recognition and model training into a single end-to-end workflow, and has reached top-tier accuracy on recent historical OCR benchmarks. It targets the researcher who needs a finished transcription rather than a pipeline to maintain.
The design assumption is different from the others. Kraken is a component; OCR4all is a process, walking material through segmentation, ground-truth production, training and recognition in a defined order. For a single collection processed once, that structure saves considerable time. For a repository ingesting new material continuously, the same structure can feel constraining, and teams often end up extracting the pieces they need. Which suits depends on whether digitisation is a project or a standing function.
Layout Analysis Decides Everything Downstream
Segmentation — deciding what is a text region, a line, a marginal note and a decorative initial — determines the ceiling on final accuracy, because a perfectly recognised line placed in the wrong reading order is still wrong. Most disappointing results trace back to this stage rather than to character recognition.
Historical pages are adversarial here. Common structures that break naive segmentation include:
- Multi-column text with rules that read as characters
- Marginal notes running vertically beside the main block
- Catchwords and signature marks at the foot of the page
- Tables where column alignment carries the meaning
- Bleed-through from the reverse of a thin leaf
Scholarly typography of this kind has a long documented history, and the conventions collected at Archiva are a useful reference for what the apparatus on an early page is actually doing before you ask software to interpret it.
Ground Truth Is the Real Cost
Ground truth — pages transcribed by hand to train and measure a model — is the dominant cost of any serious recognition project, and it is human time rather than compute. A custom model for an unusual hand typically needs tens of pages transcribed accurately before it becomes useful.
Projects consistently underestimate this and overestimate hardware. A workstation GPU trains a Kraken model in hours; producing the transcriptions to train it on takes weeks. The corollary is that ground truth is the most valuable artefact a digitisation project creates, more durable than the model trained from it, since engines are superseded while accurate transcriptions remain usable. Publishing it openly, in a documented format, is the single highest-return decision most projects can make.
Measuring Accuracy Honestly
Recognition accuracy is normally reported as character error rate, and a figure quoted without the material it was measured on is meaningless. The same engine can return under two per cent CER on clean printed text and over thirty per cent on a difficult hand.
Comparative work on folkloristic text digitisation has made the point sharply: pipeline rankings shift depending on the corpus, so an engine that wins one benchmark loses another. The practical response is to build a small held-out test set from your own material before choosing anything, and to measure candidates against it. Fifty representative pages is usually enough to separate a workable engine from an unworkable one, and it costs less than committing to the wrong pipeline for a year.
Running the Pipeline on Linux
All of these tools are Linux-native, which is why archival digitisation has consolidated on it. Tesseract and Kraken install from package managers or pip; eScriptorium and OCR4all ship container images, which is the sane way to run either given their dependency graphs.
The operational shape is unglamorous and matters more than engine choice. Master images stay untouched in preservation formats while derivatives feed the recogniser; the whole run is scriptable, so reprocessing a collection when a better model appears is a job submission rather than a project. Because every component is open source, a collection processed today can be reprocessed in five years without renegotiating a licence — which for an institution measuring its obligations in decades is the argument that actually decides it.
What Changes When Transcriptions Exist
The output of recognition is not a readable document but a searchable one, and that shift is the point. A box of unindexed correspondence becomes a corpus that can be queried, dated, cross-referenced and counted, and questions that were previously infeasible become ordinary.
The transcriptions also feed everything downstream: named-entity extraction to build indexes of people and places, linkage between collections held in different institutions, and increasingly the training corpora for models that will read the next collection. Work on historical database construction has shown entity extraction running directly on recognised text at scale. None of it is available while the material exists only as page images, which is the honest argument for spending the ground-truth weeks.
