Read text and objects in images

Status

Variants

Read text and codes in images

Extract printed text, tables or barcodes from images by optical character recognition.

Recognise objects in images

Classify images or detect the objects and shapes they contain.

What other tools call it

Meridian's classification; an independent blind rating agreed on 86.2% of 312 rows (exact class, version 2).

Background

When to use it

OCR is widely used as a form of data entry from printed paper records, such as passport documents, invoices, bank statements, receipts and business cards.^[1] Recognition of typewritten Latin-script text is still not 100% accurate, even where clear imaging is available.^[1] One study of 19th- and early 20th-century newspaper pages concluded that the character-by-character accuracy of commercial OCR software varied from 81% to 99%.^[1] Recognition of cursive text has rates even lower than recognition of hand-printed text.^[1] Without word context from a lexicon, a list of words allowed to occur, a character error rate of 1% may become an error rate of 5% or worse when each whole word must be recognised with no incorrect letters.^[1] Judge OCR output by the words and fields it gets wholly right, not by its character accuracy, and check a sample by eye before a column built from it is trusted. (Meridian's view)

Tesseract is a free, open-source OCR engine, released under the Apache License.^[3] Tesseract is optimised by default to recognise sentences of words, and disabling its dictionaries should help when most of the text, such as receipts, price lists or codes, is not dictionary words.^[4] Tesseract is known to have a problem recognising text and data in tables without custom segmentation or layout analysis.^[4] A scanned table is the case to plan for: expect to find the table's cells first and read each one, rather than reading the page as running text. (Meridian's view)

In Alteryx's Barcode tool, a barcode read from an image should not be blurry or too small, because poor image quality might affect the results.^[5] Object detection is significantly more difficult when the images it is run on are sampled from a different data distribution from its training data, which is often the case.^[6] The best object recognition algorithms still struggle with objects that are small or thin, and with images that have been distorted with filters.^[2]

The method

OCR software often pre-processes images to improve the chances of recognition, with steps such as de-skewing, despeckling, binarisation, line removal and layout analysis.^[1] Core OCR algorithms are of two basic types, matrix matching and feature extraction, which may produce a ranked list of candidate characters.^[1] Matrix matching compares an image to a stored glyph pixel by pixel, works best with typewritten text and does not work well when new fonts are encountered.^[1] Feature extraction decomposes glyphs into features such as lines, closed loops, line direction and line intersections, and is commonly seen in most modern OCR software.^[1] OCRopus and Tesseract use neural networks trained to recognise whole lines of text instead of focusing on single characters.^[1] OCR accuracy can be increased by constraining the output to a lexicon, which can be problematic when a document contains words not in it, such as proper nouns.^[1] The Tesseract engine was developed as proprietary software at Hewlett-Packard labs between 1985 and 1994, and released as open source in 2005.^[3]

Every object class has its own special features that help in classifying it, as all circles are round, and object class detection uses these features.^[6] The best algorithms for recognition tasks such as object classification and detection are currently based on convolutional neural networks.^[2] The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object classification and detection, with millions of images and 1000 object classes.^[2] Alteryx's Image Recognition tool classifies images by group, with a model trained on the user's own labelled images or one of the pre-trained models it provides.^[7] In RapidMiner, a region of an image can serve as a bounding box from an object detection model, carrying a class id and a confidence value between 0 and 1.^[8]

Compared with the alternatives

Object recognition recognises objects or object classes, identification recognises one individual instance such as a specific person's face, and detection scans image data for specific objects along with their locations.^[2] Detection based on relatively simple and fast computations is sometimes used to find smaller regions of interesting image data for more computationally demanding techniques to analyse.^[2] OCR targets typewritten text, while intelligent character recognition (ICR) also targets handwritten printscript or cursive text, usually involving machine learning.^[1] Reach for OCR only when the text exists solely as pixels; where a file already carries its text, read that text instead. (Meridian's view)

Sources

1. [1]Optical character recognition - Wikipediasecondary source
2. [2]Computer vision - Wikipediasecondary source
3. [3]Tesseract (software) - Wikipediasecondary source
4. [4]Improving the quality of the output | tessdocdocumentation
5. [5]Barcodedocumentation
6. [6]Object detection - Wikipediasecondary source
7. [7]Image Recognitiondocumentation
8. [8]Add Region - Rapidminer Documentationdocumentation