Scene Text Detection

Scene text detection finds the words and lines of text in photographs of the world, such as shop signs, street names, product packaging and posters, and returns a region for each: a box, a rotated rectangle, a quadrilateral or a polygon. It is also called text localization, the name of the corresponding ICDAR tasks. It is usually the first stage of reading text in images: a recognizer then transcribes each region, the problem of scene text recognition, and systems that do both are called end-to-end text spotters. Detection is a form of object detection, but its targets differ from ordinary objects: words are long and thin, appear at any angle and sometimes along curves, sit close to one another in lines, and have no fixed size. The task grew out of document optical character recognition, whose page-based pipeline does not carry over to natural scenes.

The field moved through three families of methods. Until about 2015, detectors found character candidates as connected components, with maximally stable extremal regions (MSER) or the stroke width transform (SWT), and grouped them into lines. From 2016 deep detectors adapted general object detectors, predicting boxes from anchors (TextBoxes, RRPN) or directly at every pixel (EAST). Segmentation-based detectors, which spread from 2018, labelled text pixels and separated touching words afterwards (PixelLink, PSENet, DBNet), which also handles curved text. Benchmarks moved in parallel, from horizontal words (ICDAR 2013) to incidental, rotated words (ICDAR 2015), long multi-lingual lines (MSRA-TD500) and curved text (Total-Text, CTW1500), and each brought its own matching rule. The ICDAR article covers the history and protocols of the ICDAR datasets.

Problem Definition

Given an image, a detector returns a set of regions, one for each text instance, each with a confidence score in most methods. A region is correct when it overlaps a ground-truth instance closely enough under the benchmark’s matching rule, and each ground-truth instance can be claimed once. Recall, precision and their harmonic mean, the F-score (also called H-mean or F-measure), summarize the result.

Three choices define a version of the problem:

• Granularity. The unit is a word in ICDAR 2013, ICDAR 2015 and Total-Text, and a text line in MSRA-TD500 and CTW1500. MSRA-TD500’s authors chose lines because Chinese text lines are hard to split into words by spacing, and even English lines need high-level information to split (Yao et al., 2012). A detector trained for one granularity is penalized on the other: EAST’s authors attribute the weaker MSRA-TD500 result of their VGG16 version to its receptive field, because that benchmark requires line-level output (Zhou et al., 2017).
• Geometry. The region can be an axis-aligned rectangle, a rotated rectangle, a quadrilateral or a polygon with many vertices. Curved text needs polygons: a rectangle around an arc of letters is mostly background.
• What is cared for. Benchmarks mark illegible text, text in other scripts and, in ICDAR 2015, very short words as “do not care” regions that neither count when found nor when missed.

Input and Output

The input is a colour photograph of arbitrary size. Benchmark images range from small web images (TextSnake’s authors note that the largest CTW1500 image is about 400 × 600 pixels) to camera images of 1296 × 864 to 1920 × 1280 pixels in MSRA-TD500 (Yao et al., 2012). ICDAR 2015 images were taken with Google Glass at 1280 × 720 (Zhou et al., 2017; Liao et al., 2020).

The output format follows the benchmark:

Format	Parameters	Used by
Axis-aligned rectangle	left, top, right, bottom	ICDAR 2013, COCO-Text
Rotated rectangle	centre, width, height, angle	MSRA-TD500 (minimum-area rectangles)
Quadrilateral	four corners, clockwise	ICDAR 2015, MLT
Polygon	many vertices	Total-Text (10 in the regulated training annotation), CTW1500 (14)

The geometric side of rotated boxes, including angle conventions, rotated intersection over union and rotated non-maximum suppression, is shared with aerial object detection, which covers it in detail. Text adds two features of its own: the reading direction (Total-Text’s regulated polygons start at the point where reading begins; Ch’ng et al., 2020) and the transcription carried by each ground-truth instance, which end-to-end evaluation uses.

Variants

• Horizontal, multi-oriented and curved text. The three orientation classes that Total-Text’s authors use to describe datasets (Ch’ng et al., 2020). Methods designed for one class often fail on the next: SWT’s authors list a curved baseline among their failure cases (Epshtein et al., 2010), and the first deep detectors could not represent curved regions at all.
• Word and line detection, as above.
• Multi-lingual detection, for example the ICDAR 2017 MLT dataset with 7,200 training, 1,800 validation and 9,000 test images in 9 languages (as described by Liao et al., 2020, and Wang et al., 2019).
• Character-level detection. CRAFT localizes individual characters and the links between them, and builds words from them (Baek et al., 2019).
• End-to-end text spotting, where detection and recognition are trained together; the recognition side belongs to scene text recognition.
• Text in video, where detections should also be stable from frame to frame; EAST’s authors note that their averaging form of non-maximum suppression gives stable output on video (Zhou et al., 2017).

Challenges

• Extreme aspect ratios. A line of text can be ten or more times longer than it is high. TextBoxes’ authors found that SSD, trained the same way as their detector, failed on words with large aspect ratios (Liao et al., 2017).
• Long instances and receptive fields. A network that predicts a whole box from one location must see the whole instance. EAST’s authors name this as their main limitation, since the largest text the detector can handle is proportional to the receptive field (Zhou et al., 2017).
• Adjacent instances. Words in a line, and lines in a paragraph, nearly touch. A plain segmentation of text pixels merges them, which motivated links (PixelLink), shrunk kernels (PSENet) and adaptive thresholds (DBNet).
• Text-like clutter. Windows, fences, bricks and foliage share the regular strokes of text. MSRA-TD500’s authors report that their false positives came mostly from windows, trees and signs that look like text (Yao et al., 2012).
• Incidental capture. Blur, low resolution, perspective and partial views; the ICDAR 2015 images were captured without aiming at the text.
• Annotation ambiguity. What counts as a word, where its boundary lies and how tightly a polygon should fit are decided by annotators. Total-Text’s authors report that even human annotations of the same text do not match each other perfectly, which led them to change the DetEval thresholds (Ch’ng et al., 2020).

Approaches

Connected components: MSER and the stroke width transform

Classical detectors fell into two groups, which Epshtein et al. (2010) call texture-based and region-based. Texture-based methods scan the image with windows at several scales and classify each window from features such as edge density or wavelet statistics, as in sliding-window detection; the SWT paper lists their cost and their difficulty with slanted text as limitations. Region-based methods first form connected components of pixels that might belong to one character and then filter and group them; this route became dominant.

Two operators supplied the components. Maximally stable extremal regions were introduced by Matas et al. (2002) for wide-baseline stereo; feature detection describes them among region detectors, as connected components of the thresholded image whose area changes least over a range of thresholds. Text characters, uniform in colour and sharply bounded, tend to be stable over a wide range of thresholds. TextBoxes’ authors describe Neumann and Matas (2012) as locating characters by classifying extremal regions and grouping them by exhaustive search (Liao et al., 2017), and the ICDAR 2015 report observed that almost all submitted methods started from MSER regions (as summarized in the ICDAR article).

The stroke width transform (Epshtein et al., 2010) starts from the observation that text, unlike foliage, has nearly constant stroke width. From each Canny edge pixel $p$ the transform follows the ray along the gradient direction $\mathbf{d}_p$ until it meets another edge pixel $q$; if the gradient there is roughly opposite, within $\pm\pi/6$, every pixel on the segment receives the width $\lVert p - q\rVert$ unless it already holds a smaller value. A second pass replaces values above the median of each ray by that median, to repair corners. Neighbouring pixels are then grouped when their stroke widths differ by a ratio of at most 3.0, and the algorithm runs twice, once for dark text on light background and once for the reverse. Components are rejected when their stroke-width variance is too large (above half the component’s mean width), when their aspect ratio falls outside 0.1–10, when their diameter exceeds ten times the median stroke width, or when their height lies outside 10–300 pixels; the thresholds were tuned on the ICDAR training set so that 99% of the true character components survive. Letters are then paired when their median stroke widths differ by less than a factor of 2.0, their heights by at most 2.0, their distance is at most three times the width of the wider letter and their average colours agree; pairs are chained when they share an end and run in a similar direction, chains of at least three letters become text lines, and a histogram of letter gaps splits lines into words. On the ICDAR 2003 data (258 training and 251 test images), scored with the 2003 measure, in which a detection’s match is the area of its intersection with the best ground-truth rectangle divided by the area of the smallest box containing both and precision and recall average these matches, the authors reported precision 0.73, recall 0.60 and $f$ 0.66 at 0.94 seconds per image, against 0.62 for the best other entry in their table.

Yao et al. (2012) kept SWT components but replaced hand-set rules with trained random-forest classifiers at two levels, components and chains of components, and let chains take any direction. With the MSRA-TD500 dataset they released, they showed how much the horizontal assumption cost: on its multi-oriented test set the SWT system reached an F-measure of 0.25 and their method 0.60 (Table 3; the same table gives precision and recall of 0.63 for their method, whose harmonic mean would be 0.63, so the table disagrees with the paper’s own definition of $f$).

Connected-component pipelines have many stages, each with its thresholds and heuristics: candidate extraction, filtering, pairing, line formation and word splitting. Errors compound, and EAST’s authors argue that the interplay of stages limits accuracy and speed (Zhou et al., 2017). This was the motivation for single-network detectors.

Regression from anchors: TextBoxes, CTPN, RRPN and SegLink

The first deep detectors that replaced the whole pipeline adapted general object detectors to text, regressing box coordinates as offsets from reference boxes (see bounding box regression and anchor boxes).

TextBoxes (Liao et al., 2017) modified SSD. It keeps VGG-16 up to conv4_3, converts the fully connected layers to convolutions and appends further layers, and attaches text-box output layers to six feature maps. Its changes target word shapes: default boxes with aspect ratios 1, 2, 3, 5, 7 and 10, each repeated with a vertical offset of half a cell because long boxes leave gaps vertically, and 1 × 5 convolution kernels whose rectangular receptive fields fit long words. Trained on the 800,000 synthetic images of SynthText and fine-tuned on ICDAR 2013, it reached an F-measure of 0.85 under the ICDAR 2013 evaluation and 0.86 under DetEval with five input scales (0.73 seconds per image on a Titan X), or 0.80 with a single 700 × 700 input in 0.09 seconds (Table 1). SSD trained with the same procedure reached 0.68. TextBoxes outputs horizontal rectangles only, and its authors named multi-oriented text as future work.

Other anchor-based detectors handled orientation and length in different ways:

• CTPN (Tian et al., 2016, ECCV) uses vertical anchors and a joint convolutional and recurrent model to detect horizontal text lines (as described by Zhou et al., 2017), extending Faster R-CNN’s region proposal network to predict text slices that heuristic rules then connect (as described by Deng et al., 2018). EAST’s table lists it at an F-score of 0.6085 on ICDAR 2015.
• RRPN (Ma et al., 2018, IEEE Transactions on Multimedia) adds rotation to both the anchors and the region-of-interest pooling of Faster R-CNN (as described by Deng et al., 2018). The paper was not read for this article.
• TextBoxes++ (Liao, Shi and Bai, 2018, IEEE Transactions on Image Processing) extends TextBoxes to oriented text with a single network (official repository README) by regressing quadrilaterals (as described by Liao et al., 2020). Its ICDAR 2015 result is listed differently by later papers: DBNet’s table gives precision 87.2, recall 76.7 and F 81.7 at 11.6 frames per second, while CRAFT’s table gives recall 78.5, precision 87.8 and H-mean 82.9 marked as multi-scale. The paper itself was not read for this article.
• SegLink (Shi et al., 2017) breaks text into locally detectable elements: segments, oriented boxes covering part of a word or line, and links between adjacent segments that belong to the same instance. An SSD-like network predicts both at six scales, and segments joined by links are combined into whole words. Because no prediction needs to see a whole line, SegLink detects long lines, including Chinese, without modification. Its authors report an F-measure of 75.0% on ICDAR 2015 and over 20 frames per second on 512 × 512 images.

Dense regression at every pixel: EAST

EAST (Zhou et al., 2017) removed anchors and proposal stages. Following DenseBox, a fully convolutional network with a U-shaped feature-merging branch (the idea of U-Net) outputs, at every pixel of a quarter-resolution map, a text score and the geometry of the instance that contains the pixel, which makes it an anchor-free detector in DBNet’s classification (Liao et al., 2020). Two geometries are offered:

• RBOX: four distances from the pixel to the top, right, bottom and left sides of a rotated rectangle, plus its angle $\theta$ (five channels);
• QUAD: the offsets from the pixel to the four corners of a quadrilateral (eight channels).

Only pixels inside a shrunk version of each quadrilateral are positive: each edge is shortened by moving its two endpoints inward along it by 0.3 times each vertex’s reference length, the length of the shorter edge at that vertex, the longer pair of opposite edges first. The score map is trained with class-balanced cross-entropy, with the weight $\beta = 1 - \sum y^*/|Y^*|$ for the positive class, instead of sampling or hard negative mining. For RBOX the box part uses an IoU loss and the angle a cosine loss,

$$L_g = -\log \mathrm{IoU}(\hat{R}, R^*) + \lambda_\theta \left(1 - \cos(\hat{\theta} - \theta^*)\right), \qquad \lambda_\theta = 10,$$

because an IoU loss does not favour large instances the way an L1 or L2 loss would; QUAD uses a smooth L1 loss normalized by the shortest edge and minimized over the vertex orderings, since public annotations order vertices inconsistently. Tens of thousands of per-pixel predictions survive thresholding, so EAST merges them row by row with a locality-aware NMS that averages neighbouring geometries weighted by score before a standard NMS.

With a PVANet2x backbone and RBOX output, EAST reported recall 0.7347, precision 0.8357 and F-score 0.7820 on the ICDAR 2015 test set at a single scale, 0.8072 with five test scales, and 13.2 frames per second at 720p on a Titan X; the best earlier method in its table reached 0.6477 (Table 3, Table 6). On MSRA-TD500 it reported 0.7608 and on ICDAR 2013 0.8737. The authors list its limitations: long text lines that exceed the receptive field, vertical text, which is rare in the ICDAR 2015 training set, and curved text, which they name as future work.

Segmentation: PixelLink, TextSnake and PSENet

Segmentation-based detectors predict, for every pixel, whether it belongs to text, with a semantic segmentation network, and turn the text pixels into instances afterwards, a form of instance segmentation. The output region follows the pixels, so curved text needs no special geometry. The difficulty moves to separating instances that touch.

PixelLink (Deng et al., 2018) predicts two things with a VGG16-based FCN: a text/non-text score for every pixel and, for each of its eight neighbours, whether the link to that neighbour joins pixels of the same instance (two plus sixteen output channels). Positive pixels joined by positive links form connected components, found with a disjoint-set structure; a box is then fitted to each component with OpenCV’s minimum-area rectangle, and components too small for text are dropped using the 99th percentile of the training data (on ICDAR 2015, boxes with a shorter side under 10 pixels or an area under 300). Training uses an instance-balanced cross-entropy that gives every instance the same total weight, so small words count as much as large ones, with online hard example mining at three negatives per positive. PixelLink was trained from scratch on the 1,000 ICDAR 2015 training images alone and reached recall 82.0, precision 85.5 and F-score 83.7 at 3.0 frames per second (Table 1), above the single-scale EAST results it quotes. Its authors attribute the data efficiency to the task: a prediction only needs to see a pixel and its neighbours, not a whole instance.

TextSnake (Long et al., 2018) represents an instance as a sequence of overlapping disks centred on its centre line, each with a radius and an orientation, so the region is the union of the disks and can bend freely. The network predicts the text region, the text centre line and the disk geometry at every pixel; centre-line pixels inside text regions are split into instances with a disjoint set, and a striding procedure walks along each centre line to reconstruct the polygon. It reported F-measures of 78.4 on Total-Text and 75.6 on CTW1500, against 42.0 and 60.4 for EAST, which its authors evaluated without fine-tuning on Total-Text (Table 1), and 82.6 on ICDAR 2015 at 1.1 frames per second.

PSENet (Wang et al., 2019), the progressive scale expansion network, predicts $n$ segmentation maps of each instance at increasing scales, from a small kernel to the full region. The kernels of adjacent words do not touch even when the words do, so connected components of the smallest map separate the instances, and a breadth-first search then grows each kernel through the successively larger maps, assigning a contested pixel to whichever kernel reaches it first. The training labels shrink each ground-truth polygon $p_n$ with the Vatti clipping algorithm by

$$d_i = \frac{\mathrm{Area}(p_n)\,(1 - r_i^2)}{\mathrm{Perimeter}(p_n)}, \qquad r_i = 1 - \frac{(1 - m)(n - i)}{n - 1},$$

so the scale ratios rise linearly from the minimum $m$ to 1; the loss is based on the Dice coefficient, with hard example mining on the full map. On the backbone side PSENet uses a ResNet with a feature pyramid network. With external data (pre-training on ICDAR 2017 MLT) and full-resolution output, it reported F-measures of 82.2 on CTW1500, 80.87 on Total-Text and 85.69 on ICDAR 2015 (Tables 2–4); the ICDAR 2015 result ran at 1.6 frames per second with the long side scaled to 2,240 pixels.

CRAFT (Baek et al., 2019) segments at the level of characters instead. It predicts a region score, a Gaussian heatmap at each character’s centre, and an affinity score between adjacent characters, and groups characters into words from both. Real datasets give only word boxes, so CRAFT trains in a weakly supervised way: an interim model estimates character boxes inside each annotated word, and these pseudo-labels are weighted by how well the number of estimated characters matches the length of the word’s transcription. Trained only on ICDAR datasets, it reported H-means of 86.9 on ICDAR 2015, 83.6 on Total-Text and 83.5 on CTW1500, the latter two without fine-tuning on those datasets except for a small link-refinement network on CTW1500.

Differentiable binarization: DBNet and DBNet++

Segmentation detectors had become accurate but slow, because of their post-processing: PSENet’s expansion and pixel-embedding clustering each cost considerable time at inference (Liao et al., 2020). DBNet (Liao et al., 2020; called DB in the paper) puts the binarization step inside the network. A feature-pyramid backbone predicts a probability map $P$ and a threshold map $T$, and an approximate binary map

$$\hat{B}_{i,j} = \frac{1}{1 + e^{-k (P_{i,j} - T_{i,j})}}, \qquad k = 50,$$

replaces the fixed-threshold step function, so the threshold at every pixel is learned with the segmentation. The authors explain the gain through the gradients: the factor $k$ amplifies them, and differently for positive and negative labels, which helps the network produce more distinctive predictions. The probability map is supervised with polygons shrunk as in PSENet, with offset $D = A(1 - r^2)/L$ for area $A$, perimeter $L$ and $r = 0.4$; the threshold map’s target is computed from the distance to the nearest polygon edge in the band between the shrunk polygon and the polygon dilated by the same offset, and its loss covers the pixels inside the dilated polygon. The loss is $L_s + \alpha L_b + \beta L_t$ with $\alpha = 1$ and $\beta = 10$: binary cross-entropy with hard negative mining (1:3) for the probability and binary maps and an L1 loss for the threshold map.

At inference the threshold branch can be dropped: the probability map is binarized at a constant 0.2, its connected components (the shrunk regions) are found, and each is dilated back by $D' = A' r'/L'$ with $r' = 1.5$, where $A'$ and $L'$ are the area and perimeter of the shrunk region. This step is cheap, and DBNet runs at real-time speed. With a ResNet-50 backbone and deformable convolutions, the authors reported F-measures of 87.3 on ICDAR 2015 (input height 1,152, 12 frames per second), 84.9 on MSRA-TD500 (32 frames per second), 84.7 on Total-Text and 83.4 on CTW1500; a ResNet-18 version reached 82.8 on MSRA-TD500 at 62 frames per second on a GTX 1080 Ti (Tables 3–6). In their ablation the binarization module alone added 3.7 and 4.9 points of F-measure with ResNet-18 on MSRA-TD500 and CTW1500. They name one limitation shared by segmentation detectors: text inside text, when one instance lies in the centre of another, cannot be separated by shrinking.

DBNet++ (Liao, Zou, Wan, Yao and Bai, 2023, IEEE Transactions on Pattern Analysis and Machine Intelligence) adds an adaptive scale fusion module. The paper could not be read for this article; in the official code, the module concatenates the upsampled pyramid features, predicts one spatial attention map per scale from them and multiplies each scale’s features by its map before concatenation, so the network can weight scales differently at each location (MhLiao/DB, decoders/feature_attention.py). The repository lists DB in OpenCV and PaddleOCR (as of November 2026).

From boxes to polygons

The change from rectangles to polygons was driven by the benchmarks. Total-Text and CTW1500 required curved regions, and regression detectors either regressed a fixed number of polygon points (Total-Text’s baseline, Poly-FRCNN, regresses polygon parameters instead of box parameters with Faster R-CNN; Ch’ng et al., 2020) or adopted region representations. DBNet’s authors summarize the trade-off: regression-based methods enjoy simple post-processing such as NMS but struggle to represent irregular shapes, while segmentation-based methods describe any shape but need post-processing to form instances (Liao et al., 2020). Mask TextSpotter detects arbitrary-shape text as instance segmentation built on Mask R-CNN (as described by Liao et al., 2020).

Datasets and Benchmarks

Benchmark	Images (train / test)	Unit	Annotation	Script	Matching rule
ICDAR 2013 (Focused Scene Text)	229 / 233	word	axis-aligned rectangle	English	DetEval, area recall 0.8 and precision 0.4, with splits and merges
ICDAR 2015 (Incidental Scene Text)	1,000 / 500	word	quadrilateral	Latin (others “do not care”)	IoU > 0.5, one-to-one
MSRA-TD500	300 / 200	line	quadrilateral fitted to a minimum-area rectangle	Chinese and English	angle < π/8 and rotated-to-axis IoU > 0.5
Total-Text	1,255 / 300	word	polygon	Latin (others “do not care”)	DetEval (0.4, 0.8 default; 0.6, 0.7 recommended) or IoU 0.5
CTW1500	1,000 / 500	line	14-point polygon	English and Chinese	polygon IoU > 0.5, ranked by confidence

ICDAR 2013 and 2015. The two ICDAR Robust Reading datasets are the standard tests for horizontal and for multi-oriented words. ICDAR 2013 scores localization with Wolf and Jolion’s DetEval framework, which credits a detection covering 80% of a word with 40% of its own area on the word, gives reduced credit to splits and does not penalize merges; ICDAR 2015 requires an IoU above 0.5, so line-level and split detections are penalized. Many papers report both an “IC13” and a “DetEval” column for ICDAR 2013, and the two can differ by several points. The ICDAR article gives both protocols, the evaluation scripts and the end-to-end tasks in detail.

MSRA-TD500. Released with Yao et al. (2012), it has 500 images of indoor scenes (signs, doorplates, caution plates) and outdoor scenes (guide boards, billboards) taken with a pocket camera, split randomly into 300 training and 200 test images. Annotators drew a four-vertex polygon around each text line, and the ground truth is the minimum-area rectangle fitted to it. Text that is too small, occluded, blurred or truncated carries a “difficult” label, and missing it is not penalized (IAPR TC11 dataset page). The protocol was designed for rotated rectangles at a time when ICDAR’s rules assumed horizontal text. A detection matches a ground-truth line if the included angle between the two rectangles is less than $\pi/8$ and their overlap exceeds 0.5, where the overlap is the IoU of the two rectangles after each has been rotated to axis-aligned about its own centre:

$$m(G, D) = \frac{A(G' \cap D')}{A(G' \cup D')}.$$

Multiple detections of the same line count as false positives, and precision and recall are $|TP|/|E|$ and $|TP|/|T|$ over the detected and ground-truth rectangles (Yao et al., 2012). In our reading, the rotate-to-axis step makes the overlap blind to the angle, which is why the separate angle test is needed. Because 300 images are too few for deep detectors, papers from EAST onward add the 400 images of HUST-TR400 to the training data (Zhou et al., 2017; Deng et al., 2018; Long et al., 2018; Liao et al., 2020), and CRAFT merges its word boxes into lines with a post-processing step to match the line-level ground truth (Baek et al., 2019).

Total-Text. Ch’ng and Chan released Total-Text in 2017 because curved text was missing from earlier datasets; the extended version (Ch’ng et al., 2020) describes 1,555 images with 11,459 annotated words, about half of them curved and the rest split almost equally between horizontal and multi-oriented, split into 1,255 training and 300 test images. Words are annotated with polygons and transcriptions; text in non-Latin scripts, watermarks and unreadable text are marked “do not care” (#). The extended version regulated the training polygons to exactly 10 vertices, placed with guiding lines to reduce annotator bias, but the test ground truth was not re-annotated, since a new version would make earlier results incomparable (repository README).

Total-Text was first scored with DetEval at its default thresholds, area precision $t_p = 0.4$ and area recall $t_r = 0.8$. Its authors found both unsuited to polygons (Ch’ng et al., 2020): $t_p = 0.4$ let a loose axis-aligned box count as fully as a tight polygon, while $t_r = 0.8$ rejected tight polygon detections that missed a sliver of a many-vertex ground truth, which even repeated human annotations fail to match. They recommend $t_p = 0.6$ and $t_r = 0.7$. Their Table 1 shows the effect: their loose box detector ranked third of six methods under the default thresholds (F 0.70) and last under the recommended ones (F 0.48), while their polygon detector rose from 0.63 to 0.73. A PASCAL-style rule at IoU 0.5 is provided as well. The official Python scripts drop detections that lie more than half inside a “do not care” region (intersection over the detection’s own area), remove the “do not care” words from the counts, and pool matches over the test set; the DetEval script uses strict inequalities and rounds the area ratios to two decimals, and the PASCAL script accepts IoU of exactly 0.5 (Deteval.py, Pascal_VOC.py).

As of November 2026 the repository applies research-use terms, effective 1 October 2026, to the annotations: non-commercial academic use, written confirmation from the licensor for projects carried out with or sponsored by companies, and no redistribution; the scripts are under the BSD-3-Clause licence, and the underlying images are excluded from both (DATASET_TERMS.md, RIGHTS_NOTICE.md).

CTW1500. SCUT-CTW1500, from Liu, Jin and colleagues (Pattern Recognition, 2019; the paper was not read for this article), has 1,000 training and 500 test images, each with at least one curved text instance, annotated at the level of text lines in English and Chinese (repository README). PSENet’s and TextSnake’s authors describe the polygons as 14-point polygons, and the official evaluation script reads 14 points per ground-truth instance; DBNet’s paper states 16 vertices for CTW1500 (Liao et al., 2020), which disagrees with both. The original evaluation script follows PASCAL VOC: detections are taken in order of confidence, each is compared with the ground-truth line of highest polygon IoU and counts as a true positive if that IoU exceeds 0.5 and the line is not yet matched (a false positive otherwise), and the script reports average precision as well as recall, precision and F-measure (voc_eval_polygon.py). An updated annotation marks all Chinese text as “do not care” because there is too little of it for training and testing, and the README states that the database is free to the academic community for research only (as of November 2026).

Others. COCO-Text adds text annotations to COCO images, and the larger multi-lingual (MLT), arbitrary-shaped (ArT) and Chinese street-view (LSVT) collections are described in the ICDAR article. Total-Text and CTW1500 are both part of ArT’s training set; their repositories ask anyone training on ArT to remove the Total-Text and CTW1500 test images first, so that results on the two benchmarks stay valid.

Evaluation Metrics

Detection is scored with precision and recall and the F-score, counted over all test images together, so images with many instances weigh more. What differs between benchmarks, and sometimes between papers on the same benchmark, is the rule that decides a match:

• Overlap measure. IoU on axis-aligned rectangles, on quadrilaterals or on polygons (exact polygon intersection in the ICDAR 2015, Total-Text and CTW1500 scripts), the rotated-to-axis overlap of MSRA-TD500, or DetEval’s pair of area recall and area precision.
• Thresholds. IoU above 0.5 (strict in the ICDAR 2015 script, inclusive in Total-Text’s PASCAL script); DetEval at (0.4, 0.8) or (0.6, 0.7).
• Assignment. Greedy in file order (ICDAR 2015), greedy by confidence (CTW1500), or DetEval’s one-to-one, split and merge stages.
• Do-not-care handling. Which detections are discarded before matching, and by which overlap.

Papers do not always use the official script. PSENet’s authors evaluated Total-Text with the same method as CTW1500 (Wang et al., 2019), and DBNet’s repository notes that its built-in evaluation is the ICDAR 2015 protocol modified for polygons, which “almost produces the same results” as Total-Text’s PASCAL protocol (MhLiao/DB README). Total-Text’s repository leaderboard lists results “reported on paper” separately from results recomputed with DetEval at the default and recommended thresholds. Numbers are comparable only under the same protocol, the same test resolution and the same external training data.

The results below are as reported by their authors, single-scale unless marked, and are not all directly comparable: backbones, input sizes, external data and GPUs differ.

Method (year)	ICDAR 2015 F	MSRA-TD500 F	Total-Text F	CTW1500 F	Source
EAST, PVANet2x (2017)	78.2 (80.7 multi-scale)	76.1	—	—	Zhou et al., Tables 3, 5
PixelLink, VGG16 2s (2018)	83.7	77.8	—	—	Deng et al., Tables 1, 2
TextSnake (2018)	82.6	78.3	78.4	75.6	Long et al., Tables 1, 2
PSENet-1s, ResNet-50, external data (2019)	85.69	—	80.87	82.2	Wang et al., Tables 2–4
CRAFT (2019)	86.9	82.9	83.6	83.5	Baek et al., Tables 1, 2
DBNet, ResNet-50 (2020)	87.3 (height 1,152)	84.9	84.7	83.4	Liao et al., Tables 3–6

Example: detecting and scoring words in a synthetic scene

The example renders a synthetic street scene with known ground truth and runs three experiments on it. The scene has four words drawn with glyph outlines: CAFE and BAR stacked so that their ground-truth rectangles touch, as lines of a sign often do, EXIT rotated by 30°, and CURVED written along an arc with an annular band as its ground-truth polygon, plus two distractors, a fence of six bars of constant width and a patch of speckle. Part A runs a minimal connected-component detector in the spirit of the classical methods: Otsu thresholding, components whose stroke width, measured with a distance transform along each component’s skeleton, varies by less than half its mean (a simplified stand-in for SWT’s variance test, not the SWT ray algorithm), and grouping of nearby components by morphological dilation. Part B scores three output representations of each detected region, the axis-aligned box, the minimum-area rotated rectangle and the region itself, under the matching rules of ICDAR 2015, MSRA-TD500 and Total-Text, using pixel areas. Because the choice of rule decides the headline result, every rule is printed. The DetEval rows apply only the one-to-one pair test, without DetEval’s split and merge stages. Part C decodes an exact “probability map” without any network: once as the union of the full ground-truth regions, the target of a segmentation network trained on full regions, and once as DBNet’s shrunk kernels ($r = 0.4$) expanded back with $D' = 1.5A'/L'$. Shrinking and expansion use Euclidean distance transforms, an approximation to the Vatti polygon clipping that DBNet uses.

import numpy as np
from matplotlib.textpath import TextPath
from matplotlib.font_manager import FontProperties
from scipy import ndimage as ndi
from scipy.spatial import ConvexHull
from skimage.draw import polygon as fill_poly
from skimage.filters import threshold_otsu
from skimage.measure import label, perimeter, regionprops
from skimage.morphology import skeletonize, disk, binary_dilation

H, W = 240, 520
FONT = FontProperties(family="DejaVu Sans", weight="bold")
rng = np.random.default_rng(0)

def fill(xy):
    """Rasterize a closed polygon given as (x, y) vertices; pixel centres inside."""
    m = np.zeros((H, W), bool)
    rr, cc = fill_poly(xy[:, 1], xy[:, 0], (H, W))
    m[rr, cc] = True
    return m

def glyph_mask(text, warp, size=40):
    """Render text with glyph outlines; warp maps local (x, y-up) points to image (x, y)."""
    m = np.zeros((H, W), bool)
    for poly in TextPath((0, 0), text, size=size, prop=FONT).to_polygons():
        m ^= fill(warp(np.asarray(poly)))          # even-odd fill keeps the holes of A, R, ...
    return m

def straight(text, cx, cy, deg, margin=3):
    """A straight word centred at (cx, cy), rotated by deg; returns glyph mask and GT rectangle."""
    v = TextPath((0, 0), text, size=40, prop=FONT).vertices
    x0, y0, x1, y1 = v[:, 0].min(), v[:, 1].min(), v[:, 0].max(), v[:, 1].max()
    t = np.deg2rad(deg); R = np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
    def warp(p):
        q = (p - [(x0 + x1) / 2, (y0 + y1) / 2]) * [1, -1]       # centre, flip y to image rows
        return q @ R.T + [cx, cy]
    box = np.array([[x0 - margin, y1 + margin], [x1 + margin, y1 + margin],
                    [x1 + margin, y0 - margin], [x0 - margin, y0 - margin]])
    return glyph_mask(text, warp), fill(warp(box))

def curved(text, cx, cy, radius, start_deg, margin=3):
    """A word written clockwise over the top of a circle; GT is the annular band around it."""
    widths = [TextPath((0, 0), ch, size=40, prop=FONT).get_extents().width + 4 for ch in text]
    g, th = np.zeros((H, W), bool), np.deg2rad(start_deg)
    top = TextPath((0, 0), text, size=40, prop=FONT).vertices[:, 1].max()
    th_first = th
    for ch, w in zip(text, widths):
        ext = TextPath((0, 0), ch, size=40, prop=FONT).get_extents()
        thc = th - (w / 2) / radius
        def warp(p, thc=thc, xc=ext.x0 + ext.width / 2):
            ang, rho = thc - (p[:, 0] - xc) / radius, radius + p[:, 1]
            return np.c_[cx + rho * np.cos(ang), cy - rho * np.sin(ang)]
        g |= glyph_mask(ch, warp)
        th -= w / radius
    a = np.linspace(th_first + margin / radius, th - margin / radius, 40)
    r_in, r_out = radius - margin, radius + top + margin
    band = np.r_[np.c_[cx + r_out * np.cos(a), cy - r_out * np.sin(a)],
                 np.c_[cx + r_in * np.cos(a[::-1]), cy - r_in * np.sin(a[::-1])]]
    return g, fill(band)

# ---------------------------------------------------------------- the synthetic scene
words = {"CAFE": straight("CAFE", 95, 52, 0), "BAR": straight("BAR", 80, 95, 0),
         "EXIT": straight("EXIT", 300, 195, 30), "CURVED": curved("CURVED", 300, 200, 110, 132)}
# shift BAR so that its GT rectangle touches CAFE's (shared edge, as in stacked text lines)
gap = np.flatnonzero(words["BAR"][1].any(1))[0] - np.flatnonzero(words["CAFE"][1].any(1))[-1] - 1
words["BAR"] = tuple(np.roll(m, -gap, axis=0) for m in words["BAR"])
names = list(words)
G = np.array([words[n][1] for n in names])                 # ground-truth region masks
ink = np.any([words[n][0] for n in names], axis=0)
fence = np.zeros((H, W), bool)
for k in range(6):
    fence[150:220, 30 + 16 * k: 36 + 16 * k] = True        # 6 bars, 6 px wide: constant "stroke width"
blob = np.zeros((H, W), bool)
blob[20:70, 440:500] = rng.random((50, 60)) < 0.35          # foliage-like speckle
img = 200 + 12 * rng.standard_normal((H, W))
img[ink | fence | blob] = 50 + 12 * rng.standard_normal((ink | fence | blob).sum())
print("GT words:", {n: int(G[i].sum()) for i, n in enumerate(names)}, "(region areas, px)")

# ---------------------------------------------------------------- Part A: classical detector
binary = img < threshold_otsu(img)
lab = label(binary, connectivity=2)
keep = np.zeros((H, W), bool); heights = []
for r in regionprops(lab):
    h = r.bbox[2] - r.bbox[0]
    if not (8 <= h <= 80 and r.area >= 20):
        continue
    comp = lab == r.label
    sw = ndi.distance_transform_edt(comp)[skeletonize(comp)]       # half stroke widths on the skeleton
    if sw.std() / sw.mean() < 0.5:                                  # stroke-width consistency test
        keep |= comp; heights.append(h)
radius = int(round(0.6 * np.median(heights)))
groups = label(binary_dilation(keep, disk(radius)))
regions = [binary_dilation(keep & (groups == g), disk(2)) for g in range(1, groups.max() + 1)]
print(f"\nPart A: {len(heights)} components kept, grouping radius {radius} px, {len(regions)} regions")
for j, D in enumerate(regions):
    ov = [(names[i], round(float((G[i] & D).sum() / (G[i] | D).sum()), 2)) for i in range(len(G)) if (G[i] & D).any()]
    ys, xs = np.nonzero(D)
    print(f"  R{j}: x {xs.min()}-{xs.max()}, y {ys.min()}-{ys.max()}, overlaps {ov if ov else 'no word'}")

# ---------------------------------------------------------------- Part B: representations and rules
def min_area_rect(mask):
    """Minimum-area rectangle (corners, angle of long side) around a mask's pixel squares."""
    ys, xs = np.nonzero(mask)
    pts = np.c_[xs, ys].astype(float)
    pts = (pts[:, None, :] + np.array([[-.5, -.5], [.5, -.5], [.5, .5], [-.5, .5]])).reshape(-1, 2)
    pts = pts[ConvexHull(pts).vertices]                  # the extreme points decide the rectangle
    best = None
    for a in np.deg2rad(np.arange(0, 90, 0.25)):
        R = np.array([[np.cos(a), np.sin(a)], [-np.sin(a), np.cos(a)]])
        q = pts @ R.T
        lo, hi = q.min(0), q.max(0)
        if best is None or np.prod(hi - lo) < best[0]:
            best = (np.prod(hi - lo), a, lo, hi, R)
    _, a, lo, hi, R = best
    corners = np.array([[lo[0], lo[1]], [hi[0], lo[1]], [hi[0], hi[1]], [lo[0], hi[1]]]) @ R
    w, h = hi - lo
    return corners, (a if w >= h else a + np.pi / 2) % np.pi, corners.mean(0), max(w, h), min(w, h)

def aabb(mask):
    ys, xs = np.nonzero(mask); m = np.zeros_like(mask)
    m[ys.min():ys.max() + 1, xs.min():xs.max() + 1] = True
    return m

reps = {"AABB": [aabb(D) for D in regions],
        "RBOX": [fill(min_area_rect(D)[0]) for D in regions],
        "polygon": regions}

def overlaps(Ds):
    inter = np.array([[(g & d).sum() for d in Ds] for g in G], float)
    aG, aD = G.sum((1, 2))[:, None], np.array([d.sum() for d in Ds], float)[None, :]
    return inter / (aG + aD - inter), inter / aG, inter / aD           # IoU, sigma, tau

def td500_ok(gmask, dmask):
    """MSRA-TD500: angle within pi/8 and IoU > 0.5 after rotating both rectangles to axis-aligned."""
    _, ag, cg, wg, hg = min_area_rect(gmask); _, ad, cd, wd, hd = min_area_rect(dmask)
    dang = abs(ag - ad); dang = min(dang, np.pi - dang)
    ix = max(0, min(cg[0] + wg / 2, cd[0] + wd / 2) - max(cg[0] - wg / 2, cd[0] - wd / 2))
    iy = max(0, min(cg[1] + hg / 2, cd[1] + hd / 2) - max(cg[1] - hg / 2, cd[1] - hd / 2))
    return dang < np.pi / 8 and ix * iy / (wg * hg + wd * hd - ix * iy) > 0.5

def match(ok):
    """Greedy one-to-one matching in ground-truth order, as in the ICDAR 2015 script."""
    used, n = set(), 0
    for g in range(ok.shape[0]):
        for d in range(ok.shape[1]):
            if ok[g, d] and d not in used:
                used.add(d); n += 1; break
    r, p = n / ok.shape[0], n / ok.shape[1]
    return n, r, p, 0 if n == 0 else 2 * r * p / (r + p)

print("\nPart B: best detection per word (IoU / sigma / tau)")
for rep, Ds in reps.items():
    iou, sg, ta = overlaps(Ds)
    cells = [f"{n} {iou[i].max():.2f}/{sg[i, iou[i].argmax()]:.2f}/{ta[i, iou[i].argmax()]:.2f}" for i, n in enumerate(names)]
    print(f"  {rep:8s}" + "  ".join(cells))
print("\n  rule                      rep       matched  R     P     F")
for rep, Ds in reps.items():
    iou, sg, ta = overlaps(Ds)
    rules = {"ICDAR 2015 IoU > 0.5": iou > 0.5,
             "DetEval 1:1 (0.8, 0.4)": (sg >= 0.8) & (ta >= 0.4),
             "DetEval 1:1 (0.7, 0.6)": (sg > 0.7) & (ta > 0.6)}
    if rep != "polygon":
        rules["TD500 angle + IoU"] = np.array([[td500_ok(g, d) for d in Ds] for g in G])
    for rule, ok in rules.items():
        n, r, p, f = match(ok)
        print(f"  {rule:25s} {rep:8s}  {n}/{len(G)}    {r:.2f}  {p:.2f}  {f:.2f}")

# ---------------------------------------------------------------- Part C: segmentation targets
def decode(prob, expand):
    lab = label(prob, connectivity=1); out = []
    for k in range(1, lab.max() + 1):
        kern = lab == k
        if expand:                                   # DB box formation: D' = A' r' / L', r' = 1.5
            d = kern.sum() * 1.5 / perimeter(kern)
            kern = ndi.distance_transform_edt(~kern) <= d
        out.append(kern)
    return out

shrunk = np.zeros((H, W), bool)
for g in G:                                          # DB label: D = A (1 - r^2) / L, r = 0.4
    shrunk |= ndi.distance_transform_edt(g) > g.sum() * (1 - 0.4 ** 2) / perimeter(g)
print("\nPart C: decoding an exact probability map (ICDAR 2015 rule on masks)")
for title, prob, expand in [("full regions, CC", G.any(0), False),
                            ("shrunk kernels, CC + expand", shrunk, True)]:
    Ds = decode(prob, expand)
    iou = overlaps(Ds)[0]
    n, r, p, f = match(iou > 0.5)
    print(f"  {title:28s} {len(Ds)} regions; best IoU " +
          ", ".join(f"{nm} {iou[i].max():.2f}" for i, nm in enumerate(names)) + f"; R {r:.2f} P {p:.2f} F {f:.2f}")

Output (NumPy 2.3.5, SciPy 1.16.3, scikit-image 0.25.2, Matplotlib 3.10.6; under a second on a laptop CPU):

GT words: {'CAFE': 4403, 'BAR': 3395, 'EXIT': 3733, 'CURVED': 7491} (region areas, px)

Part A: 30 components kept, grouping radius 20 px, 5 regions
  R0: x 438-501, y 18-71, overlaps no word
  R1: x 36-153, y 35-104, overlaps [('CAFE', 0.4), ('BAR', 0.38)]
  R2: x 213-402, y 60-125, overlaps [('CURVED', 0.63)]
  R3: x 28-117, y 148-221, overlaps no word
  R4: x 251-352, y 158-229, overlaps [('EXIT', 0.6)]

Part B: best detection per word (IoU / sigma / tau)
  AABB    CAFE 0.50/0.96/0.51  BAR 0.37/0.93/0.38  EXIT 0.48/0.97/0.49  CURVED 0.55/0.95/0.56
  RBOX    CAFE 0.50/0.96/0.51  BAR 0.37/0.93/0.38  EXIT 0.92/0.92/1.00  CURVED 0.58/0.94/0.61
  polygon CAFE 0.40/0.60/0.54  BAR 0.38/0.67/0.46  EXIT 0.60/0.60/1.00  CURVED 0.63/0.63/1.00

  rule                      rep       matched  R     P     F
  ICDAR 2015 IoU > 0.5      AABB      2/4    0.50  0.40  0.44
  DetEval 1:1 (0.8, 0.4)    AABB      3/4    0.75  0.60  0.67
  DetEval 1:1 (0.7, 0.6)    AABB      0/4    0.00  0.00  0.00
  TD500 angle + IoU         AABB      2/4    0.50  0.40  0.44
  ICDAR 2015 IoU > 0.5      RBOX      3/4    0.75  0.60  0.67
  DetEval 1:1 (0.8, 0.4)    RBOX      3/4    0.75  0.60  0.67
  DetEval 1:1 (0.7, 0.6)    RBOX      2/4    0.50  0.40  0.44
  TD500 angle + IoU         RBOX      3/4    0.75  0.60  0.67
  ICDAR 2015 IoU > 0.5      polygon   2/4    0.50  0.40  0.44
  DetEval 1:1 (0.8, 0.4)    polygon   0/4    0.00  0.00  0.00
  DetEval 1:1 (0.7, 0.6)    polygon   0/4    0.00  0.00  0.00

Part C: decoding an exact probability map (ICDAR 2015 rule on masks)
  full regions, CC             3 regions; best IoU CAFE 0.56, BAR 0.44, EXIT 1.00, CURVED 1.00; R 0.75 P 1.00 F 0.86
  shrunk kernels, CC + expand  4 regions; best IoU CAFE 0.72, BAR 0.90, EXIT 0.85, CURVED 0.69; R 1.00 P 1.00 F 1.00

The connected-component detector shows both classical failure modes at once. The stacked words CAFE and BAR fall into one region because their letters are closer vertically than the grouping distance, and the fence passes the stroke-width test, since parallel bars have a perfectly constant stroke width; the speckle patch also survives as a region. Both false positives are the kind of text-like clutter that MSRA-TD500’s authors report. In our reading, a grouping distance has to be shorter than the gap between lines and longer than the gaps between letters, and in stacked signs the two can be similar.

Part B shows how much the geometry of the output, not the detector, decides the score. For EXIT at 30°, the axis-aligned box covers 97% of the word but has an IoU of 0.48, so it is missed under the ICDAR 2015 rule, while the rotated rectangle reaches 0.92. The same loose box passes the DetEval pair test at the default thresholds, because it lies 49% on the word, above $t_p = 0.4$, and fails at Total-Text’s $t_p = 0.6$, which is the behaviour Total-Text’s authors designed the new thresholds to produce. The curved word is only gently curved here: its axis-aligned box and rotated rectangle still pass IoU 0.5 (0.55 and 0.58), against our hypothesis that both would fail, and only Total-Text’s thresholds separate them, accepting the rotated rectangle (τ 0.61) and rejecting the box (τ 0.56). The “polygon” representation, the detected ink dilated by 2 pixels, fails nearly everywhere although it lies entirely on the words (τ = 1 for EXIT and CURVED): ground-truth regions include the space between and inside letters, so a mask of strokes covers only 60 to 67% of a word. A detector’s region must describe the text area, not the ink, which is what segmentation detectors are trained to predict. The box around the merged CAFE-and-BAR region still matches CAFE under the ICDAR 2015 rule (its IoU rounds to 0.50 and passes the strict test), because CAFE is the larger word, and BAR is lost.

Part C isolates the instance-separation problem of segmentation detectors. Even a perfect full-region map merges the touching words into one component: CAFE survives at IoU 0.56 and BAR is missed. Shrunk kernels separate them, and every word is recovered. The recovered regions are smaller than the ground truth (IoU 0.69–0.90): with exact kernels, the expansion offset $D' = 1.5A'/L'$ is smaller than the offset $D$ removed in shrinking, for CAFE about 8.8 pixels against 11.9 in our calculation for its 118 × 37 pixel rectangle, and by our calculation the shortfall grows with a word’s length relative to its height, which matches the order of the IoUs (CURVED and CAFE lowest, BAR highest). DBNet’s $r' = 1.5$ was set empirically for the maps its network predicts, which need not match exact kernels. All these numbers come from one hand-built scene and show which configurations each rule rewards, not how real detectors perform.

The script was changed twice after its first run. In the first run EXIT was centred at (420, 175), where its ground-truth region touched CURVED’s and its letters lay within Part A’s grouping distance of CURVED’s, so Part A grouped the two words and Part C’s full-region map merged them, and the first run’s Part A and Part B scores and Part C full-region scores for EXIT and CURVED were those of one merged region. Only CAFE and BAR were meant to touch, so EXIT was moved below the arc, to (300, 195), after those results had been seen, with no other setting changed. The first run also took 18 seconds, against a planned estimate of “Under 5 seconds”, because the minimum-area rectangle was searched over all pixel corners; the search now uses the corners’ convex hull, which gave identical output on the first-run scene.

Applications

Scene text detection is the first stage of systems that read text in images: instant translation, aids for visually impaired people, image retrieval, geo-location and the automatic geo-coding of businesses, and robot navigation in urban environments, the applications named by Epshtein et al. (2010) and Baek et al. (2019). TextBoxes’ authors mention road signs, licence plates and product packages (Liao et al., 2017). In end-to-end systems the detector supplies regions to a recognizer, described in optical character recognition, and recognition can in turn filter detections: TextBoxes rescored candidate boxes with the CRNN recognizer and a lexicon, which raised word-spotting accuracy (Liao et al., 2017). Detectors are also used on video and on document photographs taken with phones, where the page assumptions of document OCR do not hold.

Open Problems

• Text inside text and dense layouts. Shrinking separates touching words but not a word lying at the centre of another, a limitation DBNet’s authors state for segmentation detectors in general.
• Protocol fragmentation. The same benchmark is scored with different scripts, thresholds and overlap measures, and some leaderboard numbers were computed with a protocol other than the official one. Total-Text’s revision of DetEval shows that thresholds designed for rectangles do not transfer to polygons.
• Granularity mismatch. Word-level and line-level benchmarks reward different outputs, and detectors are adapted to each with post-processing (CRAFT’s line merging for MSRA-TD500).
• Small test sets and reuse. The test sets have 200 to 500 images and have been reused for years; the general risks are covered in benchmark overfitting. The overlap of Total-Text and CTW1500 test images with ArT’s training set relies on users removing them.
• Dependence on synthetic pre-training. TextBoxes, SegLink, TextSnake, CRAFT and DBNet are pre-trained on SynthText’s 800,000 synthetic images, made from 8,000 background images (Liao et al., 2020), before fine-tuning on a few hundred or thousand real images. EAST and PSENet instead start from ImageNet-pretrained backbones and real data (PSENet states that it uses no SynthText), and PixelLink was trained from scratch on ICDAR 2015 alone.
• Scripts and languages. Most benchmarks score Latin text and mark other scripts “do not care”; multi-lingual benchmarks remain harder, with F-scores of 70.9 to 74.7 on ICDAR 2017 MLT for PSENet and DBNet in DBNet’s Table 7.