Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach

TLDR

Text segmentation is a prerequisite for many real‑world text tasks such as style transfer and scene text removal, yet it has been largely overlooked due to a lack of high‑quality datasets and dedicated studies. The study introduces TextSeg, a large‑scale fine‑annotated text dataset, and proposes TexRNet, a novel text‑specific segmentation approach, to address the lack of resources and tailored methods. TexRNet incorporates text‑specific designs—key‑feature pooling, attention‑based similarity checking, trimap and discriminator losses—to handle non‑convex boundaries and diverse textures, and is evaluated extensively on TextSeg and other datasets. TexRNet achieves nearly a 2 % performance gain over state‑of‑the‑art segmentation methods on both the new TextSeg dataset and existing benchmarks.

Abstract

Text segmentation is a prerequisite in many real-world text-related tasks, e.g., text style transfer, and scene text removal. However, facing the lack of high-quality datasets and dedicated investigations, this critical prerequisite has been left as an assumption in many works, and has been largely overlooked by current research. To bridge this gap, we proposed TextSeg, a large-scale fine-annotated text dataset with six types of annotations: word- and character-wise bounding polygons, masks, and transcriptions. We also introduce Text Refinement Network (TexRNet), a novel text segmentation approach that adapts to the unique properties of text, e.g. non-convex boundary, diverse texture, etc., which often impose burdens on traditional segmentation models. In our TexRNet, we propose text-specific network designs to address such challenges, including key features pooling and attention-based similarity checking. We also introduce trimap and discriminator losses that show significant improvement in text segmentation. Extensive experiments are carried out on both our TextSeg dataset and other existing datasets. We demonstrate that TexRNet consistently improves text segmentation performance by nearly 2% compared to other state-of-the-art segmentation methods. Our dataset and code can be found at https://github.com/SHI-Labs/Rethinking-TextSegmentation.