Abstract

Optical Chemical Structure Recognition (OCSR) — the task of converting molecular structure images into machine-readable representations — stands at a pivotal inflection point. Four distinct paradigms now compete for dominance: rule-based classical pipelines, end-to-end SMILES sequence generation, graph-first structured prediction, and Vision-Language Model (VLM)-driven approaches. This article critically surveys the strengths and limitations of each paradigm, reviews key systems published from 2020 through 2026, and argues that the future of production-grade OCSR lies in a hybrid graph-first architecture augmented by VLM reasoning — rather than in the "image captioning" metaphor that has dominated the field.

mol:CC(C)[C@@H](C#N)NC(=O)O{150, 100} ▶▶▶▶ CC(C)[C@@H](C#N)NC(=O)O

1. Introduction

Optical Chemical Structure Recognition plays a pivotal role in modern chemical informatics, enabling the automated conversion of chemical structure images from scientific literature, patents, and educational materials into machine-readable molecular representations. This capability is essential for large-scale chemical data mining, drug discovery pipelines, and Large Language Model (LLM) applications in related domains.

The scale of the challenge is staggering. In recent decades, chemistry publications and patents have increased rapidly, and a significant portion of key information is embedded in molecular structure figures, complicating large-scale literature searches and limiting the application of large language models in fields such as biology, chemistry, and pharmaceuticals.

Against this backdrop, a critical architectural debate has emerged: should OCSR models directly generate SMILES strings via autoregressive decoding, or should they reconstruct molecular graphs — atoms, bonds, and coordinates — as an intermediate representation? And now, with the rise of powerful VLMs, a third question arises: can general-purpose visual reasoning models subsume the task entirely?

This article systematically addresses all three questions. We trace the four generations of OCSR methodology, analyze their fundamental trade-offs, and propose a principled architecture for industrial-grade deployment.

2. Four Generations of OCSR

2.1 Generation 1: Rule-Based Classical Systems

OCSR tool development can be traced as far back as 1992, with the Kekule system by McDaniel & Balmuth. Classical systems such as OSRA (Filippov & Nicklaus, 2009), MolVec, and Imago rely on hand-crafted image processing pipelines: binarization, skeletonization, line segment detection, OCR text recognition, and graph assembly. These systems perform robustly on clean vector graphics but degrade on scanned documents, hand-drawn structures, and images containing abbreviations.

2.2 Generation 2: End-to-End SMILES Generation

The second generation, pioneered by DECIMER, reframed OCSR as an image captioning problem. A visual encoder (CNN or ViT) encodes the molecular image, and an autoregressive Transformer decoder generates a SMILES, DeepSMILES, or SELFIES string token by token. Models in this family — including SwinOCSR, Img2Mol, Image2SMILES, and ABC-Net — share the same fundamental architecture: a convolutional or transformer-based image encoder paired with a sequence decoder that outputs a linear molecular representation.

This paradigm is elegantly simple, but suffers from deep structural problems, analyzed in Section 3.

2.3 Generation 3: Graph-First Structured Prediction

The third generation reframes OCSR as a structured prediction problem: detect atoms and their coordinates, detect bonds and their types, and assemble a molecular graph that can be directly ingested by cheminformatics toolkits. MolScribe, MolGrapher, and MolNexTR are the canonical representatives of this line.

2.4 Generation 4: Vision-Language Models (VLMs)

While recent vision-language models (VLMs) have shown promise, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations. New systems such as **GTR-VL**, **MolSight**, and **MolParser** introduce architectural innovations — graph traversal reasoning, reinforcement learning, and curriculum learning — to overcome these limitations. This generation is explored in depth in Section 5.

3. The Fundamental Flaws of End-to-End SMILES Generation

3.1 Hallucination and Lack of Visual Grounding

When a Transformer decoder generates a SMILES string, each token is produced from learned statistical priors rather than precise visual evidence. If a six-membered ring in the input image is partially occluded or blurry, the model may "hallucinate" a benzene ring simply because aromatic systems are overrepresented in training corpora. There is no mechanism to trace a generated atom back to a specific pixel region in the source image, rendering the output fundamentally unexplainable.

3.2 Stereochemistry Failures

Existing OCSR systems face significant challenges in accurately recognizing stereochemical information due to the subtle visual cues that distinguish stereoisomers, such as wedge and dash bonds, ring conformations, and spatial arrangements. Autoregressive models are prone to correctly recovering the molecular connectivity graph while misassigning stereochemical configurations at chiral centers — a failure that is chemically catastrophic, as enantiomers can have radically different biological activities.

3.3 Syntax and Chemical Validity

SMILES is a context-sensitive grammar. Mismatched parentheses, incorrect ring-closure digits, or invalid branch nesting produce strings that cannot be parsed by RDKit or other cheminformatics toolkits. While SELFIES guarantees syntactic validity, it does not resolve the deeper issue of chemical plausibility — a SELFIES string may be syntactically valid yet correspond to a physically impossible molecule.

3.4 Incompatibility with Human-in-the-Loop Workflows

In industrial settings — patent mining, electronic lab notebooks, regulatory submissions — OCSR outputs inevitably require human correction. When the output is a flat SMILES string, there is no spatial correspondence between the string and the original image. A chemist cannot click on a misidentified atom in the image to correct it; they must mentally decode the SMILES, locate the error, and manually edit the string — a process that is both tedious and error-prone.

3.5 Loss of Layout Information

The original 2D layout of the molecule — critical for visual comparison in patent documents and for understanding stereochemistry — is entirely discarded when the output is a linear SMILES string. Wedge bonds, hash bonds, and the spatial arrangement of substituents are compressed into @/@@ and //\ notations that models find notoriously difficult to predict correctly.

4. The Graph-First Paradigm: Atoms, Bonds, and Coordinates

4.1 Core Advantages

Seamless cheminformatics integration. Once a molecular graph with 2D coordinates is constructed in memory (as an RDKit Mol object or an MDL V2000/V3000 MolBlock), conversion to any desired format — SMILES, InChI, SDF, SVG — is trivial. This decouples the recognition problem from the representation problem.

Native support for interactive correction. Because every atom and bond carries its original (x, y) coordinate in the source image, the recognition result can be overlaid directly onto the input. A chemist can click on a misidentified atom, delete a spurious bond, or add a missing functional group through an intuitive graphical interface.

Explainability and debuggability. When recognition fails, the graph-based approach makes it straightforward to diagnose where the failure occurred: Was an atom misclassified by the detection module? Was a bond missed by the detection head? Was the ring-closure algorithm confused by a crowded region? This granular error attribution is impossible with end-to-end SMILES generation.

Layout preservation. Retaining 2D coordinates ensures that the spatial arrangement of the molecule — substituent orientation, macrocycle shape, relative positioning of fused ring systems — is faithfully preserved from the original document.

4.2 Key Systems

MolScribe (MIT, J. Chem. Inf. Model., 2023)

MolScribe is an image-to-graph generation model that explicitly predicts atoms and bonds along with their geometric layout, using a Swin Transformer encoder paired with a Transformer decoder. The model jointly predicts atom types, bond types, and (x, y) coordinates, producing a structured output that is assembled into a molecular graph.

MolGrapher (IBM/EPFL, ICCV 2023)

MolGrapher (Morin et al., 2023) was presented at ICCV, proposing a graph-based approach to the visual recognition of chemical structures. The system builds a hypergraph over detected keypoints and classifies nodes using a Graph Neural Network, recovering full atom and bond identities. This detection-centric design enables MolGrapher to handle larger molecules more gracefully than captioning-based methods.

MolNexTR (J. Cheminformatics, 2024)

Chen et al. introduced MolNexTR, a generalized deep learning model for molecular image recognition, published in the *Journal of Cheminformatics* in 2024. MolNexTR combines a ConvNeXt backbone with a Vision Transformer encoder and a Transformer decoder to simultaneously output both a SMILES string and an atom-coordinate-annotated MolFile — enabling direct visualization aligned to the original structure orientation.

MarkushGrapher (IBM/ETH, CVPR 2025)

Morin et al. followed up with MarkushGrapher, a system for joint visual and textual recognition of Markush structures, presented at CVPR 2025. This extension directly addresses the challenge of variable R-group notation and combinatorial structure enumeration that appears ubiquitously in patent literature.

4.3 Persistent Challenges

The graph-first approach is not without difficulties:

  • Cascading errors: Multi-stage pipelines are susceptible to error propagation — a single missed atom detection can break an entire ring system.
  • Superatom abbreviations: Handling COOH, OTf, Boc, Ph, and similar abbreviations requires a robust OCR module coupled with an extensive chemical dictionary for expansion.
  • Implicit hydrogen completion and valence balancing: Post-processing rules are required to avoid RDKit sanitization failures.
  • Fixed-resolution encoders: Even graph-first models based on ViT encoders (e.g., MolScribe) are still constrained by quadratic self-attention complexity at high resolutions.

5. The Resolution Curse: Why Transformers Fail on Large Molecules

5.1 Quadratic Complexity and the Patch Bottleneck

The self-attention mechanism in Vision Transformers scales as \(O(N^2)\) with respect to the number of patches \(N\). For a 384×384 input with 16×16 patches, the sequence length is a manageable 576. However, a 1024×1024 image — necessary to resolve fine details of a large natural product or macrocycle — yields a sequence length of 4,096, causing memory consumption to explode quadratically.

5.2 Information Destruction via Resizing

Most current OCSR models enforce a fixed input resolution (typically 384×384 or 448×448). When a complex molecule with 80+ heavy atoms — such as paclitaxel, rapamycin, or vancomycin — is resized to fit this constraint:

  • Individual bonds shrink to 1–2 pixels, making double and triple bonds indistinguishable;
  • Text labels (e.g., OMe, NHBoc) blur into illegible blobs;
  • Dense polycyclic systems (steroids, porphyrins, fullerenes) collapse into unresolvable clusters.

5.3 The Benchmark Illusion

Standard benchmarks (USPTO, UOB, JPO, CLEF) are biased toward small, drug-like molecules. The presence of numerous Markush structures in real-world documents, along with variations in molecular image quality, drawing styles, and noise, significantly limits the performance of existing OCSR methods when they are deployed beyond their training distribution.

A 2024 comparative study evaluating eight open-source OCSR tools found F1 scores ranging from 34% to 93%, with no single tool consistently outperforming others across all categories of chemical structure images — a stark reminder that benchmark performance and real-world performance remain poorly correlated.

6. Generation 4: Vision-Language Models Enter the Arena

6.1 MolParser (DP Technology / AI for Science Institute, ICCV 2025)

MolParser is a novel end-to-end OCSR method that efficiently and accurately recognizes chemical structures from real-world documents, including difficult Markush structures. Trained using a curriculum learning approach, MolParser significantly outperforms classical and learning-based methods across most scenarios, with potential for broader downstream applications. The model uses an extended SMILES encoding rule to represent Markush structures, connection points, and abstract ring systems that standard SMILES cannot express.

6.2 GTR-VL / GTR-CoT (Shanghai AI Lab et al., arXiv 2025–2026)

GTR-VL features two key innovations: (1) the *Graph Traversal as Visual Chain of Thought* mechanism that emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions, and (2) the data-centric *Faithfully Recognize What You've Seen* principle, which aligns abbreviated structures in images with their expanded annotations.

Concretely, instead of predicting atoms and bonds in isolation or jumping straight to a SMILES string, GTR-VL performs a depth-first traversal of the molecular graph, interleaving atom and bond predictions in a single sequence to mimic human reasoning. This step-by-step approach provides mutual constraints — bonds are predicted based on previously identified atoms — which significantly reduces the search space and improves structural consistency.

For hand-drawn OCSR tasks, where datasets lack graph annotations and only provide final SMILES, reinforcement learning using the GRPO method is applied, with reward mechanisms including format reward, graph reward, and SMILES reward. To support model development, the authors constructed GTR-CoT-1.3M, a large-scale instruction-tuning dataset with meticulously corrected annotations, and introduced MolRec-Bench, the first benchmark designed for fine-grained evaluation of graph-parsing accuracy in OCSR. In scenarios involving molecular images with functional group abbreviations, GTR-Mol-VLM outperforms the second-best baseline by approximately 14 percentage points, in both SMILES-based and graph-based metrics.

6.3 MolSight (HUST, AAAI 2026)

MolSight is a comprehensive learning framework for OCSR, designed to bridge the gap between computer vision and chemical informatics. It introduces a three-stage training paradigm:
  1. In the first stage, MolSight supports pretraining using large quantities of images annotated only with SMILES, with the objective of enhancing the perception capabilities of the image encoder for molecular images.
  2. In the second stage, by introducing the additional chemical bond head and coordinate head, the performance of the model is improved.
  3. Reinforcement learning has shown tremendous potential in improving models' ability to solve complex reasoning tasks; MolSight innovatively introduces RL algorithms into OCSR tasks, utilizing images of stereoisomers that are commonly confused by the model for RL optimization, thereby further enhancing the model's understanding of chemical semantics.
MolSight is the first OCSR system to integrate Reinforcement Learning, utilizing Group Relative Policy Optimization (GRPO) to directly optimize chemical validity, and introduces Stereo-200K — a new annotated dataset of 200,000 challenging stereoisomeric molecules specifically curated to address confusion in 3D chiral structures. Extensive experiments demonstrate that MolSight achieves state-of-the-art results in accuracy, similarity, and robustness, outperforming both classical and learning-based baselines.

7. Benchmark Evolution: From Curated to Wild

The OCSR community is increasingly aware that standard benchmarks do not reflect the difficulty of real-world documents. Several new evaluation frameworks have been introduced:

Benchmark Year Key Property
USPTO / UOB / JPO / CLEF 2009–2020 Clean, small-molecule biased; standard evaluation
MolRec-Bench 2025 Addresses the gap in existing OCSR evaluation sets by providing means to assess molecular graph structure parsing results
GTR-CoT-1.3M 2025 Large-scale instruction-tuning dataset with corrected abbreviation annotations
Stereo-200K 2025 200K stereoisomeric molecules for chiral recognition training
MolParser-7M 2024 Large-scale pretraining corpus covering diverse image styles and patent sources

A recurring finding is that models achieving >90% exact match on USPTO can fail catastrophically on images drawn from total synthesis publications — underscoring the importance of distributional diversity in both training data and evaluation.

8. Current Systems: A Comparative Overview

System Institution Venue Paradigm Key Innovation
DECIMER 2.x Steinbeck Lab J. Cheminform. 2021–2024 EfficientNet + Transformer → SMILES Pioneer deep-learning OCSR; continuous iteration
MolScribe MIT (Coley & Barzilay) J. Chem. Inf. Model. 2023 Swin + Transformer → graph Explicit atom coordinates + bonds; strong small-molecule SOTA
MolGrapher IBM / EPFL ICCV 2023 CNN keypoints + GNN Hypergraph + node classification; robust on large molecules
MolNexTR Chen et al. J. Cheminform. 2024 ConvNeXt + ViT + Transformer Dual output: SMILES + MolFile with coordinates
MarkushGrapher IBM / ETH CVPR 2025 Detection + GNN Joint visual + textual recognition of Markush structures
MolParser DP Technology ICCV 2025 Image captioning + curriculum learning Markush support; extended SMILES; trained on MolParser-7M
GTR-VL Shanghai AI Lab et al. arXiv 2025–2026 VLM + graph traversal CoT + GRPO Depth-first atom-bond prediction; abbreviation alignment
MolSight HUST AAAI 2026 EfficientViT + 3-stage RL training First RL-OCSR; GRPO reward; Stereo-200K dataset
OSRA NIH (Filippov) J. Chem. Inf. Model. 2009 Classical CV / rule-based No deep learning; stable on clean vector graphics

9. A Principled Architecture for Production OCSR

Based on the preceding analysis, we recommend the following hybrid architecture for an industrial-grade OCSR system:

Input Image (arbitrary resolution — no forced resizing)
│
▼
┌──────────────────────────────────────────────────────────┐
│            High-Resolution Backbone                      │
│      (fully convolutional or linear attention)           │  ← ConvNeXt / HRNet / EfficientViT
│            O(N) or O(N log N) complexity                 │
└─────────────────────────┬────────────────────────────────┘
                          │
          ┌───────────────┼───────────────┬───────────────┐
          ▼               ▼               ▼               ▼
   ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
   │  Atom Det.  │ │  Bond Det.  │ │Stereo Head  │ │  OCR Head   │
   │ (bbox+cls)  │ │ (skeleton)  │ │(wedge/hash) │ │  (abbrev.)  │
   └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘
          └───────────────┴───────────────┴───────────────┘
                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │   Geometric + Graph Assembly   │  ← Ring perception,
                    │  (spatial matching, topology)  │    valence check,
                    │                               │    superatom expansion
                    └───────────────┬───────────────┘
                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │    RDKit Validation & Export   │  ← SanitizeMol →
                    │  (MolBlock, SDF, InChI, SVG)  │    SMILES / MolBlock / SVG
                    └───────────────┬───────────────┘
                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │   (Optional) VLM Second Pass  │  ← Low-confidence re-check,
                    │                               │    hand-drawn structures,
                    │                               │    Markush / abbreviation
                    │                               │    resolution
                    └───────────────────────────────┘

Key design principles:

  1. Never resize the input. Use fully convolutional backbones or linear-attention encoders that accept arbitrary resolutions.
  2. Predict coordinates, not sequences. Every atom and bond must be grounded in pixel space to enable overlay, debugging, and interactive correction.
  3. Decouple recognition from representation. Delegate format conversion and chemical validation to established cheminformatics toolkits (RDKit, OpenBabel).
  4. Design for human-in-the-loop correction. Overlay predictions on the original image and provide interactive editing tools — essential for achieving the >99% accuracy required in production.
  5. Use ensemble / multi-model strategies. Cross-validating outputs from multiple systems (e.g., a rule-based tool + a graph-first model) and flagging disagreements for human review is a practical path to production robustness.
  6. Address stereochemistry as a first-class problem. Train on stereochemically diverse datasets (e.g., Stereo-200K) and include dedicated stereo-prediction heads or RL post-training stages.

10. Conclusion

The prevailing practice of treating OCSR as an image-to-text translation problem — directly generating SMILES strings via Transformer decoders — represents a misapplication of NLP paradigms to a fundamentally geometric and topological task. Molecular structure diagrams are not natural language; they are precise spatial graphs governed by strict chemical rules.

While recent vision-language models have shown potential in this task, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations.

The correct paradigm for production OCSR is a graph-first reconstruction approach augmented by VLM-based reasoning: detect atoms and bonds at high resolution, recover their spatial coordinates, assemble a molecular graph, and delegate format conversion and chemical validation to established cheminformatics toolkits. For challenging cases — Markush structures, hand-drawn images, heavily abbreviated compounds — a VLM second pass using graph traversal chain-of-thought reasoning provides powerful complementary capability.

This hybrid approach offers:

  • Superior explainability — every atom and bond is spatially grounded;
  • Seamless human-in-the-loop correction — predictions overlay directly onto the source image;
  • Native support for arbitrary image resolutions — no forced downsampling of complex molecules;
  • Representational flexibility — any desired output format via RDKit;
  • Stereochemical fidelity — with dedicated stereo heads and RL-based post-training.

While it demands more engineering effort than end-to-end SMILES generation, it is the only path toward the accuracy, robustness, and interpretability required for industrial-scale chemical digitization.

The future of OCSR lies not in teaching machines to "caption" molecules, but in teaching them to "see" and "reconstruct" molecular graphs — atom by atom, bond by bond — and to reason about what they observe.

References

  1. McDaniel, J.R. & Balmuth, J.R. Kekule: OCR-optical chemical (structure) recognition. J. Chem. Inf. Comput. Sci. (1992).
  2. Filippov, I.V. & Nicklaus, M.C. Optical Structure Recognition Software to Recover Chemical Information: OSRA, an Open Source Solution. J. Chem. Inf. Model. (2009).
  3. Rajan, K. et al. DECIMER: Towards Deep Learning for Chemical Image Recognition. J. Cheminform. (2020).
  4. Rajan, K. et al. DECIMER 1.0: Deep Learning for Chemical Image Recognition using Transformers. J. Cheminform. (2021).
  5. Rajan, K. et al. Advancements in hand-drawn chemical structure recognition through an enhanced DECIMER architecture. J. Cheminform. (2024).
  6. Qian, Y. et al. MolScribe: Robust Molecular Structure Recognition with Image-to-Graph Generation. J. Chem. Inf. Model. (2023).
  7. Morin, L. et al. MolGrapher: Graph-based Visual Recognition of Chemical Structures. ICCV (2023).
  8. Chen, Y. et al. MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition. J. Cheminform. (2024).
  9. Morin, L. et al. MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures. CVPR (2025).
  10. Fang, X. et al. MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild. ICCV (2025).
  11. Wang, J. et al. GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition. arXiv:2506.07553 (2025).
  12. Zhang, W. et al. MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning. AAAI (2026).
  13. Fan, V. et al. OpenChemIE: An Information Extraction Toolkit for Chemistry Literature. J. Chem. Inf. Model. (2024).
  14. Rajan, K. et al. MARCUS: Molecular Annotation and Recognition for Curating Unravelled Structures. Digital Discovery (2025).
  15. Krasnov, A. et al. Comparing Software Tools for Optical Chemical Structure Recognition. Digital Discovery (2024).