English

Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition

Computer Vision and Pattern Recognition 2026-04-22 v2 Artificial Intelligence Biomolecules

Abstract

Optical Chemical Structure Recognition (OCSR) is critical for converting 2D molecular diagrams from printed literature into machine-readable formats. While Vision-Language Models have shown promise in end-to-end OCR tasks, their direct application to OCSR remains challenging, and direct full-parameter supervised fine-tuning often fails. In this work, we adapt DeepSeek-OCR-2 for molecular optical recognition by formulating the task as image-conditioned SMILES generation. To overcome training instabilities, we propose a two-stage progressive supervised fine-tuning strategy: starting with parameter-efficient LoRA and transitioning to selective full-parameter fine-tuning with split learning rates. We train our model on a large-scale corpus combining synthetic renderings from PubChem and realistic patent images from USPTO-MOL to improve coverage and robustness. Our fine-tuned model, MolSeek-OCR, demonstrates competitive capabilities, achieving exact matching accuracies comparable to the best-performing image-to-sequence model. However, it remains inferior to state-of-the-art image-to-graph modelS. Furthermore, we explore reinforcement-style post-training and data-curation-based refinement, finding that they fail to improve the strict sequence-level fidelity required for exact SMILES matching.

Keywords

Cite

@article{arxiv.2604.03476,
  title  = {Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition},
  author = {Haocheng Tang and Xingyu Dang and Junmei Wang},
  journal= {arXiv preprint arXiv:2604.03476},
  year   = {2026}
}
R2 v1 2026-07-01T11:53:31.394Z