English

Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach

Robotics 2025-05-20 v3 Computer Vision and Pattern Recognition Machine Learning

Abstract

Automating leaf manipulation in agricultural settings faces significant challenges, including the variability of plant morphologies and deformable leaves. We propose a novel hybrid geometric-neural approach for autonomous leaf grasping that combines traditional computer vision with neural networks through self-supervised learning. Our method integrates YOLOv8 for instance segmentation and RAFT-Stereo for 3D depth estimation to build rich leaf representations, which feed into both a geometric feature scoring pipeline and a neural refinement module (GraspPointCNN). The key innovation is our confidence-weighted fusion mechanism that dynamically balances the contribution of each approach based on prediction certainty. Our self-supervised framework uses the geometric pipeline as an expert teacher to automatically generate training data. Experiments demonstrate that our approach achieves an 88.0% success rate in controlled environments and 84.7% in real greenhouse conditions, significantly outperforming both purely geometric (75.3%) and neural (60.2%) methods. This work establishes a new paradigm for agricultural robotics where domain expertise is seamlessly integrated with machine learning capabilities, providing a foundation for fully automated crop monitoring systems.

Keywords

Cite

@article{arxiv.2505.03702,
  title  = {Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach},
  author = {Srecharan Selvam},
  journal= {arXiv preprint arXiv:2505.03702},
  year   = {2025}
}

Comments

15 pages, 9 figures

R2 v1 2026-06-28T23:23:17.326Z