English

OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization

Computer Vision and Pattern Recognition 2026-04-08 v2

Abstract

The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.

Keywords

Cite

@article{arxiv.2602.07064,
  title  = {OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization},
  author = {Minghao Han and Dingkang Yang and Yue Jiang and Yizhou Liu and Lihua Zhang},
  journal= {arXiv preprint arXiv:2602.07064},
  year   = {2026}
}

Comments

This work has been submitted to the IEEE for possible publication

R2 v1 2026-07-01T10:25:04.790Z