English

Crepe: A Mobile Screen Data Collector Using Graph Query

Human-Computer Interaction 2026-04-24 v3

Abstract

Collecting mobile datasets remains challenging for academic researchers due to limited data access and technical barriers. Commercial organizations often possess exclusive access to mobile data, leading to a "data monopoly" that restricts the independence of academic research. Existing open-source mobile data collection frameworks primarily focus on mobile sensing data rather than screen content, which is crucial for various research studies. We present Crepe, a no-code Android app that enables researchers to collect information displayed on screen through simple demonstrations of target data. Crepe utilizes a novel Graph Query technique which augments the structures of mobile UI screens to support flexible identification, location, and collection of specific data pieces. The tool emphasizes participants' privacy and agency by providing full transparency over collected data and allowing easy opt-out. We designed and built Crepe for research purposes only and in scenarios where researchers obtain explicit consent from participants. Code for Crepe will be open-sourced to support future academic research data collection.

Keywords

Cite

@article{arxiv.2406.16173,
  title  = {Crepe: A Mobile Screen Data Collector Using Graph Query},
  author = {Yuwen Lu and Meng Chen and Qi Zhao and Victor Cox and Yang Yang and Meng Jiang and Jay Brockman and Tamara Kay and Toby Jia-Jun Li},
  journal= {arXiv preprint arXiv:2406.16173},
  year   = {2026}
}

Comments

Best Paper Honorable Mention Award at CHI 2026 (Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems), Article No. 296, 20 pages

R2 v1 2026-06-28T17:16:32.250Z