Collecting mobile screen information datasets remains challenging for academic researchers. Commercial organizations often have exclusive access to mobile data, leading to a “data monopoly” that restricts academic research and user transparency. Existing open-source mobile data collection frameworks primarily focus on mobile sensing data rather than screen content. We present Crepe, a no-code Android app that enables researchers to collect information displayed on screen through simple demonstrations of target data. Crepe utilizes a novel Graph Query technique, which augments mobile UI structures to support flexible identification, location, and collection of specific data pieces. The tool emphasizes participants' privacy and agency by providing full transparency over collected data and allowing easy opt-out. We designed and built Crepe for research purposes only and in scenarios where researchers obtain explicit consent from participants. Code for Crepe will be open-sourced to support future academic research data collection.
ACM CHI Conference on Human Factors in Computing Systems