InkFlow: Connected Handwriting Recognition for Natural Mid-Air Input in Mixed Reality

要旨

Mid-air handwriting is a freeform text input modality that enables expression of individuality and creativity. In Mixed Reality, the recognition of writing strokes has conventionally depended on manual action or proximal planes. Such explicit reliance imposes cognitive load and quickly leads to fatigue. We present InkFlow, a novel bare-hand handwriting interaction approach that enables users to write continuously and naturally without explicit stroke control. We first design a user-friendly pipeline that leverages the widely adopted pinch–release gesture to intuitively collect annotated handwriting data. Next, we enhance a lightweight DS-TCN model with boundary-aware strategy to improve the learning of kinematic features. Moreover, building on cross-domain meta-learning, our approach achieves effective cross-user generalization and supports rapid personalization for new users. The comparative user study (N=30) shows the effectiveness and usability of our method and interaction design. A closed-loop online study (N=12) further demonstrates notable improvements in handwriting efficiency and physical comfort.

著者
Xufeng Jian
Beijing University of Posts and Telecommunications, Beijing, China
Qi Qi
Beijing University of Posts and Telecommunications, Beijing, China
LinPei Zhang
Beijing University of Posts and Telecommunications, Beijing, China
Haifeng Sun
Beijing University of Posts and Telecommunications, Beijing, China
pengfei ren
Beijing University of Posts and Telecommunications, Beijing, China
Xiang Chen
Beijing University of Posts and Telecommunications, Beijing, China
Guangtian Liu
Beijing University of Posts and Telecommunications, Beijing, China
Shule Cao
Beijing University of Posts and Telecommunications, Beijing, China
Jingyu Wang
Beijing University of Posts and Telecommunications, Beijing, China

会議: CHI 2026

ACM CHI Conference on Human Factors in Computing Systems

セッション: Spatial Input and 3DUI

P1 - Room 111
7 件の発表
2026-04-14 20:15:00
2026-04-14 21:45:00