Previous research shows that laypeople’s trust in a machine learning model can be affected by both performance measurements of the model on the aggregate level and performance estimates on individual predictions. However, it is unclear how people would trust the model when multiple performance indicators are presented at the same time. We conduct an exploratory human-subject experiment to answer this question. We find that while the level of model confidence significantly affects people’s belief in model accuracy, both the model’s stated and observed accuracy generally have a larger impact on people’s willingness to follow the model’s predictions as well as their self-reported levels of trust in the model, especially after observing the model’s performance in practice. We hope the empirical evidence reported in this work could open doors to further studies to advance understanding of how people perceive, process, and react to performance-related information of machine learning.
https://dl.acm.org/doi/abs/10.1145/3491102.3501967
AI systems are becoming increasingly pervasive within children's devices, apps, and services. However, it is not yet well-understood how risks and ethical considerations of AI relate to children. This paper makes three contributions to this area: first, it identifies ten areas of alignment between general AI frameworks and codes for age-appropriate design for children. Then, to understand how such principles relate to real application contexts, we conducted a landscape analysis of children's AI systems, via a systematic literature review including 188 papers. This analysis revealed a wide assortment of applications, and that most systems' designs addressed only a small subset of principles among those we identified. Finally, we synthesised our findings in a framework to inform a new ``Code for Age-Appropriate AI'', which aims to provide timely input to emerging policies and standards, and inspire increased interactions between the AI and child-computer interaction communities.
https://dl.acm.org/doi/abs/10.1145/3491102.3502057
Inspiration from design examples plays a crucial role in the creative process of user interface design. However, current tools and techniques that support inspiration usually only focus on example browsing with limited user control or similarity-based example retrieval, leading to undesirable design outcomes such as focus drift and design fixation. To address these issues, we propose the GANSpiration approach that suggests design examples for both targeted and serendipitous inspiration, leveraging a style-based Generative Adversarial Network. A quantitative evaluation revealed that the outputs of GANSpiration-based example suggestion approaches are relevant to the input design, and at the same time include diverse instances. A user study with professional UI/UX practitioners showed that the examples suggested by our approach serve as viable sources of inspiration for overall design concepts and specific design elements. Overall, our work paves the road of using advanced generative machine learning techniques in supporting the creative design practice.
https://dl.acm.org/doi/abs/10.1145/3491102.3517511
Questionnaires are fundamental learning and research tools for gathering insights and information from individuals, and now can be created easily using online tools. However, existing resources for creating questionnaires are designed for written languages (e.g. English) and do not support sign languages (e.g. American Sign Language). Sign languages (SLs) have unique visual characteristics that do not fit into user interface paradigms designed for written, text-based languages. Through a series of formative studies with the ASL signing community, this paper takes steps towards understanding the viability, potential benefit, challenges, and user interest in SL-centric surveys, a novel approach for creating questionnaires that meet the needs of deaf individuals using sign languages, without obligatory reliance on a written language to complete a questionnaire.
https://dl.acm.org/doi/abs/10.1145/3491102.3502047
Manual sign systems have been introduced to improve the communication of children with intellectual developmental disabilities (IDD). Due to the lack of learning support tools, teachers face many practical challenges in teaching manual sign to children, such as low attention span and the need for persistent intervention. To address these issues, we collaborated with teachers to develop the Sondam Rhythm Game, a gesture-based rhythm game that assists in teaching manual sign language, and ran a four-week empirical study with five teachers and eight children with IDD. Based on video annotation and post-hoc interviews, our game-based learning approach has the potential to be effective at teaching manual sign to children with IDD. Our approach improved children attention span and motivation while also increasing the number of voluntary gestures made without the need for prompting. Other practical issues and learning challenges were also uncovered to improve teaching paradigms for children with IDD.
https://dl.acm.org/doi/abs/10.1145/3491102.3517456