Visualization and Data

会議の名前
CHI 2023
Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing Users
要旨

Speech is expressive in ways that caption text does not capture, with emotion or emphasis information not conveyed. We interviewed eight Deaf and Hard-of-Hearing (DHH) individuals to understand if and how captions' inexpressiveness impacts them in online meetings with hearing peers. Automatically captioned speech, we found, lacks affective depth, lending it a hard-to-parse ambiguity and general dullness. Interviewees regularly feel excluded, which some understand is an inherent quality of these types of meetings rather than a consequence of current caption text design. Next, we developed three novel captioning models that depicted, beyond words, features from prosody, emotions, and a mix of both. In an empirical study, 16 DHH participants compared these models with conventional captions. The emotion-based model outperformed traditional captions in depicting emotions and emphasis, with only a moderate loss in legibility, suggesting its potential as a more inclusive design for captions.

著者
Caluã de Lacerda Pataca
Rochester Institute of Technology, Rochester, New York, United States
Matthew P. Watkins
Rochester Institute of Technology, Rochester, New York, United States
Roshan L. Peiris
Rochester Institute of Technology, Rochester, New York, United States
Sooyeon Lee
New Jersey Institute of Technology, Newark, New Jersey, United States
Matt Huenerfauth
Rochester Institute of Technology, Rochester, New York, United States
論文URL

https://doi.org/10.1145/3544548.3581511

動画
Contributing to Accessibility Datasets: Reflections on Sharing Study Data by Blind People
要旨

To ensure that AI-infused systems work for disabled people, we need to bring accessibility datasets sourced from this community in the development lifecycle. However, there are many ethical and privacy concerns limiting greater data inclusion, making such datasets not readily available. We present a pair of studies where 13 blind participants engage in data capturing activities and reflect with and without probing on various factors that influence their decision to share their data via an AI dataset. We see how different factors influence blind participants' willingness to share study data as they assess risk-benefit tradeoffs. The majority support sharing of their data to improve technology but also express concerns over commercial use, associated metadata, and the lack of transparency about the impact of their data. These insights have implications for the development of responsible practices for stewarding accessibility datasets, and can contribute to broader discussions in this area.

著者
Rie Kamikubo
University of Maryland, College Park, Maryland, United States
Kyungjun Lee
University of Maryland, College Park, Maryland, United States
Hernisa Kacorri
University of Maryland, College Park, Maryland, United States
論文URL

https://doi.org/10.1145/3544548.3581337

動画
Exploring Chart Question Answering for Blind and Low Vision Users
要旨

Data visualizations can be complex or involve numerous data points, making them impractical to navigate using screen readers alone. Question answering (QA) systems have the potential to support visualization interpretation and exploration without overwhelming blind and low vision (BLV) users. To investigate if and how QA systems can help BLV users in working with visualizations, we conducted a Wizard of Oz study with 24 BLV people where participants freely posed queries about four visualizations. We collected 979 queries and mapped them to popular analytic task taxonomies. We found that retrieving value and finding extremum were the most common tasks, participants often made complex queries and used visual references, and the data topic notably influenced the queries. We compile a list of design considerations for accessible chart QA systems and make our question corpus publicly available to guide future research and development.

著者
Jiho Kim
University of Wisconsin-Madison, Madison, Wisconsin, United States
Arjun Srinivasan
Tableau Research, Seattle, Washington, United States
Nam Wook Kim
Boston College, Chestnut Hill, Massachusetts, United States
Yea-Seul Kim
University of Wisconsin-Madison, Madison, Wisconsin, United States
論文URL

https://doi.org/10.1145/3544548.3581532

動画
Accessible Data Representation with Natural Sound
要旨

Sonification translates data into non-speech audio. Such auditory representations can make data visualization accessible to people who are blind or have low vision (BLV). This paper presents a sonification method for translating common data visualization into a blend of natural sounds. We hypothesize that people's familiarity with sounds drawn from nature, such as birds singing in a forest, and their ability to listen to these sounds in parallel, will enable BLV users to perceive multiple data points being sonified at the same time. Informed by an extensive literature review and a preliminary study with 5 BLV participants, we designed an accessible data representation tool, Susurrus, that combines our sonification method with other accessibility features, such as keyboard interaction and text-to-speech feedback. Finally, we conducted a user study with 12 BLV participants and report the potential and application of natural sounds for sonification compared to existing sonification tools.

著者
Md Naimul Hoque
University of Maryland, College Park, Maryland, United States
Md Ehtesham-Ul-Haque
Pennsylvania State University, University Park, Pennsylvania, United States
Niklas Elmqvist
University of Maryland, College Park, College Park, Maryland, United States
Syed Masum Billah
Pennsylvania State University, University Park , Pennsylvania, United States
論文URL

https://doi.org/10.1145/3544548.3581087

動画
Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access
要旨

Presentation slides commonly use visual patterns for structural navigation, such as titles, dividers, and build slides. However, screen readers do not capture such intention, making it time-consuming and less accessible for blind and visually impaired (BVI) users to linearly consume slides with repeated content. We present Slide Gestalt, an automatic approach that identifies the hierarchical structure in a slide deck. Slide Gestalt computes the visual and textual correspondences between slides to generate hierarchical groupings. Readers can navigate the slide deck from the higher-level section overview to the lower-level description of a slide group or individual elements interactively with our UI. We derived side consumption and authoring practices from interviews with BVI readers and sighted creators and an analysis of 100 decks. We performed our pipeline with 50 real-world slide decks and a large dataset. Feedback from eight BVI participants showed that Slide Gestalt helped navigate a slide deck by anchoring content more efficiently, compared to using accessible slides.

著者
Yi-Hao Peng
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Peggy Chi
Google Research, Mountain View, California, United States
Anjuli Kannan
Google, New York, New York, United States
Meredith Ringel. Morris
Google Research, Seattle, Washington, United States
Irfan Essa
Google, Atlanta, Georgia, United States
論文URL

https://doi.org/10.1145/3544548.3580921

動画
“The less I type, the better”: How AI Language Models can Enhance or Impede Communication for AAC Users
要旨

Users of augmentative and alternative communication (AAC) devices sometimes find it difficult to communicate in real time with others due to the time it takes to compose messages. AI technologies such as large language models (LLMs) provide an opportunity to support AAC users by improving the quality and variety of text suggestions. However, these technologies may fundamentally change how users interact with AAC devices as users transition from typing their own phrases to prompting and selecting AI-generated phrases. We conducted a study in which 12 AAC users tested live suggestions from a language model across three usage scenarios: extending short replies, answering biographical questions, and requesting assistance. Our study participants believed that AI-generated phrases could save time, physical and cognitive effort when communicating, but felt it was important that these phrases reflect their own communication style and preferences. This work identifies opportunities and challenges for future AI-enhanced AAC devices.

著者
Stephanie Valencia
Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
Richard Cave
UCL, LONDON, United Kingdom
Krystal Kallarackal
Google Research, Mountain View, California, United States
Katie Seaver
Google Research, Mountain View, California, United States
Michael Terry
Google, Cambridge, Massachusetts, United States
Shaun Kane
Google Research, Boulder, Colorado, United States
論文URL

https://doi.org/10.1145/3544548.3581560

動画