A new conversational interaction concept for document creation and editing on mobile devices for visually impaired users

by   Alireza Darvishy, et al.

This paper describes the ongoing development of a conversational interaction concept that allows visually impaired users to easily create and edit text documents on mobile devices using mainly voice input. In order to verify the concept, a prototype app was developed and tested for both iOS and Android systems, based on the natural-language understanding (NLU) platform Google Dialogflow. The app and interaction concept were repeatedly tested by users with and without visual impairments. Based on their feedback, the concept was continuously refined, adapted and improved on both mobile platforms. In an iterative user-centred design approach, the following research questions were investigated: Can a visually impaired user rely mainly on speech commands to efficiently create and edit a document on mobile devices? User testing found that an interaction concept based on conversational speech commands was easy and intuitive for visually impaired users. However, it was also found that relying on speech commands alone created its own obstacles, and that a combination of gestures and voice interaction would be more robust. Future research and more extensive useability tests should be carried out among visually impaired users in order to optimize the interaction concept.


page 1

page 2

page 3

page 4


A Tool for Conducting User Studies on Mobile Devices

With the ever-growing interest in the area of mobile information retriev...

VoiceMask: Anonymize and Sanitize Voice Input on Mobile Devices

Voice input has been tremendously improving the user experience of mobil...

Convo: What does conversational programming need? An exploration of machine learning interface design

Vast improvements in natural language understanding and speech recogniti...

Science Driven Innovations Powering Mobile Product: Cloud AI vs. Device AI Solutions on Smart Device

Recent years have witnessed the increasing popularity of mobile devices ...

Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI

Intelligent assistants that follow commands or answer simple questions, ...

A Metamodel and Prototype for Fluid Document Formats

With the transformation of computing from personal computers to the Inte...

Why Are Conversational Assistants Still Black Boxes? The Case For Transparency

Much has been written about privacy in the context of conversational and...

Please sign up or login with your details

Forgot password? Click here to reset