This is a streamlined abridgement of my paper with Pierre Albert, published at LREC’s Workshop on Legal and Ethical Issues in Human Language Technologies 2020. If you use any of this guide in your research, please do cite our paper titled “Ethically Collecting Multi-Modal Spontaneous Conversations with People that have Cognitive Impairments”:

Harvard:
Addlesee, A. and Albert, P., 2020\. Ethically Collecting Multi-Modal Spontaneous Conversations with People that have Cognitive Impairments. LREC Workshop on Legal and Ethical Issues in Human Language Technologies.

BibTeX:
@article{addlesee2020ethically,
  title={Ethically Collecting Multi-Modal Spontaneous Conversations with People that have Cognitive Impairments},
  author={Addlesee, Angus and Albert, Pierre},
  journal={LREC Workshop on Legal and Ethical Issues in Human Language Technologies},
  year={2020}
}

_Getting ethical approval to collect a crucial corpus took me over a year to complete. This was relatively fresh ground to tread, but I hope other researchers want to work on the accessibility of voice assistants for people with all varieties of cognitive impairments. _This practical guide aims to help future researchers, like me, collect these valuable datasets quickly without compromising any ethical considerations or data security.

Introduction

Over a year ago now, I decided that I wanted to work to make voice assistants (Siri, Alexa, etc…) more accessible for people with dementia. To begin this project, I (with two of my supervisors) first detailed some of the critical challenges that need to be tackled if we are to make progress towards this goal. One huge hurdle is the lack of suitable data… so let’s just collect it?

I first decided exactly what data was needed and worked to organise the task, recording location, recording equipment, etc… I planned to record both audio and video recordings of friendly, off the cuff conversations with people that have dementia. As a mathematician/computer scientist, there was one major job that I was not experienced enough to do without assistance however - the ethical approval.

source

I spent over a year talking to experts from the NHS, charities, dementia research centres, Psychology departments, ethics boards, (the list goes on) to ensure this data collection was completed both ethically and securely.

This article includes what I learned from these experts as a _practical guide _and details of a device, “CUSCO” developed by Pierre Albert, to capture and store this data as securely as possible.

The Data

I have discussed why this data is important previously but in the context of this guide, I will expand on that before briefly describing exactly what data we are collecting.

Natural Speech

While we have natural everyday conversations with one and other, we don’t talk cleanly as our speech is littered with beautiful conversational phenomena:

• Hesitations & Pauses - “I went to that great coffee shop**…** Peppers!”
• Filled Pauses - “I went to that great coffee shop… umm… Peppers!”
• Repairs - “Then go to the left… no no, to the right!”
• Repetition - “It was so so so good!”
• Backchannels - (someone on the phone) “Yep… uhu… hmm…”

When we speak to voice assistants however, we remove all of these phenomena from our speech, called Computer Talk. To illustrate this, which would you say to a voice assistant (called Jacob):

• “Jacob, my hands are wet so could you set a timer for… umm… I don’t know… a a couple minutes please.”
• “Jacob, set a timer for two minutes”

We all know which one because…

We adapt to the capabilities of the system! We learn what functionalities are available and how best to speak in order to make them work. For older adults (with a lack of experience using voice assistants) and those with cognitive impairments, these “Smart Devices” can be extremely frustrating. This is because voice assistants do not understand natural communication, with all the above phenomena (and more), which is exactly how people have always communicated.

#healthcare #artificial-intelligence #editors-pick #dementia #ethics