Disabled lady made her AI voice clone say ‘knickers’ – then she acquired banned
A woman was given an artificial intelligence device to help her speak following a diagnosis of motor neurone disease – but it quickly locked her out following her ‘saying knickers’
A woman who was granted the use of AI to communicate following losing her voice – but was banned shortly after for supposedly crude language.
Joyce Esser, who lives in the UK, shared the experience of getting her voice back through AI, but added it has plenty of limitations which led her to get banned by the production company. She was diagnosed with motor neurone disease, a progressive neurological disease that damages the nerve cells that control muscle. Joyce can move her limbs, but struggles to speak and swallow.
She has a high-tech AI tool built by ElevenLabs which has been fed recordings of their old voices to re-create them. Today, Joyce can “speak” in her old voice by typing sentences into devices, selecting letters by hand or eye gaze.
Joyce added it was an extremely emotional moment when she regained her voice, which was quickly brought down by the company’s language restrictions.
Following the incident, Joyce decided to not use her device that often. When she did, she was sent an official warning.
Joyce was waiting for her husband Paul to get ready, when her device read out: “Come on, Hunnie, get your a*** in gear!” She then added: “I’d better get my knickers on too,” according to Technology Review.
“The next day I got a warning from ElevenLabs that I was using inappropriate language and not to do it again!” Joyce admitted. She was not sure what had been flagged, as she believed it to be “normal British banter.”
Joyce assumed the warning would amount to nothing, but she was banned the next day from using her device. “I’d just got my voice back and now they’d taken it away from me … and only two days after I’d done a presentation to my local MND group telling them how amazing ElevenLabs were,” she added.
The company reinstated her account with an apology, but Joyce is still not aware what went wrong. According to their website, there are rules against threatening child safety, engaging in illegal behaviour, providing medical advice, impersonating others, interfering with elections, and more. However, there are no specific details on inappropriate language.
ElevenLabs’ terms of use state that the company does not have any obligation to screen, edit, or monitor content but add that it may “terminate or suspend” access to its services when content is “reasonably likely, in our sole determination, to violate applicable law or [the user] Terms.”
For the latest breaking news and stories from across the globe from the Daily Star, sign up for our newsletters.
