Deaf News: Live captioning software Ava launches in the UK

Ahead of the captioning tool getting its UK launch today, Liam sits down with Ava co-founder Thibault Duchemin to talk speech recognition, accessibility and his experience as a Child of Deaf Adults (CODA).

“We really started around the problem of the family dinner syndrome,” Thibault tells me over Zoom. He sits wearing a bright blue t-shirt adorned with an ampersand – the company logo – in what appears to be a large and expansive living room. I, on the other hand, sit at my wooden dining room table, glancing awkwardly at the Otter transcription app on my phone recording our conversation. For an interview about new captioning software launching today, using a rival app to take note of the conversation feels somewhat inappropriate.

Thibault adds that he had to interpret dinner table conversations himself as a CODA, something which he later couldn’t promise to his French parents when moving to the United States to study engineering.

He goes on to explain Ava’s perspective. “Instead of finding the closed caption button everywhere,” he says, “why not just own the closed caption button and be able to turn it on, anywhere you want it, when you want it?

“That’s really the premise that we started that I think is more empowering to the deaf and hard of hearing people,” Thibault continues, “because you can really master your own autonomy of the accessibility.

“If you start with autonomy, and then you allow for accessibility, then I think it’s kind of a shift in the paradigm that probably has more interesting results.”

Speaking of results, our conversation soon turns to accuracy. With voice recognition apps and software gaining a lot more attention in the era of COVID-19, the industry has certainly seen increased competition in recent months – each company battling it out to have the highest efficiency.

“First, it has to be designed to improve and to improve smartly,” explains Thibault, when asked about the accuracy rate of Ava. “A lot of companies kind of take whatever is available, but they don’t have a say in making it better. So I think that’s why, maybe over the years, you’ve seen Ava start small. We wanted to have that independence and improving to a level now that I think is best in class.

However, he doesn’t choose to compete on accuracy. “It’s really down to a level where, at some point it becomes accurate enough that it’s helpful,” he says. “What I want to focus on is understanding.

“The problem is that sometimes for casual conversation, you have the big information, and another person. So understanding is enough to be at 95%, so I would place Ava at around [that].”

The figure is backed up on Ava’s website, with an individual Community plan – priced at $14.95 per month – allowing users access to three hours of premium quality captions, which has five errors in every 100 words.

This increases to 10 errors with the Free plan, which only allows for sessions up to 10 minutes, compared to 40-minute recordings on the Community subscription.

Although accuracy may not be something which is used by Ava to compare themselves to competitors, the software still offers way to improve it. In a discussion around group conversations in restaurants, Thibault tells me another user can be invited to join a recording as a “second ear”, which can help cancel out background noise.

There is also the option to call in a human ‘Scribe’ to improve captions, too. With 10 Scribes available to help, Ava says they can be available within 30 seconds.

“We’re using AI, plus human. So together, not just against humans,” Thibault explains. “That’s really, really important. That means that the AI is doing some of the best of both worlds.

“By being efficient, you can get the first version of the captions right away, and then the human is correcting optimising for understanding.”

Yet with British Sign Language interpreters already raising concerns around the normalisation of Video Relay Services (VRS) post-coronavirus, it could well be that UK palantypists also express their fear over a stronger move towards automatic, live transcription in the future.

When asked about the potential concern over more live transcription software posing a risk to palantypists, Thibault says: “The way we think about professional captions is to have always in human professional, that can make sure the standard of quality is respected. Think of it as human-assisted captions.”

He continues to explain that Ava Scribe almost proposes a new job which is a synthesis between a palantypist and AI. “That transition needs to happen, so that this job this profession can be saved,” he says, “because the problem is happening that AI is becoming so good and so interesting for people. Now you’re seeing – even in the US – people using AI for TV captioning, which is crazy, because it’s obviously of a less quality, but it’s so cheap and you can use it 24/7.”

He finishes the call by sending me a link to try it out myself. I soon download the software onto my laptop and create an account, both of which are very straightforward processes (you can either register with your email, or login via Facebook or Google). Starting the transcription is also simplistic, with captioning available at the click of a button. Finding out if it’s on or not, however, can be a little bit difficult to begin with.

My first experience with Ava is on a call to my girlfriend on FaceTime, and while we’re able to chat freely between each other, Ava presents a one-way conversation. Instead of transcribing both of us, only my speech was registered. At first, I feared it was only able to pick up sound coming through my microphone, and not what is coming through the speakers, too. It may also have been an issue with regional accents, as well.

In terms of the transcription itself, the level of accuracy does align itself fairly close with other tools out there. The AI and speech recognition fails to understand context, and so inserts words completely unrelated to other parts of the sentence. With that being said, though, the ability to go in and edit captions afterwards – which I’m told actually helps the algorithm – is a promising feature.

Yet there is something else worth flagging about Ava, particularly following the criticism YouTube has faced for doing something similar with its automatic captions. Much like the social media platform replacing swear words with ‘[__]’, Ava’s software censors curse words uttered into its systems. It’s understandable to say that it wouldn’t be appropriate to use in family friendly settings, but as those challenging YouTube’s decision have argued, deaf adults can handle an expletive.

However, over time, I have come to accept technology designed to improve the lives of deaf people like me if it’s shown to benefit at least one person from our community. In this case, with Ava having translation capabilities, allowing for multiple people to access the same transcript and more, there’s no doubt that it will certainly have its uses.

Not only that, but in an industry dominated by smartphone transcription apps, the launch of another piece of desktop software is particularly refreshing. Deaf people deserve to have the freedom of choice with accessible technology, after all.

By Liam O’Dell. Liam is a mildly deaf freelance journalist and campaigner from Bedfordshire. He wears bilateral hearing aids and can be found talking about disability, theatre, politics and more on Twitter and on his website.