How many times have we read about amazing technology that provides automatic translation from English into sign language or automatic translation from sign language into English?
Watch this article in BSL below (or scroll down to continue in English!).
“Sign-language gloves” are described as translating sign language into English by sensing the user’s movements. The prize-winning car system Sym:Pony “can even detect sign language and read it out so the driver can stop and ask for directions. The car will translate the answer back into onscreen sign language.”
Angus Grieve-Smith, a linguist and computer programmer, has provided 10 reasons why automated translation between signed and spoken/written language is so difficult.
Although there have been some recent advances in sign language recognition, part of the problem is that most computer scientists in this research area do not have the required in-depth knowledge of sign language, and often have no connection with the Deaf community or sign linguists.
For example, one project described as translation into sign language aimed to take subtitles and turn them into fingerspelling. This is one of many reasons why much of this technology, including sign-language gloves, simply doesn’t help deaf people.
We might decide to abandon any attempts to achieve automated translation between sign languages like BSL and spoken/written languages like English.
But the ability to undertake automated translation has many benefits. For example, there are enormous amounts of video data on-line in BSL and other sign languages that are not searchable.
So even for Deaf and hearing signers, finding specific content requires watching the whole video. Studying sign languages involves analysing video footage – and this is extremely labour-intensive. Wouldn’t it be nice to be able to quickly search signed videos and find the content you’re looking for, the same way we do with English texts?
Until recently we have lacked large signed video datasets that have been precisely and consistently transcribed and translated – these are needed to train computers for automation. But sign language corpora – large datasets like the British Sign Language Corpus – bring new possibilities for this technology.
What is needed is a strategic collaboration between BSL linguists, the Deaf community, and software engineers who specialise in computer vision and machine learning.
The Engineering and Physical Sciences Research Council has recently funded just such a project – ExTOL: End to End Translation of British Sign Language) – a collaboration between the University of Surrey, University of Oxford, and DCAL (the Deafness Cognition and Language Research Centre at University College London), with the aim of building the world’s first British Sign Language to English Translation system and the first practically functional machine translation system for any sign language.
The work already done by the researchers at DCAL on transcription and analysis of the BSL Corpus will provide the essential data to be used by computer vision tools to assist with video analysis.
This will in turn help linguists increase their knowledge of the language with a long term ambition of creating the world’s first machine readable dataset of a sign language.
To achieve this the computer must be able to recognise not only the shape, motion and location of the hands but also facial expression, mouth movements, and body posture of the signer.
It must also understand how all of this activity in connected signing can be translated into written/spoken language.
The technology for recognising hand, face and body posiitons and movements is improving all the time, and we know we can make significant progress in speeding up automatic recognition and identification of these elements (e.g. recognising when someone is using their right or left hand, or recognising specific facial expressions or mouth movements).
Full translation from BSL to English is of course more complex but the automatic recognition of basic position and movements will help a lot.
The ultimate goal that this project is working towards is to take the annotated data and understanding from linguistic study and to use this to build a system that is capable of watching a human signing and turning this into written English.
This will be a world first and will be one of the first practically useful applications of this technology, based on the principle that sign language technology will only ever progress into something useful when sign linguists and Deaf signers are actively involved.
Check out the D/deaf or hard of hearing screenings + BSL interpreted introductions and Q&As at this year's The 62nd BFI London Film Festival in partnership with American Express® running until this Sunday!
Find out more at bfi.org.uk/access
Are you a BSL interpreter or otherwise interested in BSL?
Please join us on Saturday 3 Nov 2018 to celebrate ten years of the BSL Corpus and BSL Signbank!
More information here: https://bslcorpusproject.org/
The Limping Chicken is the world's most popular Deaf blog, and is edited by Deaf journalist and filmmaker Charlie Swinbourne.
The site exists thanks to our supporters. Check them out below:
- Signature: Leading awarding body for BSL qualifications
- Ai-Media: Remote captioning. Read tips for being Deaf-aware!
- Bellman & Symfon: home alerting solutions
- Deaf Umbrella: sign language interpreting and communications support. Find out more about Deaf Awareness Training Courses!
- BSL Zone: TV programmes in BSL for the Deaf community
- deafPLUS: BSL advice helpline
- Appa: Communication services for Deaf, Deafblind and hard of hearing people
- SignLive: Online video interpreting for Deaf people
- SignVideo: Instant BSL video interpreting online
- 121 Captions: captioning and speech-to-text services
- The National Theatre: Captioned and BSL accessible theatre in London
- Doncaster School for the Deaf: education for Deaf children
- Signworld: Learn BSL online!
- Association of Notetaking Professionals: The professional body representing Electronic and Manual Notetakers
- CL Legal – experienced industrial deafness solicitors
- Eyewitness Media: TV and film from a Deaf perspective
- Sign Solutions: communication support, training and translation
- InterpretersLive: On demand BSL video interpretation
- Hamilton Lodge School in Brighton: education for Deaf children
- Lipspeaker UK: specialist lipspeaking support
- Elmfield School, Bristol: Inclusive education for Deaf pupils
- Exeter Deaf Academy: education for Deaf children
- Helen Foulkes Translations: BSL translations
- RAD Tax Advice: Tax and Tax Credit info for Deaf people
- Performance Interpreting: BSL interpreting at concerts
- National Deaf Children's Society: The leading charity for deaf children
- Signed Culture: Advocating for BSL access to arts and culture
- SignHealth: healthcare charity for Deaf people
- CJ Interpreting: communication support in BSL
- British Society for Mental Health and Deafness: Promoting positive mental health for deaf people
- Action Deafness Communications: sign language and Red Dot online video interpreting
- BSLcourses.co.uk: Provider of online BSL courses