Michael Heuberger: Why I created a web form to send emails in sign language

Have you ever seen a web contact form where you can ask in Sign Language right away and send that? Or an email with a business proposal in Sign Language? No, there is no such thing.

This is a huge disadvantage for Deaf people. Many of us hate it to type when we can express ourselves much better in Sign Language.

Modern technologies still do not embrace Sign Language well enough. There is no doubt that web accessibility is poor when it comes to Sign Language. As we all know the classic problem, education for Deaf people is still lacking behind, many Deaf are often misunderstood in any sort of written communication.

It even gets worse for us when you take into account that today, in 2017, email is the most important communication medium everywhere, for businesses, education, research, anywhere. Just not for Deaf people.

Furthermore, if you agree that a language evolves and spreads faster when this is supported with technologies, then we also can agree that Sign Language doesn’t spread that fast as it should. And here I am, a Deaf programmer, trying to help a bit with some “mad science” …

Four years ago, I made some kind of a beer bet with other Deaf computer programmers and thought it would be possible to solve this with web technologies alone. Without the use of Flash, iOS apps, mobile phones, nor complicated external components. Just plain web-based stuff.

Thanks to my 23 years of hacking experience and a creative idea, I built the first prototype https://www.videomail.io/ in October 2014.

The most challenging part was how to encode a video on a webpage. This is very tricky because video encoders take at least 100 MB of space which is too large for a website.

Lots of work around web sockets was needed including mathematical calculations. After many years and lots of feedback, the prototype underwent many changes and improvements. Now, in March 2017 about 1160 videomail were sent on average each week which is quite impressive. If you haven’t checked it out yet, go ahead and type https://www.videomail.io/ in your browser now.

Some of you might ask how it works. Here’s a pretty technical sketch I drafted for a node.js meetup last year.

At first, when you load the page, your browser will ask for webcam permissions. Just click “Allow” and then you can press on the record button. A countdown of three seconds appears. When it’s zero, you can start signing anything up to three minutes.

Inside the videomail client, webcam pictures are fetched with the getUserMedia interface which sends image frames through fast web sockets to the node.js server. There ffmpeg encodes them into a video and pipes that back to the client for a preview. If the user doesn’t like it, a new video can be recorded over again.

On the right side of the preview you see a web form. There you can enter your email address and as many recipient email addresses you want. After that, enter a mandatory subject line (because all emails require a subject, don’t they?).

At last hit the send button and here comes the magic: through the SMTP email interface, recipients will receive a plain email with a beautifully embedded video inside.

It really looks like a normal email but in Sign Language! You can watch it anytime, anywhere and click on the reply button inside which will redirect you to the https://www.videomail.io/ page where you can sign your response right away. This never has been so easy before.

My plan is to let it run, to spread it around, to collect feedback and to constantly improve it further. It should remain free for Deaf people.

The client program code, the videomail client, is written in JavaScript, is open sourced, available on www.github.com/binarykitchen/videomail-client for anyone for reuse.

A WordPress plugin is coming very soon. That means, any website can integrate it in their contact form very easily. If you run a website targeted at Deaf people, why don’t you give it a try?

What my future plans are, I cannot say exactly. I always tell me friends, I have no expectations and just let the “baby” go and see what happens. When traffic ever explodes one day, then I have to think of new strategies and to step up from a hobby to a professional thing. Exciting times.

Check it out at: https://www.videomail.io/