Do bad lip reading experiences exist?

I often get asked “What is forensic lip reading?”.

When a lip reader is lip reading, they are usually doing this in real life, face to face, watching a person’s lips, facial expressions, eyes, gestures, body language, and using context to clue themselves into the topic, so that they can understand another person. Many factors affect the lipreadability of a person so the outcome is not usually a word-for-word translation. If enough of the factors affecting lip reading are present in a meaningful enough way, it all comes together beautifully to make sense to the skilled lip reader.

Forensic lip reading (or speech reading) is the transition of the skill of lip reading from real life to media – lip reading CCTV or video clips. Reading lips in 2D is much harder than reading lips in real life 3D because so many clues are missing and the medium is much tricker. Have you ever tried to lipread a flat face? This makes for a more difficult or even a bad lip reading experience for the lip reader. So many people assume, just because our lip readers can read lips, we can lip read anything you send us. We get so many videos that are #FAILs when we see them. So how can we make this a better experience for the lip reader?

There are a number of issues which affect forensic lip reading work. Pay attention to these factors and you are more likely to understand what’s possible and what isn’t.

Video quality

Broadcast quality is much easier to work with than footage from a home video camera which is likely to be blurry, shaky and grainy. We get a lot of videos from surveillance agencies and the quality varies widely. A CCTV video came in last week which was the best quality we had ever seen, however the people speaking were too far away from the camera – which meant a bad lip reading experience for us! We are sometimes sent videos which are so blurry we can’t even make out the speakers faces – there is nothing we can do with these.

Camera angle

Ideally the speaker should be facing the camera full-on but we all know real life isn’t like that. Try to ensure you have a sensible camera angle if you are using hand-held cameras. We have been sent CCTV footage with the cameras pointing downwards from a great height – all we could see were the tops of peoples heads!

The speakers

The people being lip read should be as close to the camera as possible. We can’t slap on binoculars to read lips measuring 3 millimetres as the owners are standing so far away from the camera – the camera is fixed, we are looking at your video in post-production, not real life.

Lots of people mumble! Even the best lip reader will struggle to clarify what is being said when lips don’t form the shapes they should. The speaker may have a lisp or stutter.

If the speaker does not have a very expressive face, this makes lipreading them much more difficult. Sven-Göran Eriksson is an example of this.

Facial hair can be problematic, even more so when you are lip reading a tiny face on a video.

Sunglasses remove emotion from the context and make the job much harder.

The only people who really talk directly to video cameras are newsreaders and TV presenters. In real life, people move, they walk around, they look away – so typically a lip reader’s transcript will have gaps where the speaker wasn’t looking at the camera. One memorable job was a CCTV clip of two people having a conversation, the camera was only a few inches away – unfortunately both people had their backs to the camera!

People have accents and talk fast. This is a challenge even for a skilled lip reader, necessitating more rewinding to catch those milliseconds where a whole word had been said. Could it be a crucial word for a court case? The lip reader has to build up the gist of the conversation from all the bits they are analysing, discover the context, and then fill in any missing words that match what they can see. This takes great skill.

Lighting

Try as hard as we might, we really really can’t lip read in the dark.

Give us a clue

Context, context, context! Context is so important and is a great help at clueing the lip reader into the topic. However, if your video is destined for a court appearance, please make sure you do not send us any information about the case.

Client expectations

When you want a rush job, we need to manage your expectations.

A 30 minute video will not take 30 minutes to lip read! It can take an average of 1 hour to view, interpret and transcribe each minute of visible speech footage, as the lip reader has to lip read in 2D and rewind the video several times to pick up more clues each time and mentally work out and make sense of what is seen.

A lip reader needs frequent breaks as forensic lip reading is extremely tiring.

They are also doing other work, so their workflow needs to be managed.

Also, and this is from my personal experience, I can’t work while being emailed every 2 minutes by a journalist “Have you finished yet? What have you got?” – yes, this really happens!

Your lip reader

A lip reader may not be familiar with certain accents or industry specific jargon. A knowledge of context and content is always very helpful. Some lip readers say they can lip read anything, they can lip read upside down and in any language, which is very amusing as it’s just not possible.

A lip reader should only lip read in the language they are fluent in. If they work with a translator in lip reading a language they don’t know, the risk of getting it wrong becomes far too high. The lip reader needs to be able to work out the possible homophenes in the foreign language so they can quickly see which words are matches for what is seen.

A homophene is a set of words that sound different but look alike on a person’s lips, so they cannot be distinguished by lip reading. Have a look in the mirror and say the following words without using your voice:

PAT     –     BAT     –      MAT

P.S. The words DON’T come backwards in the mirror!

You will see that the beginning of each word looks alike in your mirror. When lip reading, the only way to sort out which word was intended is by knowing the context.

Try this pair: BABIES, BAKED BEANS. Babies on toast, anyone?

In another language, you would need to know, when mentally sorting out the words to grasp the context, which visemes and which words look like. Once you know that, you can sort out the context and make sense of what you can see. This is why working with a translator will not work well.

How do you know if a lip reader is good at lip reading? Use your common sense: speak to them over Skype video or meet them face to face, and switch off your voice. If they can lip read you – ergo, they can read lips. If they can’t, and they require sign language support to understand you and slow speech, or they are listening, they clearly aren’t competent lip readers. A lip reader can ask for a Bamford-Kowal-Bench (BKB) test from their audiologist which assesses their ability to lip read. Don’t forget, showing a lip reading ability in the test booth is quite different from exercising the ability to lip read in the hearing world, which is much harder. There is no qualification available for lip readers; just ask to speak to them, speak normally, and see for yourself. I scare my own audiologist – she knows I can lip read her from behind, I only need to see one corner of her mouth and that’s it!

Our lip readers are hand picked by our team and assessed by me. They are the best forensic lip readers you will find, able to hold their own with hearing people and therefore they manage the transition well to lip reading people on videos.

You may have heard people state that lip reading is only 30% accurate.

This is misleading.

The fact is, around 30% of lip shapes are visible on the lips, the rest are not. They are inferred from clues the lip reader picks up – context, facial expression etc.

When I lip read someone, I get 100% of what they say. Not 30%.

If I am tired or stressed, or if I am distracted, my concentration is lower and I may have to ask you to repeat.

Think about it. I can’t hear and rely on lipreading to communicate. If lip reading is only 30% accurate, how the heck would I understand anyone at all?

Whilst a less than ideal video can mean a bad lip reading experience for us, you can help to make it less painful! When you’re requesting lip reading services from us, we’d be very happy to get detailed information such as:

• The context of the video (if not headed for the courts).
• Who is on the footage and which person(s) you’d like the lip reader to focus on.
• Is there a particular moment in the video that interests you? Give us the relevant timestamps.
• Which language is the speaker using, if you know this.
• Your deadline.
• Is an expert witness forensic report required?

You’re probably wondering why there are photos of snowboarding in Chamonix on this blog post. They are from our lip reader James. He’s lip reading as he skis.

Now I am pulling your leg… James sits at a computer and screws his eyes up like everyone else. He happens to be in Chamonix this week and we’re still sending him work – and bad lip reading videos to evaluate – and we’re still on speaking terms!

Our lip readers can work anywhere. All you need to do is contact us and send your video by WeTransfer, Dropbox or FTP link. We look forward to meeting your lip reading requirements.