Gaze, Facial Recognition - A More Intuitive Computing Experience

[QDaMan]

http://www.macrumors.com/2008/09/04/beyond-multi-touch-voice-gaze-facial-expression-recognition/

I thought this was a really cool patent filing on Apple's part. Could you imagine your Mac realizing your frustrated and putting on a smooth jazz playlist and popping up help dialogues?

The implication for more specific multi-touch gestures is cool too. Maybe they could detect not just touching, but SEE what the fingers are doing on the Z-axis, allowing for 3d gestures!

 

[the8thark]

Oh this is very interesting.

This is why I liken Apple to Nintendo and Sony to Microsoft and the other PC makers. Niintendo and Apple don't have the most powerful machines on the market. Fact. But everyone lives them for their ease of use and revolutionary methods of interacting with them.

And I think it's a good idea. Because there's going to come a time where PC/Microsoft/Sony's idea of faster, better, even more hi def will fail. Sure it can get infinitely better. But there'll be a point where the graphics and sound are so good that the eye will not be able to detect the minor improvements in there. We'll have reached the limits of what our biology can experience. Sure a slick ad campaign could still sell machines but the smarter of us will see right through.

But Nintendo with the DS/Wii and Apple with this new patent want to enrich the experience by expanding the ways we can interact with things.

And on a side note I think this is the first step towards real AI. As in (referring to the points mentioned in the article and adding my own comments on each):

Voice: I see that leading to improvements in artificial speech. And making it possible to have a machine that can hold down a real conversation. Each word would be like a mouse click in an app. And activate a different function of the app running inside the machine.

Finger identification - Helping out multi touch. Would help in the 3D sensing of cameras. So they could tell how big small something is. And how far away it is. Sure all this is just data to a machine. But all this data when interpreted by the machine could be one day be used in the app that powers in the future a robotic eye.

Gaze - To me this is a point recognition and tracking system. The machine would know exactly what it is looking for, say the pupils from a person's eye. And it would be constantly scanning the area for this. And when it finds it, it could follow this point of data it's just created with the camera. The military anyone? Make a big massive automated gun. Use this and set the gaze point to the enemy and to fire on it when it's in range and it could track it too. I'm sure it could be used in other ways where you need to remotely trahh something and follow it but it's too dangerous difficult for people to do. The only issue here I'd have if say two people's eyes came into the camera's view. And always scanning for the eyes potentially could be a big resource drain on the machine. Ah but these problems will surely fixed in the years to come.

Facial expression - Already exists in a way. Robots do exist that can give facial expression based on certain preprogrammed events. Say for example it feeling a certian amount of force (or being pushed) could trigger a sad face. Oh that'd be so weird getting mad at your mac for it not able to do what you want. And sensing this with the Gaze mentioned above and telling you "Do you need any help". You say yes with using the "voice" mentioned above and it'd bring up the help file for the foremost open application automatically.

Still not true AI. Far from it. But it's a step in that direction.

And sorry for taking over the topic with a long blurb. I'll return it to you now. Hope that wasn't too boring or uninformed.

 

[QDaMan]

Oh not long or boring at all that was kind of the direction I thought it would go.

I also agree that these components would definitely integrate well with a proto-AI type interface. This could revolutionize the way we use computers or at least, as is Apple's specialty, make them more intuitive and easy to use. These technologies kind of remind me of those marvelous advancements you always see on the Discovery Channel but never hear about again, except this has a possible reality in the future.

An interesting aside to this is will voice commands ever replace typical input such as a keyboard and mouse? My prognosis: no, at least not on your everyday computer. The only thing that could replace a keyboard and mouse would be a brain implant capable of performing the same functions. A keyboard and simply writing things down allows us all to be more articulate and organize our thoughts, something that would be very hard to do with just speaking. As far as voice augmenting a keyboard/mouse however, I think that will eventually be a normal part of everyday computing.