MultiTouch Video Manipulation » Gestures

Abstraction

jimmy — Fri, 03 Apr 2009 12:31:24 +0000

Alright, it’s been a while since my last post. We have 2 other big projects running and just had an exam so I have an excuse .
When I was implementing my own gesture recognition, it worked quiet well with sending test data. But once i tried it on the touch table it wasn’t working as I thought it would.
The cause for this was that I used 4 fingers for every gesture, because 1 finger was reserved for translating the video en 2 fingers were reserved for scaling and rotating.

After a little brainstorming with my counsellors, the next attempt was to make abstraction of the input. This gave me the opportunity to implement a click event and also do some preprocessing of the input.
The problem of the 4 fingers I resolved by an area on bottom right of the video player. This area kan be used to rotate and scale the video player with only 1 finger, so I can make gestures using only 2 fingers.

The next problem was that most people won’t use only 1 finger to manipulate the video player but their entire hand. This i’m trying to resolve by clustering inputpoints before I send out the events.

The last problem I noticed lately was that sometimes the touch-table doesn’t send out the coordinates of 1 point for 1 frame. This causes a release and press event when these events shouldn’t be triggered. I have an idea to solve this by keeping track of the timestamps en position of each point so that I can keep the input consistent, but more about this in the next post.

Progress

jimmy — Sat, 28 Feb 2009 21:07:19 +0000

Like I said last time, I implemented my own gesture recognition.
It’s still a work in progress but it’s coming along nice.

I also upgraded the layout of the application so it looks more like a wpf application.
You can see the result below.

Screenshot

Screenshot with border

Gesture recognition

jimmy — Tue, 10 Feb 2009 17:54:21 +0000

I have done some more tests since last time. First I’ve implemented 2 video methods of viewing a video that prof Luyten advised.

Next i tried to implement gesture recognition to trigger these views. I used the $1 Gesture recognizer to recognize the gestures. This recognizer is actually built for single touch en recognition after the gesture is finished. So I tried to use this for multi-touch gestures, and used a threshold to solve the real-time issue. When I’m sending my own generated data, this works pretty nice, but when I tried it on the multi-touch table, it only recognized 1/3 of the gestures. Obviously this method is not very useful.

My next attempt to solve this problem is to implement the gesture recognition myself, using the amount of input points and the direction each point moves to. Next time I will be able to tell if this works better to recognize the gestures.

Finally, I have been able to extract frames from a movie using only the WPF framework, so for now i don’t need an external library or application ( like ffmpeg ) to do this.