# Creating an example of video image tracking

I've made a video of two teams playing soccer. Now I would like to translate each soccer-player to a (x,y) coordinate and follow them during the match. I know that Mathematica has some image feature tracking and am looking for a suggested workflow for image analysis and an example of how to apply it to a series of images.

Has anybody experience with video and image recognition? I've made a video of two teams playing soccer. Now I would like to translate each soccer-player to a (x,y) coordinate and follow them during the match.

• Have the players got LEDs attached to their bodies? – cormullion Nov 6 '13 at 16:23
• You could try ImageFeatureTrack. But if that doesn't work, prepare to do some serious research. Attaching LEDs to their clothes when they're in the shower might be easier ;-) – Niki Estner Nov 6 '13 at 16:45
• @nikie but electronics and water... a dangerous combination :P – Yves Klett Nov 6 '13 at 16:47
• @YvesKlett: On the other hand, when the players get a strong enough electric shock, they should be easier to track ;-) – Niki Estner Nov 6 '13 at 16:54
• I salute your ambition, but this topic is much too broad for this site. – Sjoerd C. de Vries Nov 6 '13 at 18:09

Well, I suspect this question will get closed because it is a bit broad, but I've played around with image tracking and thought I'd show what I've done in case it's helpful.

I was interested in learning some physics, and have the following video:

To do the tracking, I smoothed out the images and converted them to black and white, which allows for the colored balls to who up on a black background. Note that I have already defined the symbol images which was the series of graphics that made up the video.

imagesbw = Binarize[MedianFilter[Image[ImageData[#][[All,All,1]]],4]]&/@images;


We lose the black ball, which is unfortunate, but my "physics question" was to see if I could predict the movement of the missing black ball from the information that is available. I started by collecting the position of the three visible balls using ComponentMeasurements which in my case finds four balls, one of which is the one in the upper right hand corner that doesn't move, so I'll ignore it. I can then get tracking information from the ComponentMeasurements output. Note I did play around with ImageFeatureTrack, but I didn't have much luck with it.

movement = ComponentMeasurements[#, "BoundingDiskCenter"] & /@ imagesbw;
tracking =
Graphics@Riffle[{Red, Orange, Blue, Red},
Point[Cases[# /. movement, x_ /; Dimensions[x] == {2}]] & /@
Range[4]]


There's some messiness in the data, so the line above deletes some of the points. Now I want to predict where the 8-ball is heading from the information in the point plot above. I approach this problem by assuming conservation of momentum assuming an elastic collision, ignoring friction and acceleration. There's a reason why my name isn't bobthephysicist.

There's probably no real reason to go through my ugly code to solve this problem, especially since I did so by surfing around Mathematica.SE so the solutions are already here. At the end of the day, I get the following graphic, which is a pretty decent fit, although I will admit I did need to incorporate a fudge factor in to the solution. My assumptions, as well as my not knowing the weights of the four objects, makes this task a bit daunting.

So in summary, tracking objects in Mathematica is possible, but it's not for the faint of heart. I think the process is summarized in the following steps:

• Import video as individual frames, possibly "downsampling" the video to make the data set manageable.
• Manipulating the images to remove extraneous information (colors, stationary objects, etc.)
• Use some tool such as MorphologicalComponents to identify the information of interest to you.
• Transform the output of the previous step into a usable format

That's my 2 cents -

• very nice...learned a lot – ubpdqn Nov 7 '13 at 11:51
• Thanks. This exactly what I want to know. Can you write something about what you have done after reading the images and how you manipulated all the frames. – Michiel van Mens Nov 8 '13 at 8:22
• @MichielvanMens if you import the first gif in my answer via images=Import["http://i.stack.imgur.com/sJPxW.gif"] then the code I've written will produce all of the results save for the last. My 2nd bullet point about manipulating images (which is my imagesbw symbol) is going to vary significantly based on the application and your best bet would be to start with the Image Processing tutorial in Mathematica to see what might work for your purposes. – bobthechemist Nov 8 '13 at 12:29
• @MichielvanMens As for the code that I used to "solve" my physics problem; it is ugly and opaque, includes a fudge factor or two, and wouldn't really help this answer all that much. (Which is one way of saying I'd be embarrassed to post the code.) – bobthechemist Nov 8 '13 at 12:33
• @bobthechemist Do you think perhaps you could edit this question such that its scope is that of this existing answer? I would like to see this reopened. – Mr.Wizard Aug 15 '16 at 1:57

What you are trying to do is called object tracking, and it is an active research area in computer vision. There are many algorithms for detecting and tracking multiple moving objects. I don't know of any examples in Mathematica, but here is one in MATLAB using background subtraction and Kalman filters.