# Take[] and other Mathematica functions with live streams

I have a live video feed that I would like to convert into raw numbers that could be used for a variety of different things. I would also like to do simple operations on these numbers live with as little latency as possible.

So essentially, I get a Dynamic matrix of numbers that, are in byte form (0 to 255). To make it easier, I greyscaled it, so that there is only one data point per pixel. Assuming the matrix is 360 X 240, whenever I try to Take[] from this dynamic matrix, I get an error message.

How do I use specified pieces of the matrix to define some output on this data? In other words, lets say I want to cut the video up into 18X12 blocks so that I can operate on just just that section of the live matrix video feed, how can I use Take[] to do this? I assume there is some function that is not Take[] for this.

For my specific project, I need to divide each pixel in this raw matrix data by 2 so that the data fits in the parameters 0 to 127 instead of 0 to 255, what is the most efficient way to calculate this live stream so that there is as little latency as possible? Would I use TrackSymbols? Or is there some other way to operate on a dynamic matrix?

Any help on solutions to these two problems is greatly appreciated. And a warm smile goes to you for helping out a lone wolf amateur programmer like myself.

EDIT:

Sample data can be generated from any webcam with the following code, thanks to help from Andy:

img = Dynamic[ColorConvert[CurrentImage[], "Grayscale"]];
rawimg = Dynamic[Flatten[ImageData[img[], "Byte"][[1 ;; 18, 1 ;; 12]]]]


I then want to turn this information into one DYNAMIC digit that is between 0 and 126. by dynamically taking the Mean of all the numbers in the list and dividing this number by two (and using the floor of the answer). So the input is a large matrix and the output is just a single number like "122".

• Could you provide any sample data, and desired output? – Eli Lansey Feb 15 '12 at 15:58
• Mark, please use answers only for answering questions, and not for commenting on answers. That's what the comments are for. – J. M. will be back soon Feb 15 '12 at 23:37

The question doesn't provide enough specifics to give a very detailed answer but here are a couple of things that might help get you started.

First, here I use my webcam's CurrentImage with ColorConvert to create a live grayscale video. Note the semicolon. If you are going to be doing a lot of dynamic video stuff the additional outputs will slow things down.

img = Dynamic@ColorConvert[CurrentImage[], "Grayscale"];


There are a whole host of image processing capabilities that I would try first before digging into the matrix data themselves. Chances are good, you will find what you want there without having to reinvent the wheel. Here I use ImagePartition to break the image live video into blocks 64x64 pixels. Note the use of Dynamic. This ensures that this updates rather than grabbing the value of img at evaluation.

Dynamic[ImagePartition[img[], 64]]


Now if you don't find what you need in image processing you can still get the image data. Here I obtain the first 18x12 matrix of color values using ImageData and Part.

Dynamic[ImageData[img[]][[1 ;; 18, 1 ;; 12]]]


Some notes:

One reason you might have been having difficulty is that the head of img is Dynamic not Image. This is why I take img[] every time I use it.

In:= Head[img]

Out= Dynamic

Out= Image


Also, you want to make sure you save very often and that you are familiar with the menu command Evaluation>Dynamic Updating Enabled. Working with live video will cause you a number of headaches and you don't want to lose hard work.'

Edit:

In response to the answer to your own question. First off, you've placed "Byte" in the wrong place. It should be an argument to ImageData. Once you've fixed that it should just work. Here is my interpretation of the 1/2 byte data for the first 18x12 submatrix.

Dynamic[Floor[ImageData[img[], "Byte"][[1 ;; 18, 1 ;; 12]]/2]]


Edit 2:

Now if you want the Mean of this dynamic byte data we could use

rawimg = Dynamic[Floor[ImageData[img[], "Byte"][[1 ;; 18, 1 ;; 12]]/2]];

Dynamic[Mean[rawimg[]]]

• Thank You Andy, Your post is helpful as it introduced me to "ImagePartition" and it did not occur to me to partition the image before trying to get half-bytes. The problem is still not solved, however, since I'm still getting errors when trying to convert to "Byte" img = Dynamic@ColorConvert[CurrentImage[], "Grayscale"]; Dynamic[ImageData[img[]][[1 ;; 18, 1 ;; 12]], "Byte"] Does not produce Bytes. And remember, I actually need half Bytes. Dynamic[ImagePartition[ImageData[img[], Scaled[{1/20, 1/20}]], "Byte"]] Also does not produce Bytes. Dynamic[ImageData[ImagePartition[img[], Scaled[ – J Mark Inman Feb 15 '12 at 18:28
• You put the "Byte" in the wrong place. It should be inside the brackets for ImageData. Try something like ImageData[img[],"Byte"][[1 ;; 18, 1 ;; 12]]] instead – Heike Feb 15 '12 at 18:43
• Ok, I'm very close to a solution with this help. The final step on this problem is to be able to operate on these numbers. Let's say I wanted the Mean of this Part and then divide it by 2 so that I'm within the data parameters. When I try to do this, I get the following: img = Dynamic[ColorConvert[CurrentImage[], "Grayscale"]]; rawimg = Dynamic[Flatten[ImageData[img[], "Byte"][[1 ;; 18, 1 ;; 12]]]] in: Dynamic[Mean[rawimage]] out: Mean[{122, 124, 144... }] – J Mark Inman Feb 15 '12 at 19:15
• Ok, I'm very close to a solution with this help. The final step on this problem is to be able to operate on these numbers. Let's say I wanted the Mean of this Part and then divide it by 2 so that I'm within the data parameters. When I try to do this, I get the following: in: img = Dynamic[ColorConvert[CurrentImage[], "Grayscale"]]; rawimg = Dynamic[Flatten[ImageData[img[], "Byte"][[1 ;; 18, 1 ;; 12]]]] in: Dynamic[Mean[rawimg]] out: Mean[{242, 243, 242... }] I also tried: Dynamic[Total[rawimg]] and got the following: {242 + (ImageSizeCache -> {531.701, 198., 9.}}, {243 + (ImageSize – J Mark Inman Feb 15 '12 at 19:26
• Your rawimage is a Dynamic object similar to img, so you probably need to take the first part again, i.e. do something like Mean[rawimage[]]. – Heike Feb 15 '12 at 19:28