Quantitative video analysis

From FourMs

Jump to: navigation, search

This page documents some techniques that are useful if you want to carry out quantitative analysis of musical material. Please also refer to the introduction to qualitative video analysis for more information.

Contents

Tools

There are several tools that allow for carrying out quantitative video analysis, but many of these require that you develop the patches yourself:

  • Eyesweb is a Windows-based platform for (mainly) video analysis and synthesis. This is a development environment, so you will need to learn how to create your own patches for this to work.
  • Max/MSP/Jitter is a commercial Mac/Win development platform for realtime music and video applications.
  • Matlab is a commercial programming language where the user can write text-based scripts to run.

VideoAnalysis

We have developed a standalone application called VideoAnalysis which allows for simple analysis on a set of video files. The application runs on both Windows and OSX computers and can import all files accessible from QuickTime.

  • Open the application by double clicking the application (OSX) or the exe-file (Windows).
  • Drag and drop a folder with video files in the available space in the upper left side of the application. It is not possible to only import a single file, then you will need to place it in a folder first.
  • Choose whether you want to export a motiongram along with your data.
  • Click in the box next to "Export to data" to start the analysis.
  • The result of the analysis is written to the same folder as where you started the application from (sorry about that, we will change it in the near future).

The data generated by VideoAnalysis

The data is exported to textfiles, named after the video files. The text files have 19 columns, where the first column displays time, the contents of the next 9 columns are data with an applied medianfilter, and the last nine columns are unfiltered data.

Column 1: Time stamp

This is the QuickTime timecode, which is a frame-independent timecode. Each row in the text file contains data for a single video frame.

Column 2: X position

The horizontal position of the center of gravity. X = 0 means that the center of gravity is located to the left in the picture

Column 3: Y position

The vertical position of the center of gravity. Y = 0 means that the center of gravity is located in the bottom of the picture

Column 4: X velocity

The first derivative of X position. The difference between the X[n] and X[n-1], where X is the X position (in pixels) and n is the sample index. X velocity > 0 means that the center of gravity moves to the right.

Column 5: Y velocity

The first derivative of Y position. The difference between the Y[n] and Y[n-1], where Y is the Y position (in pixels) and n is the sample index. Y velocity > 0 means that the center of gravity moves upwards.

Column 6: Absolute velocity

The first derivative of position. The distance component of the polar representation of X and Y velocities. Absolute velocity = √(Xvelocity^2 + Yvelocity^2).

Column 7: Direction

This is the direction of the movement of the center of gravity from the previous frame to current frame.
Direction = 0 means a movement to the right
Direction = pi/2 means an upwards movement
Direction = -pi og pi means a movement to the left
Direction = -pi/2 means a downwards movement

Column 8: Absolute acceleration

The second derivative of position, i.e. first derivative of absolute velocity. The difference between V[n] and V[n-1] where V is absolute velocity, and n is the sample index. This implies that if the absolute acceleration is 0, then the absolute velocity is equal in two succeding frames. If absolute acceleration is greater than 0, the speed increases.

Column 9: Change in direction

This value is meant to show the angular velocity of the center of gravity. This is not properly implemented yet, as there is no compensation for the flipping between direction = pi and direction = -pi.

Column 10: Quantity of motion (QoM)

A normalized value for the total movement in the current frame. QoM = 1 means that all the pixels in the video change from one picture to another, QoM = 0 means that there is no movement in the picture since the previous frame.

Columns 11 through 19 are the unfiltered equivalents of columns 2 through 10.

Todo

  • Translate to English
  • Evaluate the Rotation and Orientation representation. Perhaps use sin(angle) instead of the angle values to avoid flipping between -pi and pi.
  • Implement user-selectable destination folder
  • Implement single-file import
  • Correct mac/windows differences (e.g. ctrl-click vs cmd-click)
  • Complete the documentation of the extracted data
  • Correct the text file time tag to show frame number

Plotting

MS Excel

To import the text file with data:

  • File -> Open: then select the file
  • Original data type: choose "Delimited"
  • Delimiters: choose "Space"
  • If you are using an OS with settings other than UK/US you probably use comma as decimal separator. Then you should click on the "Advanced" button and choose to use . (dot) as the decimal separator. Otherwise your data will look like text strings and not numbers.

There are lots of ways of working with the data in a spreadsheet, and this is beyond this short guide. Often the first thing to do is to start by visualising the data. There are two types of displays that are useful for looking at movement data: time-series and XY-plots.

For more info about plotting you can refer to Excel's help file, or take a look at one of many online tutorials.

OpenOffice

  • Open OpenOffice and select Spreadsheet
  • The text files exported from VideoAnalysis use the English-language convention of using the dot (.) as decimal separator. This may cause problems on Norwegian (and other non-English systems), where comma (,) is often used as decimal separator. To circumvent this go to Options -> Language and choose English as "locale setting". (On OSX you find Options under OpenOffice, while it is under Tools on Windows).
  • Open file (File -> Open)
  • Remember to choose file type "Text CSV"
  • Choose delimiter "Space" and then click on Finish.

Plotting in OpenOffice is as easy as in Excel. Start by selecting the row(s) you want to plot, and then click on the Chart button (or use the menu). Then select the type of plot you want to create.

SPSS

SPSS is a statistical package with lots of useful tools for plotting and statistical analysis.

Matlab

Matlab is probably the most advanced tool for plotting scientific data, but it also has a steep learning curve.

Personal tools