Extracting exact start time (in milliseconds/frames) from video file

[jbraun]

Hi,
I've searched around before posting, but I can't seem to find an answer.  I'm trying to take a directory full of video clips, and create a JSON data file with each file's exact start time and duration in Milliseconds or frames.  I would use this to place each clip on a timeline positioned exactly on the frame.  Then, if 2 cameras are shooting at the same time, theoretically, the clips from both cameras would be in sync. However, all of the clips that I tried (.MTS files from a few different cameras) show that "SubSecCreateDate" or "SubSecDateTimeOriginal" are undefined.

So I had an idea.  Using the -ee tag, I can get the DateTimeOriginal from each frame of the video.  So I could get a pretty good/frame accurate start time by taking the second of the first frame's DateTimeOriginal value and count how many frames it takes to change to the next second. Multiply the number of frames by the frame rate and it should be a pretty easy calculation.

So, has anyone done something like this?
-Jim
P.S. - Wow... what an amazing amount of research you've done! Amazing.

[Phil Harvey]

Hi Jim,

I don't recall anyone doing anything like this before, but you may run into problems because I don't think the MDPM metadata is stored for each frame in M2TS videos.  For samples I have here, it seems to be stored about twice per second.  But maybe your camera is different.

There could be timestamps for each frame, but if so ExifTool does not decode these.

- Phil

[jbraun]

Hi Phil,
I'm not sure exactly what -ee does, but I did a simple: exiftool -ee 00209.mts >209.txt.  The output included 210 records like this:

Date/Time Original              : 2010:03:26 19:57:01-05:00
Aperture Setting                : 4.0
Gain                            : 0 dB
Image Stabilization             : n/a
Exposure Time                   : 1/120
Make                            : Canon
Recording Mode                  : XP+

Exiftool reports a duration of 6.93 seconds, and there was 1 record for the first second, 30 each for the next 6, and 29 for the last second.  Pretty close.

I haven't done extensive testing, but I took another little file, 3.51 seconds long, and it returned :
03:56 - 15 records
03:57 - 29 records
03:58 - 30 records
03:59 - 31 records

So, it may not be perfect, but I would position this clip at 03:56:500 Milliseconds, and save a lot of time.

I tried it with another clip shot with a different Canon camera, but still AVCHD, and got 630 records for a 20.97 second video, which works out again, but it doesn't work with a clip shot with MP4 files shot on my android or downloaded from Youtube.

That's all the testing that I've done.  I would think that there are others that would want to know when a video starts with more granularity than just one second, so I'm game to keep testing.
Thank you
Jim

[Phil Harvey]

Hi Jim,

The -ee just extracts all of the MDPM metadata.  It is really up to the camera how often it writes these.  My point is that it is not necessarily tied to the frame rate.

- Phil

Edit: I just checked some of my samples.  Canon and Panasonic write a whole pile of MDPM records each second as you found.  Sony cameras seem to write only 2 per second.

[jbraun]

Thanks Phil,
Got it. Can I assume that MDPM records are loosely (or exactly) equal to a single frame of video?  If so, it's easy to calculate frame rate. But more importantly, if the average MDPM records per second is 30, and there are only 10 MDPM records written for the first second of video  in a video clip, then we could assume that if the reported "DateTimeOriginal" was  28 seconds, the actual/approximate time would be 28.66 Milliseconds... right?

Is there a better way to come up with a more accurate/granular start time for the clip?  Is my camera not reporting an MDPM code "0x0013 - Timecode" value, or is that "SubSecDateTimeOriginal"? Or, am I making this more difficult than it should be?

Using the -ee tag to dump all MDPM data for a clip just to get at the first few seconds worth of data doesn't seem to scale well, so, either I'm the village idiot, or I may be the only person around that wants to know this.. I did see one post where a person wanted to rename a file with the millisecond notation, since even still cameras are able to take more than one picture per second...

Thanks for all of your help and hard work.
Jim

[Phil Harvey]

Hi Jim,

if the average MDPM records per second is 30, and there are only 10 MDPM records written for the first second of video  in a video clip, then we could assume that if the reported "DateTimeOriginal" was  28 seconds, the actual/approximate time would be 28.66 Milliseconds... right?

I agree with your calculation, but can't say how accurate it is.

Is there a better way to come up with a more accurate/granular start time for the clip?  Is my camera not reporting an MDPM code "0x0013 - Timecode" value, or is that "SubSecDateTimeOriginal"? Or, am I making this more difficult than it should be?

The TimeCode tag would be ideal, but if you don't see this in the ExifTool output then it doesn't exist.  Of my samples, only the Panasonic AG-AF100 writes this tag. (But I only have M2TS samples from 50 or so cameras.)

I can't think of any better way to do it.

- Phil

[jbraun]

OK, Thanks.
Quickly... from your experience, if someone would help me write some code to parse MDPM data to figure out a better start time, do most camcorders post MDPM data in a similar way that I'm seeing?

Jim
(any chance of getting a -ee tag with x seconds of data from the start? )

[Phil Harvey]

Hi Jim,

Are you really sure you want to do that?  This stuff is a real pain to decode.

I'm not sure what you mean by "similar way that I'm seeing".  If you mean 30 times per second, then as I said Canon and Panasonic seem to do this, but Sony does not.

I don't think there is much chance to getting the -ee seconds-from-start-of-video feature that you were hoping for.

- Phil

[mcarmack]

Hello,
I am trying to do the same thing as jbraun: extract the exact start time of a video file. His solution of counting how many MDPM records are in the first second (and how many are in a full second) is the only one I have found so far in my search. Are there other ways to go about this? And if Jim's solution is it, will that work with iPhone video? (you mention that different cameras have different rates of writing MDPM records.)

My problem: I am taking a video file (from an iphone) and trying to display it in synchronization with data collected via a sensor array. I will have exact time from the sensors, and I want to be sure I am displaying the sensor data and video exactly synched up.

Any help is greatly appreciated.
Mary

[Phil Harvey]

My only suggestion is to try running exiftool -ee -a on your videos to see what you get.

- Phil