Wrong duration detected for VCD DAT (MPEG) file

[stan]

OS: Windows 7 SP1 32-bit

ExifTool Version: 9.61

Command Line: exiftool -s3 -duration Test.dat

Console Output: 9833 days 14:52:44

MediaInfo Report:

Code Select Expand


General
Complete name                  : Test.dat
Format                         : CDXA/MPEG-PS
File size                      : 810 MiB
Duration                       : 1h 33mn
Overall bit rate mode          : Variable
Overall bit rate               : 1 215 Kbps
Writing library                : encoded by TMPGEnc (ver. 2.58.44.152)

Video
ID                             : 224 (0xE0)
Format                         : MPEG Video
Format version                 : Version 1
Format settings, BVOP          : No
Format settings, Matrix        : Custom
Format settings, GOP           : M=3, N=24
Duration                       : 1h 33mn
Bit rate mode                  : Variable
Bit rate                       : 1 079 Kbps
Width                          : 352 pixels
Height                         : 288 pixels
Display aspect ratio           : 4:3
Frame rate                     : 25.000 fps
Standard                       : PAL
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.426
Time code of first frame       : 00:00:00:00
Time code source               : Group of pictures header
Stream size                    : 719 MiB (89%)
Writing library                : TMPGEnc 2.58.44.152

Audio
ID                             : 192 (0xC0)
Format                         : MPEG Audio
Format version                 : Version 1
Format profile                 : Layer 2
Duration                       : 1h 33mn
Bit rate mode                  : Constant
Bit rate                       : 112 Kbps
Channel(s)                     : 2 channels
Sampling rate                  : 44.1 KHz
Compression mode               : Lossy
Stream size                    : 74.7 MiB (9%)




I remuxed the DAT file using MP4Box and ran ExifTool on the resultant MP4:

Command Line: exiftool -s3 -duration Test.mp4

Console Output: 1:33:12

MediaInfo Report:

Code Select Expand


General
Complete name                  : Test.mp4
Format                         : MPEG-4
Format profile                 : Base Media
Codec ID                       : isom
File size                      : 793 MiB
Duration                       : 1h 33mn
Overall bit rate mode          : Variable
Overall bit rate               : 1 189 Kbps
Encoded date                   : UTC 2014-05-20 18:13:39
Tagged date                    : UTC 2014-05-20 18:13:39
Writing application            : My MP4Box GUI 0.6.0.6 <http://my-mp4box-gui.zymichost.com>

Video
ID                             : 1
Format                         : MPEG Video
Format version                 : Version 1
Format settings, BVOP          : No
Format settings, Matrix        : Custom
Format settings, GOP           : M=3, N=24
Codec ID                       : 6A
Duration                       : 1h 33mn
Bit rate mode                  : Variable
Bit rate                       : 1 073 Kbps
Maximum bit rate               : 2 091 Kbps
Width                          : 352 pixels
Height                         : 288 pixels
Display aspect ratio           : 1.222
Original display aspect ratio  : 4:3
Frame rate mode                : Constant
Frame rate                     : 25.000 fps
Standard                       : PAL
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.424
Time code of first frame       : 00:00:00:00
Time code source               : Group of pictures header
Stream size                    : 716 MiB (90%)
Writing library                : TMPGEnc 2.58.44.152
Encoded date                   : UTC 2014-05-20 18:13:39
Tagged date                    : UTC 2014-05-20 18:16:01

Audio
ID                             : 2
Format                         : MPEG Audio
Format version                 : Version 1
Format profile                 : Layer 2
Codec ID                       : 6B
Duration                       : 1h 33mn
Bit rate mode                  : Constant
Bit rate                       : 112 Kbps
Maximum bit rate               : 117 Kbps
Channel(s)                     : 2 channels
Sampling rate                  : 44.1 KHz
Compression mode               : Lossy
Stream size                    : 74.7 MiB (9%)
Encoded date                   : UTC 2014-05-20 18:15:30
Tagged date                    : UTC 2014-05-20 18:16:01


Is there any way for ExifTool to support the original DAT format properly?

[Phil Harvey]

Is it possible to upload the video somewhere for me to download?  If not, just email me the first megabyte of the video and I'll take a look.  My mail is philharvey66 at gmail.com

- Phil

[stan]

Hi Phil, thanks for the reply. I have mailed you the first couple of MBs of the file. Hope it helps. If you really do need the whole shebang though then 800+MB is gonna take a while due to the miserable upload speeds and bandwidth/data caps we're blessed with here.

[stan]

A quick follow-up: I used cdxa2mpeg (part of the GNU VCDImager suite) to extract the actual MPEG data from the DAT file by stripping the RIFF container and mode2 sector headers.

Besides the Duration, ExifTool has problems with most other properties of the original DAT file such as Num Channels, Sample Rate, Avg Bytes Per Sec and Bits Per Sample as you can see from the output below:

Code Select Expand


exiftool Test.dat

ExifTool Version Number         : 9.61
File Name                       : Test.dat
File Size                       : 810 MB
File Type                       : RIFF
MIME Type                       : application/unknown
Encoding                        : Unknown (0)
Num Channels                    : 0
Sample Rate                     : 1096307989
Avg Bytes Per Sec               : 1
Bits Per Sample                 : 0
Duration                        : 9833 days 14:52:44


ExifTool does a much better job with the extracted MPEG but seems to be unable to determine the Duration as can be seen from the output below:

Code Select Expand


exiftool Test.mpg

ExifTool Version Number         : 9.61
File Name                       : Test.mpg
File Size                       : 800 MB
File Type                       : MPEG
MIME Type                       : video/mpeg
Image Width                     : 352
Image Height                    : 288
Aspect Ratio                    : 4:3, 625 line, PAL, CCIR601
Frame Rate                      : 25 fps
Video Bitrate                   : Variable
MPEG Audio Version              : 1
Audio Layer                     : 2
Audio Bitrate                   : 112 kbps
Sample Rate                     : 44100
Channel Mode                    : Stereo
Mode Extension                  : Bands 4-31
Copyright Flag                  : False
Original Media                  : False
Emphasis                        : None
Image Size                      : 352x288


So besides better support for VCD DAT files it would be much appreciated if ExifTool can also properly report the duration of MPEG videos. Thanks!

[Phil Harvey]

I got the .DAT sample, thanks.

The problem with the CDXA is that this format seems to repurpose the RIFF "fmt " chunk, and fill it with incompatible information.  I'll look into this.

ExifTool does not report the duration for your mpeg because it can't easily be calculated from the metadata since it has a variable bit rate.

- Phil

[stan]

ExifTool does not report the duration for your mpeg because it can't easily be calculated from the metadata since it has a variable bit rate.

Interesting. I wonder how MediaInfo and all the video players are able to display the duration immediately? Are they calculating it somehow or simply guessing, because they sure don't seem to need to read the entire file for it. For that matter even Windows Explorer is able to display the information immediately as soon as the file is selected (for the record I have no third-party codec packs installed):

[Phil Harvey]

If you find out, please let me know.

- Phil

[stan]

If you find out, please let me know.

I looked around and found the following, but I'm no programmer so can't make head nor tail of it. Hope you can though! File_Mpegv.cpp in the MediaInfo source has something that looks like it may be relevant:

Code Select Expand


void File_Mpegv::Streams_Finish()
{
    //Duration
    if (PTS_End>PTS_Begin)
    {
        if (PTS_End_temporal_reference<temporal_reference_Max)
        {
            int64u tc_ToAdd=tc/((progressive_sequence || picture_structure==3)?1:2); //Progressive of Frame
            PTS_End+=tc_ToAdd*(temporal_reference_Max-PTS_End_temporal_reference-1); //TODO: handle cases with 3:2 Pulldown (different progressive_sequence / picture_structure)
        }
        Fill(Stream_Video, 0, Video_Duration, float64_int64s(((float64)(PTS_End-PTS_Begin))/1000000));
    }
    else if (Frame_Count_NotParsedIncluded!=(int64u)-1)
    {
        Fill(Stream_Video, 0, Video_FrameCount, Frame_Count_NotParsedIncluded);
        if (FrameRate)
            Fill(Stream_Video, 0, Video_Duration, Frame_Count_NotParsedIncluded/FrameRate*1000, 0);
    }
    else if (!TimeCodeIsNotTrustable && Time_End_Seconds!=Error && FrameRate)
    {
        TimeCode Time_Begin_TC;
        Time_Begin_TC.FramesPerSecond=(int8u)ceil(FrameRate);
        Time_Begin_TC.DropFrame=(FrameRate-ceil(FrameRate))?true:false;
        Time_Begin_TC.Hours=(int8u)(Time_Begin_Seconds/3600);
        Time_Begin_TC.Minutes=(int8u)((Time_Begin_Seconds%3600)/60);
        Time_Begin_TC.Seconds=(int8u)(Time_Begin_Seconds%60);
        Time_Begin_TC.Frames=(int8u)Time_Begin_Frames;
        TimeCode Time_End_TC;
        Time_End_TC.FramesPerSecond=(int8u)ceil(FrameRate);
        Time_End_TC.DropFrame=(FrameRate-ceil(FrameRate))?true:false;
        Time_End_TC.Hours=(int8u)(Time_End_Seconds/3600);
        Time_End_TC.Minutes=(int8u)((Time_End_Seconds%3600)/60);
        Time_End_TC.Seconds=(int8u)(Time_End_Seconds%60);
        Time_End_TC.Frames=(int8u)Time_End_Frames;
        int32u Time_Begin_FrameCount=Time_Begin_TC.ToFrames();
        int32u Time_End_FrameCount=Time_End_TC.ToFrames();
        Fill(Stream_Video, 0, Video_FrameCount, Time_End_FrameCount-Time_Begin_FrameCount, 0);
        Fill(Stream_Video, 0, Video_Duration, (Time_End_FrameCount-Time_Begin_FrameCount)/FrameRate*1000, 0);
    }

    ...

}


Does any of this help?

[Phil Harvey]

Right, thanks.  It seems to be parsing the video stream to get the timecode of the last frame.  Smart, but not something I think that I would do in ExifTool.

I really try to avoid parsing video streams if at all possible because I simply don't have the manpower to support all of the codecs.

- Phil

[stan]

That's that, I suppose. Looking forward to any DAT-related improvements if at all possible. Thanks for spending time on all this, appreciate it.

- Stan

P.S. When you said that the MPEG's duration can't easily be calculated from the metadata since it has a variable bit rate, does that mean that ExifTool does display the duration for MPEGs with constant bit rates? (Sorry, don't have any with me right now to confirm.)

[Phil Harvey]

Hi Stan,

Yes, it calculates approximate duration by taking the file size and dividing it by the total bitrate if the bitrate is constant.

- Phil