Extracting RIFF data from both .wav and .flac files

Started by kmturley, October 19, 2022, 01:01:48 AM

Previous topic - Next topic
Print
Go Down Pages1
User actions

kmturley

October 19, 2022, 01:01:48 AM
Wave files can contain unofficial metadata, such as Sampler Chunk - "smpl": https://sites.google.com/site/musicgapi/technical-documents/wav-file-format#smpl

These are used for audio looping players and samplers avoiding to loading multiple samples.

I have one such file here:

https://github.com/studiorack/basic-harmonica/blob/bf42d5bab7470cc201e3c4b6dee7925b19db6bff/samples/harmonica_1.wav

and a flac file converted using the official flac command line tool: flac harmonica_1.wav --keep-foreign-metadata

https://github.com/studiorack/basic-harmonica/blob/main/samples/harmonica_1.flac

When running these tools I can confirm the metadata exists in each file:



However I do see a different in the number of bytes (I believe as flac has riff inserted in multiple places)

I can also convert the .flac file back to .wav and it is the same size, and contains the metadata:
Code Select
flac harmonica_1.flac --keep-foreign-metadata
When using other tools I can read the data:

Code Select
$ sndfile-info har.wav
smpl : 60
  Manufacturer : 0
  Product      : 0
  Period       : 20833 nsec
  Midi Note    : 64
  Pitch Fract. : 0
  SMPTE Format : 0
  SMPTE Offset : 00:00:00 00
  Loop Count   : 1
    Cue ID : 131072  Type :  0  Start : 12707  End : 47221  Fraction :     0  Count :     0
  Sampler Data : 0https://linux.die.net/man/1/sndfile-info

This only works for .wav files. There is a feature request for libsndfile to support 'smpl' in flac files: https://github.com/libsndfile/libsndfile/issues/59

Code Select
$ metaflac ./har.flac --list
smpl<aQ@�1u�METADATA block #7
  type: 2 (APPLICATION)
  is last: false
  length: 20
  application ID: 72696666
  data contents:https://xiph.org/flac

However as you can see the result returned are different. I would like a both .wav and .flac RIFF 'smpl' data to be returned in the same format, so I can verify the results match.

https://exiftool.org appears to be tool to do that. But it also produced inconsistent results between .wav and .flac:

Code Select
$ exiftool -a -G1 -s ./har.wav
[ExifTool]      ExifToolVersion                 : 12.42
[System]        FileName                        : har.wav
[System]        Directory                       : .
[System]        FileSize                        : 95 kB
[System]        FileModifyDate                  : 2022:10:11 21:16:37-07:00
[System]        FileAccessDate                  : 2022:10:15 14:39:46-07:00
[System]        FileInodeChangeDate             : 2022:10:15 14:39:50-07:00
[System]        FilePermissions                 : -rw-r--r--
[File]          FileType                        : WAV
[File]          FileTypeExtension               : wav
[File]          MIMEType                        : audio/x-wav
[RIFF]          Encoding                        : Microsoft PCM
[RIFF]          NumChannels                     : 1
[RIFF]          SampleRate                      : 48000
[RIFF]          AvgBytesPerSec                  : 96000
[RIFF]          BitsPerSample                   : 16
[RIFF]          Manufacturer                    : 0
[RIFF]          Product                         : 0
[RIFF]          SamplePeriod                    : 20833
[RIFF]          MIDIUnityNote                   : 64
[RIFF]          MIDIPitchFraction               : 0
[RIFF]          SMPTEFormat                     : none
[RIFF]          SMPTEOffset                     : 00:00:00:00
[RIFF]          NumSampleLoops                  : 1
[RIFF]          SamplerDataLen                  : 0
[RIFF]          SamplerData                     : (Binary data 20 bytes, use -b option to extract)
[RIFF]          UnshiftedNote                   : 64
[RIFF]          FineTune                        : 0
[RIFF]          Gain                            : 0
[RIFF]          LowNote                         : 0
[RIFF]          HighNote                        : 127
[RIFF]          LowVelocity                     : 0
[RIFF]          HighVelocity                    : 127
[RIFF]          Comment                         : Recorded on 7/10/2022 in Edison.
[RIFF]          Software                        : FL Studio 20
[Composite]     Duration                        : 0.99 s
and for flac:

Code Select
$ exiftool -a -G1 -s ./har.flac
[ExifTool]      ExifToolVersion                 : 12.42
[System]        FileName                        : har.flac
[System]        Directory                       : .
[System]        FileSize                        : 83 kB
[System]        FileModifyDate                  : 2022:10:11 20:59:37-07:00
[System]        FileAccessDate                  : 2022:10:15 14:44:12-07:00
[System]        FileInodeChangeDate             : 2022:10:15 14:42:26-07:00
[System]        FilePermissions                 : -rw-r--r--
[File]          FileType                        : FLAC
[File]          FileTypeExtension               : flac
[File]          MIMEType                        : audio/flac
[FLAC]          BlockSizeMin                    : 4096
[FLAC]          BlockSizeMax                    : 4096
[FLAC]          FrameSizeMin                    : 3442
[FLAC]          FrameSizeMax                    : 6514
[FLAC]          SampleRate                      : 48000
[FLAC]          Channels                        : 1
[FLAC]          BitsPerSample                   : 16
[FLAC]          TotalSamples                    : 47222
[FLAC]          MD5Signature                    : f89646c0d3056ec38c3e33ca79299253
[Vorbis]        Vendor                          : reference libFLAC 1.4.1 20220922
[Composite]     Duration                        : 0.98 s
How can I read this data consistently regardless of .flac or .wav file?

My guess is that flac does not actually use a RIFF block to store the data? https://github.com/xiph/flac/search?q=flac__foreign_metadata_new

Do I need a custom exiftool config to parse the data?
Creator of StudioRack: open-source audio plugin ecosystem
https://studiorack.github.io/studiorack-site/

Phil Harvey

#1
October 19, 2022, 01:22:59 PM
The metadata from the RIFF file is stored in a set of "riff" Application metadata blocks in the FLAC file.  You have access to the binary data of these blocks if you add the -u option with ExifTool, but currently ExifTool doesn't do any further decoding.

There are a number of registered FLAC Application block types, and maybe it makes sense for ExifTool to support some of these.  I'll look into this possibility, starting with the "riff" type if you think this would give you what you want.  Here is the relevant section of the exiftool -v3 output of these blocks from your FLAC sample:

Code Select
FLAC metadata block, type 2:
      0070: 72 69 66 66 52 49 46 46 dc 71 01 00 57 41 56 45 [riffRIFF.q..WAVE]
  Application = riffRIFF.q.WAVE
  - Tag 0x0002 (16 bytes):
      0070: 72 69 66 66 52 49 46 46 dc 71 01 00 57 41 56 45 [riffRIFF.q..WAVE]
FLAC metadata block, type 2:
      0084: 72 69 66 66 66 6d 74 20 10 00 00 00 01 00 01 00 [rifffmt ........]
      0094: 80 bb 00 00 00 77 01 00 02 00 10 00             [.....w......]
  Application = rifffmt .....w...
  - Tag 0x0002 (28 bytes):
      0084: 72 69 66 66 66 6d 74 20 10 00 00 00 01 00 01 00 [rifffmt ........]
      0094: 80 bb 00 00 00 77 01 00 02 00 10 00             [.....w......]
FLAC metadata block, type 2:
      00a4: 72 69 66 66 64 61 74 61 ec 70 01 00             [riffdata.p..]
  Application = riffdata.p.
  - Tag 0x0002 (12 bytes):
      00a4: 72 69 66 66 64 61 74 61 ec 70 01 00             [riffdata.p..]
FLAC metadata block, type 2:
      00b4: 72 69 66 66 73 6d 70 6c 3c 00 00 00 00 00 00 00 [riffsmpl<.......]
      00c4: 00 00 00 00 61 51 00 00 40 00 00 00 00 00 00 00 [....aQ..@.......]
      00d4: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 [................]
      00e4: 00 00 02 00 00 00 00 00 a3 31 00 00 75 b8 00 00 [.........1..u...]
      00f4: 00 00 00 00 00 00 00 00                         [........]
  Application = riffsmpl<aQ@...1u.
  - Tag 0x0002 (72 bytes):
      00b4: 72 69 66 66 73 6d 70 6c 3c 00 00 00 00 00 00 00 [riffsmpl<.......]
      00c4: 00 00 00 00 61 51 00 00 40 00 00 00 00 00 00 00 [....aQ..@.......]
      00d4: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 [................]
      00e4: 00 00 02 00 00 00 00 00 a3 31 00 00 75 b8 00 00 [.........1..u...]
      00f4: 00 00 00 00 00 00 00 00                         [........]
FLAC metadata block, type 2:
      0100: 72 69 66 66 69 6e 73 74 07 00 00 00 40 00 00 00 [riffinst....@...]
      0110: 7f 00 7f 00                                     [....]
  Application = riffinst.@..
  - Tag 0x0002 (20 bytes):
      0100: 72 69 66 66 69 6e 73 74 07 00 00 00 40 00 00 00 [riffinst....@...]
      0110: 7f 00 7f 00                                     [....]
FLAC metadata block, type 2:
      0118: 72 69 66 66 61 63 69 64 18 00 00 00 01 00 00 00 [riffacid........]
      0128: 3c 00 80 00 00 00 00 00 02 00 00 00 04 00 04 00 [<...............]
      0138: 00 00 00 00                                     [....]
  Application = riffacid..<....
  - Tag 0x0002 (36 bytes):
      0118: 72 69 66 66 61 63 69 64 18 00 00 00 01 00 00 00 [riffacid........]
      0128: 3c 00 80 00 00 00 00 00 02 00 00 00 04 00 04 00 [<...............]
      0138: 00 00 00 00                                     [....]
FLAC metadata block, type 2:
      0140: 72 69 66 66 4c 49 53 54 04 00 00 00 61 64 74 6c [riffLIST....adtl]
  Application = riffLIST.adtl
  - Tag 0x0002 (16 bytes):
      0140: 72 69 66 66 4c 49 53 54 04 00 00 00 61 64 74 6c [riffLIST....adtl]
FLAC metadata block, type 2:
      0154: 72 69 66 66 4c 49 53 54 44 00 00 00 49 4e 46 4f [riffLISTD...INFO]
      0164: 49 43 4d 54 21 00 00 00 52 65 63 6f 72 64 65 64 [ICMT!...Recorded]
      0174: 20 6f 6e 20 37 2f 31 30 2f 32 30 32 32 20 69 6e [ on 7/10/2022 in]
      0184: 20 45 64 69 73 6f 6e 2e 00 00 49 53 46 54 0d 00 [ Edison...ISFT..]
      0194: 00 00 46 4c 20 53 74 75 64 69 6f 20 32 30 00 00 [..FL Studio 20..]
  Application = riffLISTDINFOICMT!Recorded on 7/10/2022 in Edison.ISFT.FL Studio 20
  - Tag 0x0002 (80 bytes):
      0154: 72 69 66 66 4c 49 53 54 44 00 00 00 49 4e 46 4f [riffLISTD...INFO]
      0164: 49 43 4d 54 21 00 00 00 52 65 63 6f 72 64 65 64 [ICMT!...Recorded]
      0174: 20 6f 6e 20 37 2f 31 30 2f 32 30 32 32 20 69 6e [ on 7/10/2022 in]
      0184: 20 45 64 69 73 6f 6e 2e 00 00 49 53 46 54 0d 00 [ Edison...ISFT..]
      0194: 00 00 46 4c 20 53 74 75 64 69 6f 20 32 30 00 00 [..FL Studio 20..]
- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Phil Harvey

#2
October 19, 2022, 02:08:22 PM
Or, more specifically, is this what you are looking for?:

Code Select
> exiftool samples_harmonica_1.flac -G1
[ExifTool]      ExifTool Version Number         : 12.49
[System]        File Name                       : samples_harmonica_1.flac
[System]        Directory                       : .
[System]        File Size                       : 83 kB
[System]        File Modification Date/Time     : 2022:10:19 13:10:52-04:00
[System]        File Access Date/Time           : 2022:10:19 14:05:21-04:00
[System]        File Inode Change Date/Time     : 2022:10:19 13:10:52-04:00
[System]        File Permissions                : -rw-r--r--
[File]          File Type                       : FLAC
[File]          File Type Extension             : flac
[File]          MIME Type                       : audio/flac
[FLAC]          Block Size Min                  : 4096
[FLAC]          Block Size Max                  : 4096
[FLAC]          Frame Size Min                  : 3442
[FLAC]          Frame Size Max                  : 6514
[FLAC]          Channels                        : 1
[FLAC]          Total Samples                   : 47222
[FLAC]          MD5 Signature                   : f89646c0d3056ec38c3e33ca79299253
[Vorbis]        Vendor                          : reference libFLAC 1.4.1 20220922
[RIFF]          Encoding                        : Microsoft PCM
[RIFF]          Num Channels                    : 1
[RIFF]          Sample Rate                     : 48000
[RIFF]          Avg Bytes Per Sec               : 96000
[RIFF]          Bits Per Sample                 : 16
[RIFF]          Manufacturer                    : 0
[RIFF]          Product                         : 0
[RIFF]          Sample Period                   : 20833
[RIFF]          MIDI Unity Note                 : 64
[RIFF]          MIDI Pitch Fraction             : 0
[RIFF]          SMPTE Format                    : none
[RIFF]          SMPTE Offset                    : 00:00:00:00
[RIFF]          Num Sample Loops                : 1
[RIFF]          Sampler Data Len                : 0
[RIFF]          Sampler Data                    : (Binary data 20 bytes, use -b option to extract)
[RIFF]          Unshifted Note                  : 64
[RIFF]          Fine Tune                       : 0
[RIFF]          Gain                            : 0
[RIFF]          Low Note                        : 0
[RIFF]          High Note                       : 127
[RIFF]          Low Velocity                    : 0
[RIFF]          High Velocity                   : 127
[RIFF]          Acidizer Flags                  : One shot
[RIFF]          Root Note                       : High C
[RIFF]          Beats                           : 2
[RIFF]          Meter                           : 4/4
[RIFF]          Tempo                           : 0
[RIFF]          Comment                         : Recorded on 7/10/2022 in Edison.
[RIFF]          Software                        : FL Studio 20
[Composite]     Duration                        : 0.87 s
- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

kmturley

#3
October 19, 2022, 11:05:29 PM
Thanks for the quick response. Yes that is exactly it!

How did you get that output? is it a custom .ExifTool_config file?
If you could point me in the correct direction that would be very useful.

I plan on sharing an exiftool tool solution with the wider audio developer community (Audio Programmer and SFZ) and also run it in CI pipelines to validate that samples libraries contain the correct metadata for samplers.
Creator of StudioRack: open-source audio plugin ecosystem
https://studiorack.github.io/studiorack-site/

Phil Harvey

#4
October 20, 2022, 07:52:57 AM
Quote from: kmturley on October 19, 2022, 11:05:29 PMHow did you get that output?

With ExifTool 12.49 (released yesterday).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

kmturley

#5
October 21, 2022, 01:17:07 AM
Verified, thank-you for the help and quick turnaround.

I made a donation and shared exiftool with the audio developer community in Discord
Creator of StudioRack: open-source audio plugin ecosystem
https://studiorack.github.io/studiorack-site/

Phil Harvey

#6
October 21, 2022, 06:54:49 AM
I got the donation, thanks!

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).
Print
Go Up Pages1
User actions