ExifTool Forum

ExifTool => Newbies => Topic started by: TechnoPhil on February 09, 2012, 11:41:11 AM

Title: LargeFileSupport not enabled
Post by: TechnoPhil on February 09, 2012, 11:41:11 AM
Hi there,
i can use exiftool on my mac and ubuntu, but i have a problem when i try to check the exif for big files!

This is the message:
"Warning                         : End of processing at large atom (LargeFileSupport not enabled)"

How can i fix it?

Thank you guys!

Filippo
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on February 09, 2012, 12:45:24 PM
Hi Flippo,

You can enable this with the following config file (https://exiftool.org/config.html):

%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);


But for this to work your Perl and C libraries must both support a 64-bit file seek.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: TechnoPhil on February 09, 2012, 01:27:33 PM
Ok thanks,
but where can i find this config file?
I am running exiftool under Mac and Buntu!  ;)
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on February 09, 2012, 01:30:29 PM
Use the config file in my previous post, and follow the instructions in the sample config file (linked from my previous post) to activate it.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: TechnoPhil on February 09, 2012, 02:40:40 PM
Great, now it works for both operating systems!
I used "touch" and "nano" command in my terminal!

Sometimes exiftool does not show me some data, for example a video bit-rate, image size of the video ... is that a bug or exif data loss? :(
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on February 09, 2012, 02:48:23 PM
ExifTool attempts to show you all of the available metadata in a file.  Some video formats are problematic because they don't always store everything you would want as metadata.  Duration is the big thing that would be nice to have, but this isn't stored in many formats. (Things like this may be calculated by processing the entire video stream, but ExifTool doesn't do this.)

- Phil
Title: Re: LargeFileSupport not enabled
Post by: TechnoPhil on February 09, 2012, 03:07:38 PM
Thanks you very much for your support, very kind and useful!

;)
Title: Re: LargeFileSupport not enabled
Post by: beroe on March 26, 2015, 12:39:58 PM
I was able to suppress the LargeFileSupport error by changing my config as instructed, but I only get the most basic exif output for videos over 4GB in size. I know there is still info there, because ffprobe can report the missing metadata like frame rate and dimensions from these files.

Is there something else I need to do to access the exif from these files?

Below is an example of a table of output, sorted by file size and you can see where the info ends...

Thanks!


2950 MB SHOGUN_S007_S001_T103.MOV 2015:03:16 16:29:52 0:00:37 3840 2160 .Apple ProRes 422 HQ 23.976
2959 MB SHOGUN_S007_S001_T092.MOV 2015:03:16 16:12:20 0:00:35 3840 2160 .Apple ProRes 422 HQ 23.976
2985 MB SHOGUN_S007_S001_T152.MOV 2015:03:16 18:46:54 0:00:37 3840 2160 .Apple ProRes 422 HQ 23.976
2997 MB SHOGUN_S007_S001_T009.MOV 2015:03:16 01:17:24 0:00:37 3840 2160 .Apple ProRes 422 HQ 23.976
3063 MB SHOGUN_S007_S001_T085.MOV 2015:03:16 16:05:52 0:00:37 3840 2160 .Apple ProRes 422 HQ 23.976
3070 MB SHOGUN_S007_S001_T006.MOV 2015:03:16 00:44:08 0:00:38 3840 2160 .Apple ProRes 422 HQ 23.976
3087 MB SHOGUN_S007_S001_T101.MOV 2015:03:16 16:27:20 0:00:37 3840 2160 .Apple ProRes 422 HQ 23.976
3323 MB SHOGUN_S007_S001_T104.MOV 2015:03:16 16:30:31 0:00:40 3840 2160 .Apple ProRes 422 HQ 23.976
3382 MB SHOGUN_S007_S001_T002.MOV 2015:03:15 21:46:16 0:00:40 3840 2160 .Apple ProRes 422 HQ 23.976
3387 MB SHOGUN_S006_S001_T007.MOV 2015:03:14 21:59:58 0:00:40 3840 2160 .Apple ProRes 422 HQ 23.976
3430 MB SHOGUN_S007_S001_T138.MOV 2015:03:16 17:18:46 0:00:41 3840 2160 .Apple ProRes 422 HQ 23.976
3478 MB SHOGUN_S007_S001_T116.MOV 2015:03:16 16:45:05 0:00:43 3840 2160 .Apple ProRes 422 HQ 23.976
3716 MB SHOGUN_S007_S001_T153.MOV 2015:03:16 18:48:18 0:00:45 3840 2160 .Apple ProRes 422 HQ 23.976
4017 MB SHOGUN_S007_S001_T090.MOV 2015:03:16 16:09:10 0:00:48 3840 2160 .Apple ProRes 422 HQ 23.976
4184 MB SHOGUN_S007_S001_T114.MOV
4196 MB SHOGUN_S006_S001_T011.MOV
4251 MB SHOGUN_S007_S001_T088.MOV
4768 MB SHOGUN_S007_S001_T113.MOV
4781 MB SHOGUN_S007_S001_T083.MOV
5123 MB SHOGUN_S007_S001_T144.MOV
5187 MB SHOGUN_S007_S001_T001.MOV
5319 MB SHOGUN_S007_S001_T155.MOV
5376 MB SHOGUN_S007_S001_T145.MOV
6138 MB SHOGUN_S007_S001_T148.MOV
6328 MB SHOGUN_S007_S001_T003.MOV
7211 MB SHOGUN_S007_S001_T127.MOV
7771 MB SHOGUN_S007_S001_T084.MOV
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on March 27, 2015, 09:49:03 AM
Are you on Linux?  I'll see if I can reproduce this when I get back next week.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: beroe on March 27, 2015, 09:57:34 PM
Thanks for looking into it.
I am on OSX, Yosemite. Not sure what version of exiftool I have, but fairly recent.
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on March 28, 2015, 07:18:47 AM
OK, thanks.  That makes things easier, but now I just need to locate a >4GB MOV file.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on March 31, 2015, 02:40:54 PM
I've tested this on a 6 GB MOV file on Yosemite, but when I use "exiftool -api largefilesupport=1", the metadata is extracted properly.  If I could have one of your samples to test, it would be useful.  My email is philharvey66 at gmail.com

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on April 03, 2015, 09:28:27 PM
It is possible that this problem is caused by a metadata atom exceeding 16 MB in size (see this thread (https://exiftool.org/forum/index.php/topic,6391.0.html)).  If so, ExifTool 9.91 (to be released soon) may solve the problem.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: beroe on April 13, 2015, 07:05:20 PM
Thanks Phil.

Oops, I was not adding anything about largefilesupport to my command line — I thought that just putting it in the config file was sufficient.

However, if I try exiftool -api largefilesupport=1 MyFile.MOV at the command line, I get an error:
File not found: largefilesupport=1

Not familiar with the syntax on those flags what am I doing wrong...?

Also, for testing, the normal command looks like "works" on large files, in the sense that it gives the file name and size, but it does not give specifics like frame rate...

This command below works on smaller files generated by the same camera to output frame rate, but doesn't report anything for big ones (I get 77 fields from small files, and only 13 for big ones if I run exiftool without any arguments.)

exiftool -tab -v0 -S  -FileName -FileSize -VideoFrameRate MyFile.MOV
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on April 13, 2015, 09:06:21 PM
Quote from: beroe on April 13, 2015, 07:05:20 PM
Oops, I was not adding anything about largefilesupport to my command line — I thought that just putting it in the config file was sufficient.

It is.

QuoteHowever, if I try exiftool -api largefilesupport=1 MyFile.MOV at the command line, I get an error:
File not found: largefilesupport=1

What version of ExifTool are you using?  The -api option was added in 9.40.

QuoteThis command below works on smaller files generated by the same camera to output frame rate, but doesn't report anything for big ones (I get 77 fields from small files, and only 13 for big ones if I run exiftool without any arguments.)

If you can't get it to work, upload a sample and mail me the link (philharvey66 at gmail.com), and I'll take a look.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: beroe on April 13, 2015, 09:22:17 PM
I have 9.0.2. I'll try again after updating and see how things look.

Thanks again.

Not sure where to host a 6GB file, but I can try temporarily putting it on a local ftp and e-mail a link to you.
Title: Re: LargeFileSupport not enabled
Post by: beroe on April 13, 2015, 09:56:47 PM
I upgraded to 9.92 and .. IT WORKS!

Thanks. Sorry about that rookie mistake -- I was originally on a ship when I was testing and never checked for updates when I got back.



Title: Re: LargeFileSupport not enabled
Post by: NMAAHC on February 19, 2016, 03:39:22 PM
Hi,

I'm having the same issue with "Warning : End of processing at large atom (LargeFileSupport not enabled)"

I'm looking for my exiftool config file in my home directory but ls -a isn't listing anything that it could be.

Do I have to create one first?

Thanks,
Bleakley
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on February 19, 2016, 03:51:25 PM
Hi Bleakley,

Yes, you need to create it.  Check out the sample config file (https://exiftool.org/config.html) and example.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: k_grons on March 16, 2016, 09:53:56 AM
Hi,

- I updated my ExifTool version to 10.13.
- I verified that my config file exists in /usr/local/bin and includes:
%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);
1; #end


When I run:
$ exiftool <.mov file over 100 GB>

I get the error Warning                         : End of processing at large atom (LargeFileSupport not enabled)


Any idea what the issue may be?
Title: Re: LargeFileSupport not enabled
Post by: Hayo Baan on March 16, 2016, 11:14:01 AM
You config file should be in your user's home directory, not in bin!
Title: Re: LargeFileSupport not enabled
Post by: k_grons on March 16, 2016, 12:45:46 PM
Hi Hayo,

Thanks! I moved .ExifTool_config to my home directory, but still getting the same error:

Warning                         : End of processing at large atom (LargeFileSupport not enabled)

I installed ExifTool using the Mac installer, and it's in /usr/local/bin - could that be the issue?

Thanks again.
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on March 16, 2016, 02:25:13 PM
The config file should work in your home directory.  FAQ 11 (https://exiftool.org/faq.html#Q11) has some tips for helping with config file problems.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: k_grons on March 17, 2016, 01:08:32 PM
Phil,

Thank you - I was able to solve the issue using the troubleshooting procedures described in FAQ 11.

Thanks!
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on March 17, 2016, 01:10:00 PM
So, what was the problem?
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 05, 2016, 07:25:57 AM
Hi Phil,
I have the same problem of getting the error message "End of processing at large atom (LargeFileSupport not enabled)". I have added the entry "%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);"
in my config file as you advised. I have also performed the test with "Writing XMP-xmp:NewXMPxmpTag" and could see that tag, so obviously my config-file is recognised. But nonetheless the error message persists.I attach the file for your convenience.

I have to meta tag a large amount of videos, many of which are above 4GB and I use the Windows software TagScanner to add the metadata - which is actually tags that I designed myself. I can verify that they exist for small files, but not for the one that exceed the critical file size.
Can you help out?

On a personal note: I used to windsurf a lot when I was young and my fathers company made the booms and masts for many companies. The one you are using in your profile picture might actually have come out of our workshop :)
I attach a pic of myself (Loongtime ago) for your entertainment :)
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 05, 2016, 07:42:15 AM
Nice duck gybe with that polyester clark-foam board!  Looks like Hatteras.  I've got lots of picture of myself that are very similar. :)  I need some replacement masts though, so if you have any constant-curve 25-75% carbon 25 DIN standard-diameter 4.65m masts around, let me know.

The problem with your config file is that %Image::ExifTool::UserDefined::Options is defined twice, and the one at the bottom of the file is overriding your definition.

- Phil


Title: Re: LargeFileSupport not enabled
Post by: Harald on October 06, 2016, 10:19:19 AM
Hi Phil,

SUPER COOL! That was it and it works like a charm now.  :-*
By the way: we have recently bought the UK based DAM Assetbank. You might be interested to hear that in the backend they use your tool to read and write metadata. I guess this is in line with your license or you get a revenue from it.

Regarding the mast: unfortunately the company had to stop producing windsurfing equipment 20 years ago when the mountain bikes wiped out the windsurfing market. :(
But anyway, we were only making aluminium masts. I can imagine it is hard to get equipment nowadays.  Assuming that you are in the states and I am in Europe, shipping would have been an issue :)
Thanks a lot for your help again. There might be the one or other question upcoming as we have the huge task of migrating 80.000 pictures from two organisations into the DAM. Very much appreciated to see you are as responsive as your tool is amazing.

- Harald
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 06, 2016, 10:53:45 AM
Hi Harald,

Quote from: Harald on October 06, 2016, 10:19:19 AM
You might be interested to hear that in the backend they use your tool to read and write metadata. I guess this is in line with your license or you get a revenue from it.

ExifTool is free.  Donations are all voluntary, even for commercial use.

Quoteunfortunately the company had to stop producing windsurfing equipment 20 years ago when the mountain bikes wiped out the windsurfing market. :(

Yup...  Windsurfing has had its day I guess.  I should have bought more masts back when I could. :P

QuoteThere might be the one or other question upcoming as we have the huge task of migrating 80.000 pictures from two organisations into the DAM.

ExifTool should be up to the task.  One ExifTool client I know has run it on 30.000.000.000 images (without any donations I might add).

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 17, 2016, 06:47:38 AM
Hi Phil,
yes, everything that arises comes to an end.  That is even true for Windsurfing :)
Quite clever, those Buddhists :)

I consider your making Exiftool available for free even for commercial use to be incredibly generous and I rejoice ins this great attitude. Very inspiring! Thank you for this great example in a time of incredible greed!
:D
I have progressed with the project and have added extra meta tags to our videos (.mp4) to ingest them into the DAM. I used the Windows Software TagScanner not to have to dive into the seemingly complex task of defining them in the config file.

When I use Exiftool to list the tag it works fine. For example:
------------------------------------------------------------------------------------------------------------------------
FPMTMBP:Desktop FPMTmrm$ exiftool -youtubelink /Users/FPMTmrm/Desktop/20100923_2000_LZR_LOP.mp4
YOUTUBE LINK                    : https://www.youtube.com/watch?v=Qgas5fmDry4
------------------------------------------------------------------------------------------------------------------------

But when I try to write to the same file:
------------------------------------------------------------------------------------------------------------------------
FPMTMBP:Desktop FPMTmrm$ exiftool -youtubelink='www.youtube.com' /Users/FPMTmrm/Desktop/20100923_2000_LZR_LOP.mp4
Warning: Tag 'youtubelink' does not exist
Nothing to do.
------------------------------------------------------------------------------------------------------------------------

Any idea why Exiftool recognises the metatag when listing, but not for writing?
In case needed, I uploaded the video here:
https://fpmt.box.com/s/11wowk6o8j0d37ur62ycskk95bxk5xjv
I had to share this link via email to grant you access. Sorry for the bother!

Thanks a lot in advance for your help! Have a great week!
kind regards,
Harald

_____________________________
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 17, 2016, 08:08:49 AM
Hi Harald,

ExifTool has only limited write ability for MP4 videos.  See the QuickTime tags documentation (https://exiftool.org/TagNames/QuickTime.html) for details.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 17, 2016, 10:11:58 AM
Hi Phil,
thanks for the quick reply.
Unfortunate for us that Exiftool is not able to change tags outside the QuickTime specification. I will have to try using TagScanner with an Excel file.
Maybe worthwhile altering the error message from "tag xxxx does not exist" to "not supported" or similar.

In any case, thanks for your help. Exiftool will still be very helpful for our images.

kind regards,
Harald

_____________________________

Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 17, 2016, 11:35:40 AM
Hi Harald,

Quote from: Harald on October 17, 2016, 10:11:58 AM
Maybe worthwhile altering the error message from "tag xxxx does not exist" to "not supported" or similar.

I see your point.  It does not exist in ExifTool's database, but it is confusing if you were thinking it meant in your file.

I'll have to think about this though.  Changing the wording of messages isn't something I take lightly because doing so may break some scripts that use ExifTool.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 21, 2016, 04:40:33 AM
Hi Phil,
QuoteChanging the wording of messages isn't something I take lightly because doing so may break some scripts that use ExifTool.
I see. There is always implications that one does not see from outside.

A completely different question: I notice you support reading tags of Word files, but when I try to write to a .docx file it says
QuoteError: Writing of DOCX files is not yet supported
Do you plan to add this any time soon? We need to put the current file path on the local drive to the metadata before ingesting to the DAM and I wonder how to do that.
Thanks a lot in advance for your help.
kind regards,
Harald

_____________________________
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 21, 2016, 07:16:12 AM
First, I've considered your "TAG not supported" suggestion and will implement this in the next release.

Second, support for DOCX is not going to happen soon.  ExifTool would have to support writing ZIP files as well as arbitrary XML to be able to do this.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 21, 2016, 08:38:15 AM
Hi Phil,

thanks a lot for the quick response. Too bad, but to be expected.
And I guess, you are not aware of any tool that would be suitable to write metadata to .doc files in batch, e. g. from a CSV file?
kind regards,
Harald

_____________________________
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 21, 2016, 08:43:01 AM
Hi Harald,

Sorry, I don't know of any such tools.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 21, 2016, 09:04:48 AM
Hi Phil,
I just realised that there might have been a misunderstanding regarding the MP4 files. You wrote
QuoteExifTool has only limited write ability for MP4 videos.  See the QuickTime tags documentation for details.
Yet I found that I can successfully write Country and City to a video with Exiftool, both not being QuickTime tags, right? At least I could not find "City" in the link you provided.
Actually it even created new entries for Country. I had added a COUNTRY-tag with Mp3tag, but when I reset the tag with Exiftool it showed up with small letters and listing them with "Exiftool -country" would result in
-COUNTRY: Untited States  -> from MP3tag
-Country: India                  -> from Exiftool

So it had left the one from MP3tag alone and written to its own specification.
All our videos in question are H264 .mp4 videos, usually encoded with handbrake or Adobe Premiere Media Encoder. As far as I understand, these are not QuickTime videos, correct?

Yet when I look at the embedded metadata of an asset that I had ingested into Assetbank - which uses Exiftool at the backend - all tags show up as QuickTime:...., even City, Country and Youtube link, where at least the last one is obviously not an official tag.

I assume either
a) these are actually XMP tags that are listed incorrectly in Assetbank or
b) it is possible to write extra QuickTime tags but you chose not to implement this in Exiftool.

Please let me know whether I am wrong and if it is indeed possible to extend XMP-tags and write to them, how I could achieve this.
Sorry for bothering you that much.

best regards,
Harald
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on October 21, 2016, 09:23:09 AM
Hi Harald,

ExifTool can write XMP to MP4 files.  The Country tag you wrote must be XMP.

I can't speak for Assetbank, but you should see this in XMP if you read it back again using ExifTool with the -G option.

- Phil
Title: Re: LargeFileSupport not enabled
Post by: Harald on October 21, 2016, 01:32:06 PM
Hi Phil,

thanks, the -G option helped me a lot and clarified the whereabouts for many tags. Thanks for all your help.

kind regards,
Harald

_____________________________
Title: Re: LargeFileSupport not enabled
Post by: gheppell on February 17, 2020, 02:40:15 PM
Pardon my technological ignorance, but i am trying to get around the largefillesupport enabled issue, but am confused.  I note that you point to a config file as a workaround.  However, I do not know how or where to enable such a file.  Is this a text file?  Where do I put the file within my directories (Win 10)?  Should I be typing something at a command line?  If possible, I need step by step instructions as to how to get around the largefile issue.

thank you
Title: Re: LargeFileSupport not enabled
Post by: StarGeek on February 17, 2020, 03:07:50 PM
Here is the example.config file (https://exiftool.org/config.html).  Download it and save it in the same directory as exiftool.  It needs to be renamed to .ExifTool_config (dot at the front). If you're using windows, that can't be done on the desktop.  It either has to be done when save the file or through CMD.

One that is done, open it up in a text editor (not a word processor).  Go to the bottom of the file and locate this line
%Image::ExifTool::UserDefined::Options
You'll see several options set there.  After the RequestAll and before the );, add this new line
LargeFileSupport => 1,
Title: Re: LargeFileSupport not enabled
Post by: sarcha on November 16, 2022, 03:54:58 PM
Hey Phil.

I am facing the same problem. I have exiftool version 12.3.3.0
I am trying to extract metadata of 4GB MP4 file.

I am trying the following command

exiftool -api -largefilesupport=1 -ee -G3 -s C0011.MP4 > metadata011.csv

but the file extracted is missing large amount of metadatmetadata011.csv a.

Additionally it is giving the following warning: Warning: End of processing at large atom (LargeFileSupport not enabled)

I have also installed the .ExifTool_config file in the home directory as referred by you.

I have also attached the output file that is generated for reference
Can you please suggest what can be done here?

Regards,
Sarang Chahakar
Title: Re: LargeFileSupport not enabled
Post by: Phil Harvey on November 16, 2022, 09:33:29 PM
It should be -api largefilesupport=1

(no dash before largefilesupport)

- Phil