Hang with ExifTool in NSOperation (Mac Cocoa)

[DesertNomad]

I am calling the following in an NSOperation:

ExifTool* tool = new ExifTool([toolPath cStringUsingEncoding:NSUTF8StringEncoding]);
      
// read metadata from the file
TagInfo* info = tool->ImageInfo([[[self element] path] cStringUsingEncoding:NSUTF8StringEncoding]);
if (info)
{
    // do stuff

    delete info;
}

delete tool;

It works well with a queue running 4 operations simultaneously and 50 or so in the queue.

The problem arises when I try to cancel and then requeue the same file for gathering metadata. The NSOperation does not stop immediately because the ExifTool is running. If I create a new NSOperationQueue (such as in a new document), then enqueue the same file to be processed by ExifTool, it sometimes hangs.

Could there be an odd interaction with the monitoring process that ExifTool uses? Should two ExifTool objects be able to retrieve meta data from the same file at the same time?

[Phil Harvey]

Absolutely, two ExifTool processes should be able to read from the same file.

However, you should note that deleting the ExifTool object doesn't happen instantaneously.  It waits until the exiftool process terminates.  I don't know if this could cause problems inside your Cocoa NSOperation.

- Phil

[DesertNomad]

Thanks for this.

ExifTool::sNoSigPipe = 1; // this seems to help a lot
ExifTool::sNoWatchdog = 1; // this seems to help slightly

There is no good error when it hangs and I have not been able to track down where the hang is, but it feels like a race condition. I am only using on ExifTool per thread (per NSOperation really)

Does "delete tool" (on the ExifTool object) block until it quits or does it happen asynchronously? I am assuming it blocks.

[Phil Harvey]

If setting sNoSigPipe helps, then you probably aren't letting the exiftool process terminate properly.  (ie. pulling the plug before it is done)

Does "delete tool" (on the ExifTool object) block until it quits or does it happen asynchronously? I am assuming it blocks.

Yes, it blocks.

- Phil

[DesertNomad]

I'm pretty sure it is done correctly. This is the main function of an NSOperation.

-(void)main
{
   if (![self isCancelled])
   {   
      NSMutableArray* metadata = [[[NSMutableArray alloc] init] autorelease];
      
      ExifTool* tool = new ExifTool([[self toolPath] cStringUsingEncoding:NSUTF8StringEncoding]);
      TagInfo* info = tool->ImageInfo([[[self mediaObject] path] cStringUsingEncoding:NSUTF8StringEncoding]);
      if (info)
      {
         for (TagInfo* i = info; i; i = i->next)
         {
            NSMutableDictionary* dict = [[[NSMutableDictionary alloc] init] autorelease];
            [dict setValue:@(i->name) forKey:@"Name"];
            [dict setValue:@(i->value) forKey:@"Value"];
            [metadata addObject:dict];
         }
         // we are responsible for deleting the information when done
         delete info;
      }
      // we are responsible for deleting the information when done
      delete tool;
      
      //   deliver our completed data on the main queue
      if (![self isCancelled])
      {
         dispatch_async(dispatch_get_main_queue(), ^{
            [[self delegate] didFinishOperationForMediaObject:[self mediaObject] metadata:metadata];
         });
      }
   }
}

[Phil Harvey]

I'm not much of a Cocoa programmer, but shouldn't you be creating an autorelease pool for this thread?

- Phil

[DesertNomad]

NSOperation provides its own autorelease pool in this case.

[Phil Harvey]

OK.  So that isn't the problem then.

I see nothing obviously wrong with your code.  And while it should be good for stress testing, and I would like to figure out why this is hanging, it isn't the way that the C++ ExifTool object is meant to be used.  The idea is to create the ExifTool object in advance and keep it running to avoid the lag due to the startup time.

- Phil

[DesertNomad]

What would be he best way to do that?

The docs say that I should only have one ExifTool object per thread and I have one thread per file. To process 100 files, I create an NSOperationQueue which I limit to 4 operations at the same time and then queue up all 100 operations. They get fed through the queue and each one performs the above code. The threads are created by the NSOperationQueue as needed (but only 4 at a time).

I use this technique to extract images and file properties as well so I'm just trying to add ExifTool into the mix.

I'm not sure if the hang is related to tearing down the tool or trying to have too many going as I have tried dropping the same set of 100 files into two different documents (each of which has their own operation queue).

It's certainly a good stress-case.

[DesertNomad]

Some more data below. This will sometimes hang because it can't create the tool object. Any idea if trying to create tool objects too quickly might cause this? I don't really see where or how the ExifTool constructor is returning a value to the caller, so not sure where this is failing.

Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec
*** error for object 0x7fa98d045800: pointer being freed was not allocated

Thread 0 Crashed:: Dispatch queue: NSOperationQueue 0x7fa98a544fa0 :: NSOperation 0x7fa98a6540d0 (QOS: USER_INITIATED)
0   libsystem_kernel.dylib           0x00007fff8958af06 __pthread_kill + 10
1   libsystem_pthread.dylib          0x00007fff9009c4ec pthread_kill + 90
2   libsystem_c.dylib                0x00007fff8cb586df abort + 129
3   libsystem_malloc.dylib           0x00007fff9a5ed041 free + 425
4   libsystem_c.dylib                0x00007fff8cb59463 __cxa_finalize_ranges + 345
5   libsystem_c.dylib                0x00007fff8cb59767 exit + 55
6   com.myapp       0x0000000108163909 ExifTool::ExifTool(char const*, char const*) + 1369 (ExifTool.cpp:206)
7   com.myapp       0x00000001081639b5 ExifTool::ExifTool(char const*, char const*) + 37 (ExifTool.cpp:211)
8   com.myapp       0x00000001081d526c -[MetadataOperation main] + 300 (MDPropertiesOperation.mm:59)
9   com.apple.Foundation             0x00007fff8a18ea5a -[__NSOperationInternal _start:] + 654
10  com.apple.Foundation             0x00007fff8a18aa44 __NSOQSchedule_f + 194
11  libdispatch.dylib                0x00007fff9567440b _dispatch_client_callout + 8
12  libdispatch.dylib                0x00007fff9567903b _dispatch_queue_drain + 754
13  libdispatch.dylib                0x00007fff9567f707 _dispatch_queue_invoke + 549
14  libdispatch.dylib                0x00007fff95677d53 _dispatch_root_queue_drain + 538
15  libdispatch.dylib                0x00007fff95677b00 _dispatch_worker_thread3 + 91
16  libsystem_pthread.dylib          0x00007fff900994de _pthread_wqthread + 1129
17  libsystem_pthread.dylib          0x00007fff90097341 start_wqthread + 13

-(void)main
{
   if (![self isCancelled])
   {
      NSMutableArray* metadata = [[[NSMutableArray alloc] init] autorelease];
      
      ExifTool* tool = new ExifTool([[self toolPath] cStringUsingEncoding:NSUTF8StringEncoding]);
   
      delete tool;
      
      //   deliver our completed properties data on the main queue
      if (![self isCancelled])
      {
         dispatch_async(dispatch_get_main_queue(), ^{
            [[self delegate] didFinishOperationForMediaObject:[self mediaObject] options:[self options] metadata:metadata];
         });
      }
   }
}

[Phil Harvey]

Responding to your previous post...

First I would create 4 NSOperation tasks that each create an ExifTool object then go into a sleep (wait) state.

Then I would have a shared queue to which you push the file names and delegates to be messaged after the file has been processed.  The NSOperation task should wake up when a new filename is pushed into the queue.  Each NSOperation then tries to grab the next filename/delegate from the queue.  If successful, it runs ImageInfo on the file and builds the metadata object, then sends a message to the delegate.  The NSOperation task should then loop back to grab another file from the queue, and go to sleep if there isn't one available.

This technique should be up to 60x faster than the way you are doing it now.

- Phil

Edit:  You may see some additional performance benefits by letting the ExifTool object queue the commands, but doing this may be a bit more difficult.  If you don't do this, you may want to decrease the wait time in ExifTool::Complete from 1000 microseconds (maybe I should make this configurable).

Edit2:  I have just released a new version of cpp_exiftool that allows you to configure this wait time.

[DesertNomad]

I wonder if the delete process is what is causing this?

When I delete the tool object, the NSOperation ends immediately thereafter and the thread dies. is there a good way to tell when the tool has actually been deleted so that I could keep my NSOperation thread around until then? You said the delete tool would block until it is really gone but I'm wondering if killing the thread too soon after is causing some issue.

This would affect things even if I were to somehow sleep the NSOperation threads (they aren't really designed to do that) so that the tool didn't have to relaunch for each file.

Could the watchdog process somehow get messed up by my thread (and not the whole app) ending? I've never really run into this before with NSOperation.

[Phil Harvey]

Looking closely at the Exiftool destructor, I found a possible problem because I wasn't setting mCmdQueue to NULL after deleting it.  I doubt this is related to the problem you see, but you might want to grab the new version of 1.06 just in case.

- Phil

[Phil Harvey]

Looking back at your debug stack trace, the constructor failed when the watchdog process exits.  Odd.  Cocoa is catching the exit of the forked process, which is bad.  So disabling the watchdog will solve this one.  What happens when you set sNoWatchdog in this test?

- Phil

[Phil Harvey]

BTW,

The docs say that I should only have one ExifTool object per thread

Where did you see this?  You should be able to have any number of ExifTool objects per thread.  The limitation is that you may not access a single ExifTool object asynchronously from multiple threads.

- Phil