News:

If this site goes down it is because of the crappy DreamHost web hosting: They will disable your "unlimited traffic" web server if a single bot hammers the site with a moderate load.  If necessary, an alternate ExifTool forum page is available at https://sourceforge.net/p/exiftool/discussion/

Main Menu

Copy PNG metadata via Regex to import into Lightroom Hierarchical tags?

Started by TushyBhutt, November 11, 2022, 05:47:30 PM

Previous topic - Next topic

TushyBhutt

Hi,

I've looked through the forums but I cannot find an answer, or my search skills suck (likely the latter).  I have a PNG file created by an AI image generator that writes info in the "parameters" tag.

I'd like to import as hierarchical keywords into Lightroom.  Unfortunately, there are no easy tag separators and I need to use commas, colons, and periods to classify the structure properly, so the code below won't work

-sep "," -XMP-LR:HierarchicalSubject<Parameters

Using the Regex101 website, I was able to wrangle the 835 characters into 27 groups and need to put most of them into LR.  Ten or eleven will become parent keywords, the rest go in as children.

How can I get Exiftool to export the data in the PNG Parameters via a regex expression and import it into Lightroom keywords?  Here is some sample output for you to review, if it helps:

Current PNG data:

professional [digital airbrush:photo:0.5] art of an imp with wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting.Negative prompt: logo, text, signature, (bad art).Steps: 75, Sampler: DPM++ 2M Karras, CFG scale: 12, Seed: 3334373882, Size: 768x1152, Model hash: 81761151, Model: Stable 1.5, Denoising strength: 0.7, Eta: 0.89, First pass size: 0x0



Regex mask:

(.*)((\.)(Negative prompt:)) (.*)((\.)(Steps:)) (\d+), (Sampler:)(.*), (CFG scale:) (\d+), (Seed:) (\d+), (Size:) (.*), (Model hash:) (.*), (Model:)(.*), (Denoising strength:) (.*), (Eta:) (.*), (First pass size:) (.*)



Desired outcome for LR keywords

Prompt|professional [digital airbrush:photo:0.5] art of an imp with wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting
Negative prompt|logo, text, signature, (bad art)
Steps|75
Sampler|DPM++ 2M Karras
CFG scale|12
Seed|3334373882
Size|768x1152
Model hash|81761151
Model|Stable 1.5
Denoising strength|0.7
Eta|0.89
First pass size|0x0



I am using Exiftool 12.5 on Windows 10 64-bit with the ExifToolGui front end.  Thanks for any help you can provide!

TB

Phil Harvey

Can you post the binary output of the "PNG data" (the output with the ExifTool -b) option?  I suspect some of the periods are probably newlines, and we will need the exact data to test things out.  Use this command:

exiftool -b -Parameters FILE > out.txt

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

TushyBhutt

First of all, thanks for an awesome tool!  Each time I ran the command you gave in ExifToolGui or JExifToolGui, I got a file not found error, but here's the output from ExifToolGUI log:

Quoteprofessional [digital airbrush:photo:0.5] art of an imp with sword, wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting
Negative prompt: logo, text, signature, (bad art)
Steps: 75, Sampler: DPM++ 2M Karras, CFG scale: 12, Seed: 3334373882, Size: 768x1152, Model hash: 81761151, Model: Stable1.5, Denoising strength: 0.7, Eta: 0.89, First pass size: 0x0, aesthetic_score: 6.6, sampler: DPM++ 2M Karras, cfg_scale: 12, sd_model_hash: 81761151, seed: 3334373882
Error: File not found - out.txt
<-END-

Looks like some of the "." are newline indicators for sure, but not read properly in the GUItool (see first screenshot).

 Also, I've attached the PNG itself (the list above and the metadata are slightly different from my first post because I reran the image in the generator)

StarGeek

I'm assuming this is with Stable Diffusion?  Is that what is setting the PNG:Parameters tag?

Here is a RegEx101 example for parsing the data. I removed the capture of the static strings because you know what they are and capturing just complicates the regex.  Also the substitution is using new lines \n to make it easier to read.  For your actual command you would use something else with the -sep option.

Phil, if Stable Diffusion is creating the PNG:Parameters tag, you might consider adding it to exiftool.  I'll dig around deeper to make sure.  SD is becoming very popular, especially among table top RPGs such as D&D, for it's ability to generate images like this.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

TushyBhutt

Quote from: StarGeek on November 12, 2022, 10:33:21 AMI'm assuming this is with Stable Diffusion?  Is that what is setting the PNG:Parameters tag?

Here is a RegEx101 example for parsing the data. I removed the capture of the static strings because you know what they are and capturing just complicates the regex.  Also the substitution is using new lines \n to make it easier to read.  For your actual command you would use something else with the -sep option.

Phil, if Stable Diffusion is creating the PNG:Parameters tag, you might consider adding it to exiftool.  I'll dig around deeper to make sure.  SD is becoming very popular, especially among table top RPGs such as D&D, for it's ability to generate images like this.

Hi, yes it is from Stable Diffusion, one of the localized front-ends adds the parameters in (few of them do as far as I know).

Thank you for the substitution output in Regex101, how do I use that in Exiftool to read back into the XMP-LR:HierarchicalSubject ? I'm OK with fiddling with Regex, however blindly, it's getting the output from PNG:Parameters into the other tag that I am not sure how to do :)

StarGeek

Try this, assuming Windows CMD.  If otherwise, change double quotes into single quotes.  I'm also using ## as a separator.  If that pattern is used as part of the Parameters data at some point, it would have to be changed

exiftool -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" /path/to/files/

Example output.  The -b (-binary) option is used to make it clear that each item is a separate entry
C:\>exiftool -P -overwrite_original -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" Y:\!temp\bbb\test.png
    1 image files updated

C:\>exiftool -G1 -a -s -b -HierarchicalSubject Y:\!temp\bbb\test.png
Prompt|professional [digital airbrush:photo:0.5] art of an imp with sword, wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting
Negative prompt|logo, text, signature, (bad art)
Steps|75
Sampler| DPM++ 2M Karras
CFG scale|12
Seed|3334373882
Size|768x1152
Model hash|81761151
Model| Stable1.5
Denoising strength|0.7
Eta|0.89
First pass size|0x0
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

TushyBhutt

Quote from: StarGeek on November 12, 2022, 03:05:51 PMTry this, assuming Windows CMD.  If otherwise, change double quotes into single quotes.  I'm also using ## as a separator.  If that pattern is used as part of the Parameters data at some point, it would have to be changed

exiftool -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" /path/to/files/

Example output.  The -b (-binary) option is used to make it clear that each item is a separate entry
C:\>exiftool -P -overwrite_original -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" Y:\!temp\bbb\test.png
    1 image files updated

C:\>exiftool -G1 -a -s -b -HierarchicalSubject Y:\!temp\bbb\test.png
Prompt|professional [digital airbrush:photo:0.5] art of an imp with sword, wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting
Negative prompt|logo, text, signature, (bad art)
Steps|75
Sampler| DPM++ 2M Karras
CFG scale|12
Seed|3334373882
Size|768x1152
Model hash|81761151
Model| Stable1.5
Denoising strength|0.7
Eta|0.89
First pass size|0x0


Ah clever, adding the "##" as a token to become a separator into the regex output!  It worked like a charm!  Thank you so much!

blue-j

QuotePhil, if Stable Diffusion is creating the PNG:Parameters tag, you might consider adding it to exiftool.  I'll dig around deeper to make sure.  SD is becoming very popular, especially among table top RPGs such as D&D, for its ability to generate images like this.

I second this suggestion.  I'll create some DALL-E 2 images and look under the hood as well.

Also note the png:aesthetic_score field in the sample.  - J

blue-j

Attached is a sample of output from DALL-E 2.  It seems to have Maker Notes that are not recognized.  I wasn't able to get images out of GPT-3 at the moment, but will try again later when I have more time.

- J


StarGeek

Did some searching and I can't find any documentation on what tags are actually being added by the front ends. I think these tags would be better served as a config file for the time being.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

blue-j

I'd personally like to be proactive, at least with Stable Diffusion.  Adding new PNG tags is silly and surely borne of a lack of knowledge about metadata.  Can we suggest a better option?  I'd be happy to be an ambassador.  Even a custom XMP or Maker Notes namespace would be better.  Do you have an idea of what would be best?

Regarding Dall-E 2 and OpenAI, I don't understand what is going on with their Maker Notes, so I can't judge their implementation.  Were you able to see any data?

- J

blue-j

Quote from: StarGeek on November 14, 2022, 10:25:56 AMDid some searching and I can't find any documentation on what tags are actually being added by the front ends. I think these tags would be better served as a config file for the time being.

I created a couple definitions... FWIW

CFG (Classifier-Free Guidance) Scale
The higher the value, the more the output will be in line with the input, but it will be at the expense of image quality; the lower the value, the more likely it is to be distant from the prompt or input image, but the output will be of higher quality

Seed
Seed is the unique representation of a specific image, a kind of master key making it possible to regenerate exactly the same image, with variations

I'll share more as I do more research.

- J


blue-j

On more research, the Stable Diffusion ecosystem is rather broad, and changing the fields for them would break a fair number of applications.  That might be worth it, but I'm unsure.  -J

StarGeek

Quote from: blue-j on November 14, 2022, 01:27:58 PMEven a custom XMP or Maker Notes namespace would be better.  Do you have an idea of what would be best?

XMP would be best and easiest to deal with.  "Prompt" and "Negative Prompt" would be list type tags.  The rest could be simple strings or if you wanted to get technical, integer/real.

I'm hoping to upgrade my 10 year old computer real soon, and SD is on my list of things to try.

Going off (exiftool) topic, is Nvidia better for the CUDA stuff or is GPU ram better.  I was looking things over and the Nvidea GPUs in my price range were 10-12 gb ram, while there were some 16 gig Radeon GPUs in the same range.  Woefully ill-informed on things at the moment.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

blue-j

I'm sorry to have to tell you that I have no idea!  I don't deal with that aspect of the work, and don't run it locally.  I know CUDA is highly recommended and NVIDIA is a leader in the field... I will ask my AI team for you.  - J

StarGeek

Quote from: blue-j on November 14, 2022, 05:48:31 PMI know CUDA is highly recommended and NVIDIA is a leader in the field

That does follow with what I've understood.  I do have a GeForce RTX 3080 10GB marked as the most likely purchase.  It's at the very top of my price range, but it does come with a gift card which would be used toward other parts of the build.

QuoteI will ask my AI team for you.
I highly appreciate that.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

blue-j


blue-j

OK, I have heard back from true experts who do this stuff every day.  NVIDIA with CUDA and CUDNN is their recommendation.  They shared this article:

https://www.projectpro.io/article/gpus-for-machine-learning/677#mcetoc_1ggng87r41d

- J

StarGeek

Quote from: blue-j on November 15, 2022, 12:25:15 AMHave you played with the online version, DreamStudio?
https://beta.dreamstudio.ai/

Not yet.  I made an account a while back but there isn't much with the free credits and I don't have money to spare.

Quote from: blue-j on November 15, 2022, 01:52:06 AMOK, I have heard back from true experts who do this stuff every day.  NVIDIA with CUDA and CUDNN is their recommendation.  They shared this article:

https://www.projectpro.io/article/gpus-for-machine-learning/677#mcetoc_1ggng87r41d

Many thanks.  That will help a lot.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

TushyBhutt

Quote from: StarGeek on November 14, 2022, 03:59:25 PM
Quote from: blue-j on November 14, 2022, 01:27:58 PMEven a custom XMP or Maker Notes namespace would be better.  Do you have an idea of what would be best?

XMP would be best and easiest to deal with.  "Prompt" and "Negative Prompt" would be list type tags.  The rest could be simple strings or if you wanted to get technical, integer/real.

I'm hoping to upgrade my 10 year old computer real soon, and SD is on my list of things to try.

Going off (exiftool) topic, is Nvidia better for the CUDA stuff or is GPU ram better.  I was looking things over and the Nvidea GPUs in my price range were 10-12 gb ram, while there were some 16 gig Radeon GPUs in the same range.  Woefully ill-informed on things at the moment.

NVIDIA has CudA, AMD and Intel do not.  AMD and Intel ARC work (kinda) on Linux, not so well in Windows. I have an NVIDIA 3060 with 12GB of VRAM and it was $500 CDN, outputs an 80 step 768x1152 image in about a minute.

TushyBhutt

Quote from: StarGeek on November 12, 2022, 03:05:51 PMTry this, assuming Windows CMD.  If otherwise, change double quotes into single quotes.  I'm also using ## as a separator.  If that pattern is used as part of the Parameters data at some point, it would have to be changed

exiftool -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" /path/to/files/

Example output.  The -b (-binary) option is used to make it clear that each item is a separate entry
C:\>exiftool -P -overwrite_original -Sep "##" "-HierarchicalSubject<${Parameters;s/(.*)\nNegative prompt: (.*)\nSteps: (\d+), Sampler:(.*), CFG scale: (\d+), Seed: (\d+), Size: (.*), Model hash: (.*), Model:(.*), Denoising strength: (.*), Eta: (.*), First pass size: (.*)/Prompt|$1##Negative prompt|$2##Steps|$3##Sampler|$4##CFG scale|$5##Seed|$6##Size|$7##Model hash|$8##Model|$9##Denoising strength|$10##Eta|$11##First pass size|$12/}" Y:\!temp\bbb\test.png
    1 image files updated

C:\>exiftool -G1 -a -s -b -HierarchicalSubject Y:\!temp\bbb\test.png
Prompt|professional [digital airbrush:photo:0.5] art of an imp with sword, wearing old cloak, curly pale red-blonde hair, with few tattoos, dynamic bright atmospheric lighting
Negative prompt|logo, text, signature, (bad art)
Steps|75
Sampler| DPM++ 2M Karras
CFG scale|12
Seed|3334373882
Size|768x1152
Model hash|81761151
Model| Stable1.5
Denoising strength|0.7
Eta|0.89
First pass size|0x0


Ooooh.... follow up question!  Sometimes not all fields are output and sometimes not in the same order.  I would imagine that this is not something Regex could easily account for? The names are always the same, just not order/occurrence depending on the setting chosen to write to the file.

StarGeek

Quote from: TushyBhutt on November 17, 2022, 08:25:39 PMI have an NVIDIA 3060 with 12GB of VRAM and it was $500 CDN, outputs an 80 step 768x1152 image in about a minute.

Ah, ok.  That answers one question I couldn't figure out.  I had read somewhere that the 3060 had something that was to discourage bit coin miners, but I wasn't sure if it would affect other computations.

Quote from: TushyBhutt on November 17, 2022, 08:27:16 PMOoooh.... follow up question!  Sometimes not all fields are output and sometimes not in the same order.  I would imagine that this is not something Regex could easily account for? The names are always the same, just not order/occurrence depending on the setting chosen to write to the file.

That would be doable, but I think a bit different.  My original thought was to use a Perl split rather than regex but the fact that it was only 3 string separated by new lines and the final bit was all comma separated made me discard it.

I have to go run an errand, but the basic idea now would be to split on new line, then split the last string on the comma(space), them run all that through map. A bit more Perl-ish and complex.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

TushyBhutt

Quote from: StarGeek on November 18, 2022, 11:13:42 AM
Quote from: TushyBhutt on November 17, 2022, 08:27:16 PMOoooh.... follow up question!  Sometimes not all fields are output and sometimes not in the same order.  I would imagine that this is not something Regex could easily account for? The names are always the same, just not order/occurrence depending on the setting chosen to write to the file.

That would be doable, but I think a bit different.  My original thought was to use a Perl split rather than regex but the fact that it was only 3 string separated by new lines and the final bit was all comma separated made me discard it.

I have to go run an errand, but the basic idea now would be to split on new line, then split the last string on the comma(space), them run all that through map. A bit more Perl-ish and complex.

Oh, cool.  But please don't put too much work into it if I'm the only one asking.  In the meantime I can use the script to capture maybe one variable at a time.  Is there a flag that would append the PNG data into XMP:LR instead of just overwrite it?

TushyBhutt

Quote from: StarGeek on November 18, 2022, 11:13:42 AMI have to go run an errand, but the basic idea now would be to split on new line, then split the last string on the comma(space), them run all that through map. A bit more Perl-ish and complex.

Would this link help? Someone used PERL to extract the PNG data from a StableDiffusion image

https://github.com/Eugenii10/exiftool-sd-format

blue-j

I haven't seen "SD Extras" in the wild, so that's helpful!  Might you have a sample that evidences that?  I also wonder if they store whether CLIP was used or not.  I have yet to see it.

- J

TushyBhutt

Quote from: blue-j on November 20, 2022, 03:04:05 PMI haven't seen "SD Extras" in the wild, so that's helpful!  Might you have a sample that evidences that?  I also wonder if they store whether CLIP was used or not.  I have yet to see it.

- J

Are you looking for evidence of readable data in a PNG file from Stable Diffusion, or a program that reads it?  ImageGlass can do it with Exiftool as a plugin and the code above according to this link:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/1569#discussioncomment-4175458

I haven't tried it myself, I'm trying to get the PNG data into Lightroom :)

Also the screenshot there shows a weird cutoff for keywords fourteen and fifteen, so looks like the code could be tweaked

blue-j

Our software already reads all mentioned SD and OpenAI metadata. It is not yet available to the public though.

- J


TushyBhutt

Quote from: blue-j on November 26, 2022, 09:14:43 PMOur software already reads all mentioned SD and OpenAI metadata. It is not yet available to the public though.

- J



Is that software ExifTool or something else?  I don't think the new Stable 2.0 release from Thursday has anything new but may be worth checking.

blue-j

Our software uses ExifTool but also other tools, and post-processing.

Stable 2 is out!  Nice!  Will check, thanks so much for the heads up.  : )

- J

blue-j

I don't see any change in metadata using Stable v2 using DreamStudio.  - J

TushyBhutt

Quote from: blue-j on November 29, 2022, 01:12:31 PMI don't see any change in metadata using Stable v2 using DreamStudio.  - J

The AUTO1111 implementation now has more fields, I think  New fields include Hires sizing, hires steps, and hires upscaler among others. 

According to this link:

https://github.com/Eugenii10/exiftool-sd-format/issues/1#issuecomment-1368368763

The repo has code to extract metadata but I don't know which script it is.  Even if I did, I have no Python skills so I have on ideas how to extract it to put it into Lightroom, let alone automate it. 

TushyBhutt

Quote from: StarGeek on November 18, 2022, 11:13:42 AMI have to go run an errand, but the basic idea now would be to split on new line, then split the last string on the comma(space), them run all that through map. A bit more Perl-ish and complex.

Must've been a long errand :)

I think I found the python script(s) that generate the parameters, but am not sure.  One seems to be a module that can be called by other scripts:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/generation_parameters_copypaste.py

And then there's this one that calls the one above and writes to the file.  I think.  It's an extension for AUTO1111 so I am sure the base program has the same functionality, but I couldn't track down that one so I chose the "next best" one.

https://github.com/adieyal/sd-dynamic-prompts/blob/bug/copypaste-parameters/ui/save_params.py