Removing all MetaData from PDF files

[craisin]

I am new to all this (I must say it all looks absolutely terrific!)

I want to REMOVE the metadata and hidden data from PDF files while still retaining the originals.

What command line do I write to do this?
I have tried running it already, but the original files seem to stay there4 and no new files are written.

Many thanks

Chris (craisin)
Australia

[Phil Harvey]

Hi Chris,

The command

exiftool -all= some.pdf

will "remove" all metadata from the file "some.pdf".  A file called "some.pdf_original" will be saved.  This is the original file before removing the metadata.  To delete the original when writing, use -overwrite_original.

I used quotes around "remove" because for PDF files the metadata is only removed from the document information dictionary, and not actually deleted from the file.  This allows the information to be recovered later with this command:

exiftool -pdf-update:all= some.pdf

You can read about this in the PDF Tags documentation.

It sounds from your description as if you would like to delete this information permanently from the file.  You should be able to do this by using the acrobat distiller to linearize the PDF after removing the metadata with ExifTool, but I haven't actually tested this myself.

- Phil

[craisin]

Thanks for such a quick and comprehensive response.

I DO want to remove the data permanently, so I will investigate the linearization (what a word!)
agter running the files through ExifTool.

Actually, would you need to do that, or would linearization remove it anyway?

Cheers
Chris

[Phil Harvey]

Hi Chris,

Simply linearizing the PDF will not remove the metadata.  I'm not sure how the acrobat distiller handles metadata, but unless it has a special option to remove it then you would need something like ExifTool to do this for you first.

- Phil

[Phil Harvey]

I just had another idea.

On Mac OS when you print a file you can set the output to a PDF file, and the output PDF retains no meta information from the original (I believe).  So you could also remove the metadata using a technique like this, but note that the print conversion may affect the look of the document since it could change the layout for printing.

- Phil

[craisin]

I tried running Distiller but it says it does not process PDF files (it appears to only process Postscript files)...This is using the Distiller from Acrobat 5.0 (Full package).

Any ideas?

Cheers
Chris

[Phil Harvey]

Hi Chris,

I had thought that the distiller could linearize a PDF, but it seems I may be wrong about this.

I did a quick search but couldn't find a definitive answer, but I'd be surprised if Acrobat Pro won't do this.

- Phil

Edit: Searching adobe.com, I can't find any reference to Acrobat Pro being able to do this, so maybe it can't either.  But you don't need to linearize the PDF... any optimization or simply re-saving the PDF after removing metadata with ExifTool may do.

[craisin]

Thanks for that.

Is there a command line switch in ExifTool that could be incorporated to save the origibak file to a new name so that the removal of the Metadata is assured.

Perhaps something like the following?:

                    exiftool -all= "c:\data\test.pdf" > "c:\data\test.new.pdf"

Or perhaps if I extract a SPECIFIC metadata tag from the file in conjunction with the "-all=" switch it will write the file out (would that output file then need to be saved elsewhere to ensure the actual Metadata has gone?)

Also - I do not understand what the "DIR" seems to be doing in a lot of your command line operations. Can you please clarify that for me?

Many thanks

Chris

[Phil Harvey]

Hi Chris,

You can set the output file name to anything you want with the -o option:

exiftool -all= FILE -o OUTFILE

But I'm not sure what you are hoping to accomplish.  The changes to PDF files are always reversible with -pdf-update:all=.

When I use all capitals in a command, it symbolizes something that you must type in based on your specific situation.  FILE is any file name (such as "flowers.jpg").  DIR is any directory name (such as "c:\pictures").  However, exiftool accepts both file and/or directory names for most commands.

- Phil

[craisin]

Thanks Phil,

It was just that you said that if I saved the file after "removing" the Metadata then it would permamenty remove that information, but it appears to not be the case by your comments.

Since on a MAC you can print to a PDF file, I will try the same on my windows system using a PDF Printer to see what happens.

Thanks again for your help.

Cheers
Chris

[Phil Harvey]

It was just that you said that if I saved the file after "removing" the Metadata then it would permamenty remove that information, but it appears to not be the case by your comments.

Sorry for not being clear.  I meant saved by any PDF editor, such as Acrobat.

- Phil

[craisin]

OK - gotcha.

So if I open the file using acrobat then save it, the Metadata will then be really removed?

Cheers
Chris

[Phil Harvey]

Hi Simon,

It should be.  But I don't have Acrobat so I can't verify this.

- Phil

[craisin]

Thanks Phil - I am Chris, but I suppose the message was meant for me LOL

You are (after all) very busy at your end I suppose....Thanks again for you help.

Cheers
Chris

[Phil Harvey]

Ooops. Sorry Chris.  Yes, the message was for you.

- Phil

[craisin]

Thats OK Phil.

As a matter of interest for those who may be interested in how I succeeded in getting rid of the Metadata by using exifTool and then Acrobat, I wrote the following code:

Code Select Expand


Option Compare Database
Option Explicit
'
'This code is VBA code designed to accept filenames from the user (via a pop-up
'File-Dialog where user can select multiple files at the one time. The code then
'processes each file, removing the metadata in the file then saving the file back
'out into either the original file of (optionally) a new file with extension
'".nometa.pdf"
'
'Note! At the moment this code is deigned to work with PDF files and would require
'      changes to be made in the code to accomodate non-PDF files.
'
'Note!:
'This module must have the following references included
'within the VBA Editor environment
'  Visual Basic for Applications
'  Microsoft Access 11.o Object Library  (or equivalent/later)
'  Microsoft Office 14.0 Object Library  (or equivalent/later)
'  Adobe Acrobat 5.0 Type Library        (or equivalent/later)
'Include these by clicking on "Tools/References" within the editor

Public Sub RemoveMetaFromFiles()
Dim AcroPDDoc As Acrobat.CAcroPDDoc
Dim bOverwriteOriginal As Boolean
Dim cExt As String
Dim cFile As String
Dim cFiles As String
Dim cMsg As String
Dim cRoot As String
Dim fs As Object
Dim nEndSize As Long
Dim nFile As Integer
Dim nFiles As Long
Dim nStartSize As Long
Dim oDlg As FileDialog
Dim oFile As Object

'Change the following line to have value "True" if you DO
'want to overwrite the original file rather than writing to a new
'file with extension ".nometa.pdf"
bOverwriteOriginal = False

Set AcroPDDoc = CreateObject("AcroExch.PDDoc")

Set oDlg = FileDialog(msoFileDialogFilePicker)
oDlg.InitialFileName = "*.pdf"
oDlg.AllowMultiSelect = True
oDlg.ButtonName = "Select"
oDlg.Title = "Select files from which MetaData is to be extracted:"
oDlg.Filters.Add "PDF Files", "*.pdf", 1
oDlg.Filters.Add "All Files", "*.*", 2
oDlg.InitialView = msoFileDialogViewDetails
Set fs = CreateObject("Scripting.FileSystemObject")
oDlg.InitialFileName = GetSetting(Application.Name, "Setup", "InitialFileName", "*.pdf")
If oDlg.Show = -1 Then
  cMsg = "MetaData removed:" + vbCrLf + vbCrLf
  nFiles = oDlg.SelectedItems.Count
  For nFile = 1 To nFiles
    SaveSetting Application.Name, "Setup", "InitialFileName", oDlg.InitialFileName
    Call SysCmd(acSysCmdSetStatus, "Removing Metadata: " + CStr(nFile) + "/" + cFiles + ":" + cFile)
    cFile = oDlg.SelectedItems(nFile)
    If fs.FileExists(cFile) Then
      Set oFile = fs.getfile(cFile)
      nStartSize = nStartSize + oFile.Size
      cMsg = cMsg + Space(4) + Format(nStartSize, "###,###,###")
      cExt = LCase(Mid(cFile, InStrRev(cFile, ".")))
      cRoot = Replace(cFile, cExt, "")
     
      StripMeta cFile, cExt, bOverwriteOriginal
     
      nEndSize = nEndSize + oFile.Size
      Select Case cExt
        Case ".pdf"
          'Open the newly stripped PDF file
          If AcroPDDoc.Open(cRoot + ".nometa.pdf") Then
            'then  save it removing unreferenced objects
            AcroPDDoc.Save PDSaveCollectGarbage + PDSaveFull, cRoot + ".nometa.pdf"
            AcroPDDoc.Close
          End If
      End Select
      cMsg = cMsg + " => " + Format(nEndSize, "###,###,###") + ": " + cRoot + ".nometa.pdf" + vbCrLf
    End If
  Next
  Call SysCmd(acSysCmdClearStatus)
  Set AcroPDDoc = Nothing
  MsgBox cMsg, vbInformation + vbOKOnly
End If
End Sub
'
'

Private Sub StripMeta(cFile As String, cExt As String, bOverwriteOriginal As Boolean)
   Dim cExitTool As String
   Dim cOutFile As String
   
   'Note!: Comment out/amend the following to point to the exifTool Utility on your system
   cExifTool = "D:\exifTool\exifTool.exe"
   'cExifTool = "c:\Program Files\exifTool\exiftool.exe"
   
   Select Case cExt
     Case ".pdf"
       'The following lines remove the Metadata from the document dictionary only
       If bOverwriteOriginal Then
          Shell cExifTool + " -all= " + cFile, vbMinimizedFocus
       Else
          cOutFile = Replace(cFile, cExt, "") + ".nometa.pdf"
          Shell cExifTool + " -all= -o " + cOutFile + " " + cFile, vbMinimizedFocus
       End If
   End Select
End Sub



[Phil Harvey]

Hi Chris,

Great.  Thanks for the code.

Out of interest, is the resulting PDF linearized?  (ExifTool will report "Linearized: Yes" if it is.)

- Phil

[d4jna]

Thats OK Phil.

As a matter of interest for those who may be interested in how I succeeded in getting rid of the Metadata by using exifTool and then Acrobat, I wrote the following code:

Code Select Expand


Option Compare Database
Option Explicit
'
'This code is VBA code designed to accept filenames from the user (via a pop-up
'File-Dialog where user can select multiple files at the one time. The code then
'processes each file, removing the metadata in the file then saving the file back
'out into either the original file of (optionally) a new file with extension
'".nometa.pdf"
'
'Note! At the moment this code is deigned to work with PDF files and would require
'      changes to be made in the code to accomodate non-PDF files.
'
'Note!:
'This module must have the following references included
'within the VBA Editor environment
'  Visual Basic for Applications
'  Microsoft Access 11.o Object Library  (or equivalent/later)
'  Microsoft Office 14.0 Object Library  (or equivalent/later)
'  Adobe Acrobat 5.0 Type Library        (or equivalent/later)
'Include these by clicking on "Tools/References" within the editor

Public Sub RemoveMetaFromFiles()
Dim AcroPDDoc As Acrobat.CAcroPDDoc
Dim bOverwriteOriginal As Boolean
Dim cExt As String
Dim cFile As String
Dim cFiles As String
Dim cMsg As String
Dim cRoot As String
Dim fs As Object
Dim nEndSize As Long
Dim nFile As Integer
Dim nFiles As Long
Dim nStartSize As Long
Dim oDlg As FileDialog
Dim oFile As Object

'Change the following line to have value "True" if you DO
'want to overwrite the original file rather than writing to a new
'file with extension ".nometa.pdf"
bOverwriteOriginal = False

Set AcroPDDoc = CreateObject("AcroExch.PDDoc")

Set oDlg = FileDialog(msoFileDialogFilePicker)
oDlg.InitialFileName = "*.pdf"
oDlg.AllowMultiSelect = True
oDlg.ButtonName = "Select"
oDlg.Title = "Select files from which MetaData is to be extracted:"
oDlg.Filters.Add "PDF Files", "*.pdf", 1
oDlg.Filters.Add "All Files", "*.*", 2
oDlg.InitialView = msoFileDialogViewDetails
Set fs = CreateObject("Scripting.FileSystemObject")
oDlg.InitialFileName = GetSetting(Application.Name, "Setup", "InitialFileName", "*.pdf")
If oDlg.Show = -1 Then
  cMsg = "MetaData removed:" + vbCrLf + vbCrLf
  nFiles = oDlg.SelectedItems.Count
  For nFile = 1 To nFiles
    SaveSetting Application.Name, "Setup", "InitialFileName", oDlg.InitialFileName
    Call SysCmd(acSysCmdSetStatus, "Removing Metadata: " + CStr(nFile) + "/" + cFiles + ":" + cFile)
    cFile = oDlg.SelectedItems(nFile)
    If fs.FileExists(cFile) Then
      Set oFile = fs.getfile(cFile)
      nStartSize = nStartSize + oFile.Size
      cMsg = cMsg + Space(4) + Format(nStartSize, "###,###,###")
      cExt = LCase(Mid(cFile, InStrRev(cFile, ".")))
      cRoot = Replace(cFile, cExt, "")
     
      StripMeta cFile, cExt, bOverwriteOriginal
     
      nEndSize = nEndSize + oFile.Size
      Select Case cExt
        Case ".pdf"
          'Open the newly stripped PDF file
          If AcroPDDoc.Open(cRoot + ".nometa.pdf") Then
            'then  save it removing unreferenced objects
            AcroPDDoc.Save PDSaveCollectGarbage + PDSaveFull, cRoot + ".nometa.pdf"
            AcroPDDoc.Close
          End If
      End Select
      cMsg = cMsg + " => " + Format(nEndSize, "###,###,###") + ": " + cRoot + ".nometa.pdf" + vbCrLf
    End If
  Next
  Call SysCmd(acSysCmdClearStatus)
  Set AcroPDDoc = Nothing
  MsgBox cMsg, vbInformation + vbOKOnly
End If
End Sub
'
'

Private Sub StripMeta(cFile As String, cExt As String, bOverwriteOriginal As Boolean)
   Dim cExitTool As String
   Dim cOutFile As String
   
   'Note!: Comment out/amend the following to point to the exifTool Utility on your system
   cExifTool = "D:\exifTool\exifTool.exe"
   'cExifTool = "c:\Program Files\exifTool\exiftool.exe"
   
   Select Case cExt
     Case ".pdf"
       'The following lines remove the Metadata from the document dictionary only
       If bOverwriteOriginal Then
          Shell cExifTool + " -all= " + cFile, vbMinimizedFocus
       Else
          cOutFile = Replace(cFile, cExt, "") + ".nometa.pdf"
          Shell cExifTool + " -all= -o " + cOutFile + " " + cFile, vbMinimizedFocus
       End If
   End Select
End Sub



Thank you so much. Just what I needed. Works brilliantly.

[craisin]

No worries....it was written so long ago i forgot all about it!

(I just LOVE programming, don't you? LOL   

[metaclean]

Hello,
sorry for post into this old thread, but I think I have found a easy way to clean metadata permanently.
Just uncompress and compress the pdf with pdftk.


pdftk file.pdf output file.pdf.tmp uncompress
exiftool -all= file.pdf.tmp
pdftk file.pdf.tmp output file.pdf compress


exiftool -pdf-update:all file.pdf works till you compress the file. If you uncompress it again, you however can't recover the metadata too.
I checked the file with a hexeditor (compressed and uncompressed) and can't find any metadata or xmp stream.
Maybe this is a easy and safe way to clean metadata? Any suggestions?

Greetings
(sorry for my english, I'm not a native english speaker.)

PH Edit: Struck out the first command (it is not necessary, see later post)

[Phil Harvey]

Interesting.  But is the uncompress step really necessary?

- Phil

[craisin]

Thanks for your comment Phil.

The advantage of the code I provided is that the original is retained (with the metadata) whould it be required for another purpose.
Also it is a one line run of code.

Still, every approach is interesting.

Nice to see every successful resolution.  :-)

[metaclean]

But is the uncompress step really necessary?

After some tests I can say that the first command is obsolete and just compress at the end is enough.

[IWTA]

Hi Phil!
It was an unpleasant surprise for me to find out that after the "DELETE" function of metadata from the PDF file, they actually did not delete. Not removed from the word "completely."
Why doesn't ExifTool remove all metadata from the PDF format, but just hide it? After all, anyway, metadata is available for viewing through the simplest text editor. What is the point of this action? If we are talking about "DELETING" metadata, then why do we mean "HIDE" metadata?
There is a need to completely remove the metadata for the PDF file, but if I understood correctly, is it impossible to do this in ExifTool? Or I'm wrong?

[Phil Harvey]

I don't know what else I can say other than what has already been said in this topic.

- Phil

[StarGeek]

There is a need to completely remove the metadata for the PDF file, but if I understood correctly, is it impossible to do this in ExifTool? Or I'm wrong?

As mention elsewhere in this thread, it's not possible to remove metadata completely with exiftool.  Make sure you read this whole thread and the leading paragraphs on the PDF Tags page, especially note #2.

[IWTA]

Phil
The reason for the current implementation of the "removal" (hiding) of metadata from PDF files is the complexity of implementing a complete deletion? Or was it all originally conceived? Here is what I wanted to clarify.

[StarGeek]

Previous post by Phil on the subject

There isn't much chance that I will add a permanent delete feature to ExifTool because the PDF structure is very complicated and doing this would be a lot of work.  Bascially, the only reason I was able to add a write feature for PDF at all is because I was able to do an incremental update (which avoids the problem of having to rewrite the entire file).

[IWTA]

Thanks StarGeek
I have not seen this message. Now everything is clear to me!

[StarGeek]

I knew it existed, but it took me far too long to find it.  To the point it became less about quoting and more of a search and destroy mission. 

[Phil Harvey]

Actually, thinking about this now, I realize that it may be sufficient, and maybe not too difficult, to just zero out the old data.  The file still wouldn't be linearized, but at least the old metadata would be gone.  I'll look into this.

- Phil

[Phil Harvey]

It turns out there are 3 problems with this:

1. It is harder than I had hoped to simply zero out the existing metadata.

2. The solution wouldn't be complete because there could already be unused objects containing old metadata in the original PDF, and ExifTool wouldn't be able to zero out these.

3. It has been advertised that ExifTool PDF edits are reversible, and some users may be relying on this feature.

- Phil

[IWTA]

Phil
Removing metadata (not hiding) in PDF files is a very urgent problem. It is relevant because it is one of the most common formats for exchanging documents, and there are no alternatives for deleting metadata in PDF files. Anyway, this problem needs to be somehow solved. Hiding metadata is not a way out. And I don't know whether this "hiding" of metadata is necessary at all? Hide metadata from yourself? But that makes no sense! And from other people, such "hiding" of metadata does not work.
I think it's correct to delete (rather than "hide") metadata HOW IT IS POSSIBLE for this format. And before or after deleting the metadata, display a message saying that deleting metadata in this format will be irreversible and possibly incomplete.
And IF there are people who need the function of "hiding" metadata (but I doubt it), then you can implement the "hiding" through an additional command.

[Phil Harvey]

The technique is to run ExifTool, then re-linearze the PDF with a PDF utility.

[Hayo Baan]

The technique is to run ExifTool, then re-linearze the PDF with a PDF utility.

IWTA, to re-linearise after removing metadata you can e.g. use qpdf (on a Mac you can install it easily with brew: brew install qpdf).

For example:

Code Select Expand

# Strip metadata example.pdf and re-linearise:
exiftool -all= -overwrite_original example.pdf
mv example.pdf temp.pdf
qpdf --linearize temp.pdf example.pdf

[IWTA]

Phil
Is it possible to add the LINEARIZATION FUNCTION CODE from the qpdf program to the ExifTool program? So that in the end result, to completely remove metadata from the PDF file, the metadata was "hidden" first (a function already implemented) and re-linearization was automatically started immediately. This will allow you to completely delete metadata using only one already familiar metadata removal command in ExifTool and without using third-party applications. Is it possible to do this?

[Phil Harvey]

See my previous post.

[IWTA]

Using a bunch of ExifTool and qpdf programs doesn't guarantee complete removal of metadata?

[Hayo Baan]

If you follow the procedure as I outlined in my previous post, all metadata should be gone, completely and irreversible.

[StarGeek]

Then there's also the option of using the OS Print to PDF driver.

[IWTA]

Code Select Expand

# Strip metadata example.pdf and re-linearise:
exiftool -all= -overwrite_original example.pdf
mv example.pdf temp.pdf
qpdf --linearize temp.pdf example.pdf

I found a .bat file on the network with the following contents:

Code Select Expand

exiftool -overwrite_original -all:all="" %1
qpdf --pages %1 1-z -- --empty output.pdf
REM qpdf --linearize %1 output.pdf
move output.pdf %1
REM pause
I made a shortcut to this .bat file and if I need to clear the document of metadata, I drag the desired PDF file to this icon and the metadata is deleted. But after processing in this way, tails remain. If you open the file after processing in a text editor, then there you can see in which program the document was created. Also, the values ​​remain DocumentID, OriginalDocumentID, LastModified.
I created a .bat file from the command you proposed and dragged the document onto the icon of this .bat file, but file processing never happened.
Maybe need to correct the metadata deletion command that you proposed so that it can be used as a .bat file?

[IWTA]

Then there's also the option of using the OS Print to PDF driver.

What good PDF printer would you recommend to use to remove metadata from a file so that the contents of the document do not change?

[StarGeek]

What good PDF printer would you recommend to use to remove metadata from a file so that the contents of the document do not change?

To be honest, it is not something I've tried myself, as I don't create PDFs.  But it is something I've read about on many occasions.

Windows has one built in with Windows 10.  I'm pretty sure the Mac also has one, though I don't know for how long (found a web page that says at least 10 years).  No idea on Linux.

If you're on an earlier version of Windows, looking around I find ones by Bullzip and CutePDF.  Foxit Reader supposedly includes one as well.

[obetz]

If you're on an earlier version of Windows, looking around I find ones by Bullzip and CutePDF.  Foxit Reader supposedly includes one as well.

https://pdf24.org/ PDF24 Creator is widespread and feature-rich. Don't know about their motivation (business model?).

Foxit became insecure bloatware over the years, similar to Adobe.

CutePDF has been known to install third-party software but currently it might be clean (don't know)

Oliver

[StarGeek]

Foxit became insecure bloatware over the years, similar to Adobe.

Yeah, I had heard that as well.  Which is why I wasn't sure as I no longer have it installed.

CutePDF has been known to install third-party software but currently it might be clean (don't know)

Ah, yes, that was mentioned in the post I read.  But there is an Ninite installer and a Chocolatey installer for it, so that can help avoid the 3rd party stuff.

[suddenfear1]

Sorry, I have to make a comment on this.  You say that ExifTool supports PDF on the front page of your website.  "PDF   R/W   Adobe Portable Document Format   R3   R3   R/W/C   R3   R/W/C PDF, R Photoshop", which is listed under files supported.  But your program doesn't support PDF.  You can't even remove the metadata.  I was only alerted to this because I happened to be reading documentation in the Known Problems section, and I noticed one sentence without any explanation - "PDF - The original metadata is never actually removed."  I then had to do a web search to find this entry on the forum.  That is despicable, and you should clearly indicate on the front page of your site that your program is NOT suitable for editing PDFs. 

[Phil Harvey]

ExifTool does edit PDF metadata, but it uses the incremental update technique.  Read more about this here.

Since you seem to be very concerned about the removal of existing metadata from PDF's, you should be aware that ExifTool gives this warning when you try to remove all metadata:

Code Select Expand

> exiftool -all= a.pdf
Warning: [minor] ExifTool PDF edits are reversible. Deleted tags may be recovered! - a.pdf

- Phil

Edit: I have added a note to this effect in the Supported File Types table on the home page.

[StarGeek]

This Foxit PDF page gives a quick overview of incremental updates.