Return to site
Return to site

Duplicate Detective 1 94 – Find And Remove Duplicate Files

broken image
Duplicate Detective 1 94 – Find And Remove Duplicate Files
Introduction
  • Duplicate File Detective is a feature-rich software application whose purpose is to locate duplicate files on your computer, allowing you to delete them and, thus clear up space on your hard drive.
  • Duplicate Detective – Find and Delete Duplicate Files 1.94 Description AWARDED MACWORLD ‘MAC GEM' 2014 – The Best Duplicate Finder For Mac – On Sale: 40% Off Limited Time Only!
  • Duplicate File Detective uses state-of-the-art file content analysis technology to show you exactly where your duplicate files are (even when the file names are different). You can search multiple folders, drives, removable storage devices, even network shares – all in a single pass. Then move, zip, or remove duplicate files safely with assistance from our innovative SmartMark duplicate file.
This tutorial shows how to find and optionally delete similar or duplicate pages within the same PDF document using the AutoSplit™ plug-in for the Adobe® Acrobat®. This operation detects similar pages and presents them to the user for a review. The user can review the results and select/unselect individual pages from the list of duplicates for a possible deletion or extraction. You can perform the following operations:
  • Find duplicate and near-duplicate pages
  • Bookmark duplicate pages
  • Extract duplicate pages into a separate PDF document
  • Delete duplicate pages from the document
  • Save page similarity report
The plug-in provides two different methods for detecting duplicate or near-duplicate pages:

Duplicate Detective 1 94 – Find And Remove Duplicate Files Onto

Compare Page Text Only
Use this method to compare page text regardless of its visual appearance. It computes page similarity based on text content only and completely ignores text appearance, layout, images and graphics that might be present on the page. It is the best method to detect duplicates in most document types.
Compare Visual Appearance of the Pages
This method compares pages 'as images' and detects pages that look exactly the same. This method does not compare any invisible text that may be present on the page. It is not advised to use this method on scanned paper documents.
Using Scanned Paper Documents
Quite often this operation is used to find duplicate pages in the scanned paper documents. The scanned documents need to be OCRed prior to using them for any text-based processing. The OCR is a process of recognizing text in scanned documents and making them searchable. It is essential to understand that text recognition in scanned documents is prone to errors and it is rarely 100% accurate. The number of errors depends on scanning resolution and original document quality. In most common cases, a scanned page may contain between 1 to 10 recognition errors where certain letters are incorrectly identified. For example, depending on the font, the lowercase letter l can look exactly like the numeral 1 . The uppercase letter O is often misidentified as the numeral 0, or uppercase letter S as the numeral 5 and etc. Since many alphanumeric symbols share similar, or identical, physical characteristics, differentiation often poses a challenge. This is why a similarity-based comparison comes useful to detect small differences between pages that are produced by the text recognition process. Low quality scanned documents may contain a large number of errors making them unusable for any reliable text-based comparison. See the following tutorial on how to OCR scanned documents and asses their suitability for the text-based processing. .
Prerequisites
You need a copy of the Adobe® Acrobat® along with the AutoSplit™ plug-in installed on your computer in order to use this tutorial. You can download trial versions of both the Adobe® Acrobat® and the AutoSplit™ plug-in.
Contents
Method 1 - Comparing Page Text Only ↑overview
This method compares page similarity only based on their page content. The visual appearance, text position and order is irrelevant. This method also ignores any images and graphics present on the pages. The modified cosine similarity metric is used to calculate how similar are two pages based on their text content.
Step 1 - Open a PDF File
Start the Adobe® Acrobat® application and open a PDF file using 'File > Open.' menu.
Step 2 - Open the 'Find Duplicate Pages' Dialog
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Step 3 - Specify Settings
Check the 'Compare only page text (ignore visual appearance of the pages)' option.
Using Predefined Settings
The text-based method provides a number of predefined parameters sets that are suitable for comparing different kinds of documents with a different amount of recognition errors. Each predefined set of parameters provides different conditions for similarity calculations:
  • Custom Settings - all settings are specified by user
  • Scanned Paper Document: High Quality
  • Scanned Paper Document: Medium Quality
  • Fax Document: Low Quality
  • Non-scanned PDF: exact match
  • Non-scanned PDF: fuzzy match
  • Exact match (with text order)- this method does not use cosine similarity
Settings appear below the menu after selecting a predefined parameter set.
Here are the settings used by the predefined sets:
Click 'Edit.' to customize page similarity settings:
The text comparison method uses 3 parameters to limit how different two 'similar' pages can be. By varying these parameters, it is possible to detect pages that have a different degree of similarity.
  • Minimal allowed page text similarity (in percent) - this is the value of cosine similarity metric expressed in percent. Specify minimum allowed page text similarity between 70 and 100 (in percent).
  • Maximum allowed page length difference (in characters).
  • Maximum allowed page text difference (in words).
Use these settings to experiment with processing settings when it is necessary to adjust processing algorithm for a specific document.
Use Sample Pages
Optionally, click 'Set From Page Sample.' to specify page similarity settings based on the two sample pages:
Select two pages that can be consider identical. The software will auto-calculate page similarity and the statistics will appear in the left lower corner of the dialog.
Click 'OK' to save the current similarity settings.
Specify Text Filtering Options
There are several parameters that control the page content that is being analyzed by the text comparison algorithm. Use these options when comparing scanned paper documents that may contain various text recognition errors. These options exclude certain kind of characters from processing. In many cases, it may help to compute a more accurate similarity metric.
  • Ignore text case - this option ignores text case while comparing text.
  • Ignore punctuation (,.!?-) - this options excludes all punctuation characters from comparison.
  • Ignore non-alphanumeric characters - this options ignores all characters except letters and digits.
Click 'OK' to save page similarity settings.
Click 'OK' to start searching the current PDF document for the duplicate pages:
Step 4 - Inspect Duplicate Pages
The 'Delete Duplicate Pages' dialog shows a list of duplicate or near-duplicate pages. Click on a page record to display a corresponding page in the viewer. Examine pages and select/unselect pages for deletion.
Optionally, click 'Save Report.' to create a page similarity report in HTML format. Or click 'Bookmark Pages' to create bookmarks in PDF for selected duplicate pages.
The plug-in allows to preview/compare the found duplicate or near-duplicate pages. The page similarity (in %) and the number of mismatched words is displayed for each pair of pages. Here are the examples computed for the pair of the scanned paper documents:
Note that the apperance and location of the text do not affect the results.
These two pages are considered identical despite the difference in the text color:
These two pages are considered identical despite the difference in the content layout:
These two pages are consider 94% similar despite the difference in the text order, layout and absence of the image:
Step 5 - Extract or Bookmark Duplicate Pages
Optionally, use 'Bookmark Pages' button to bookmark all checked pages. This is useful if you are not planning to delete the found duplicate pages from the document. Use checkboxes in front of the pages to select/unselect them from the processing set.
Use 'Extract Pages.' button to extract all checked pages into a separate PDF document. This operation will not remove pages from the current document.
Use 'Save Report.' button to save page similarity computation report into HTML file. It contains page similarity details, shows differences between pages and lists missing words. It can be very useful for the in-depth analysis.
Step 6 - Delete Duplicate Pages
Use checkboxes in front of the pages to select/unselect pages from being deleted. Press 'Delete Pages' button in the 'Delete Duplicate Pages' dialog to remove all checked pages from the current PDF document:
Click 'OK' button to confirm. Pages will be permanently removed.
Method 2 - Comparing Visual Appearance Only ↑overview
This method compares pages 'as images' and detects pages that look exactly the same. This method does not compare any invisible text that may be present on the page. It is not advised to use this method on scanned paper documents.
Step 1 - Open a PDF File
Start the Adobe® Acrobat® application and open a PDF file using 'File > Open.' menu.
Step 2 - Open the 'Find Duplicate Pages' Dialog
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Step 3 - Specify Settings
Check the 'Compare visual appearance for exact match (can be used to compare images)' option.
Click 'OK' to start searching for duplicate pages.
Step 4 - Inspect Duplicate Pages
The 'Delete Duplicate Pages' dialog shows a list of duplicate or near-duplicate pages. Click on a page record to display the corresponding page in the side-by-side view. Examine pages and select/unselect pages for a possible deletion.
Optionally, click 'Save Report.' to create a page similarity report in HTML format. Or click 'Bookmark Pages' to create bookmarks in PDF for selected duplicate pages.
This method is based on creating a smaller (sampled) copies of the pages and comparing them 'as images'. The following example shows two identical pages that contain only graphics and no searchable text:
If pages are visually identical, then the software detects them as duplicates:
These two pages are considered different due to the 'Approved' stamp on the one of the pages:
These two pages are consider identical by this method:
Unlike the text-based comparison method, if the color or style of the text is different, then the pages are not considered identical:
Step 5 - Delete Duplicate Pages
Click 'Delete Pages' in the 'Delete Duplicate Pages' dialog to proceed.

Duplicate Detective 1 94 – Find And Remove Duplicate Files Pdf

Click 'OK' button to delete pages from the current PDF documents. Pages will be permanently removed.
Comparing Multiple PDF Documents
This operation can be used to find and remove duplicate pages from the multiple PDF documents. The approach is to combine one or more documents into a single PDF file and run 'Find and Delete Duplicate Pages' operation on the resulting file. This will essentially produce a single document without any duplicates. Optionally, it is possible to extract all detected duplicate pages into a separate PDF document.
Step 1 - Combine Multiple PDF Documents ↑overview
Start the Adobe® Acrobat® application and select 'Tools' from the menu. Select 'Combine Files' icon from the Tools list.
Click 'Add Files.' in the 'Combine Files' menu and select PDF files to merge for comparison.
Click the 'Combine' button in the menu to merge selected PDF files.
Step 2 - Find Duplicate Pages
The combined output PDF file would appear on the screen. If not, open the combined PDF file.
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Check the 'Compare visual appearance for exact match (can be used to compare images)' option. Click 'OK' to start searching for duplicate pages.
Duplicate Detective 1 94 – Find And Remove Duplicate Files
Introduction
  • Duplicate File Detective is a feature-rich software application whose purpose is to locate duplicate files on your computer, allowing you to delete them and, thus clear up space on your hard drive.
  • Duplicate Detective – Find and Delete Duplicate Files 1.94 Description AWARDED MACWORLD ‘MAC GEM' 2014 – The Best Duplicate Finder For Mac – On Sale: 40% Off Limited Time Only!
  • Duplicate File Detective uses state-of-the-art file content analysis technology to show you exactly where your duplicate files are (even when the file names are different). You can search multiple folders, drives, removable storage devices, even network shares – all in a single pass. Then move, zip, or remove duplicate files safely with assistance from our innovative SmartMark duplicate file.
This tutorial shows how to find and optionally delete similar or duplicate pages within the same PDF document using the AutoSplit™ plug-in for the Adobe® Acrobat®. This operation detects similar pages and presents them to the user for a review. The user can review the results and select/unselect individual pages from the list of duplicates for a possible deletion or extraction. You can perform the following operations:
  • Find duplicate and near-duplicate pages
  • Bookmark duplicate pages
  • Extract duplicate pages into a separate PDF document
  • Delete duplicate pages from the document
  • Save page similarity report
The plug-in provides two different methods for detecting duplicate or near-duplicate pages:

Duplicate Detective 1 94 – Find And Remove Duplicate Files Onto

Compare Page Text Only
Use this method to compare page text regardless of its visual appearance. It computes page similarity based on text content only and completely ignores text appearance, layout, images and graphics that might be present on the page. It is the best method to detect duplicates in most document types.
Compare Visual Appearance of the Pages
This method compares pages 'as images' and detects pages that look exactly the same. This method does not compare any invisible text that may be present on the page. It is not advised to use this method on scanned paper documents.
Using Scanned Paper Documents
Quite often this operation is used to find duplicate pages in the scanned paper documents. The scanned documents need to be OCRed prior to using them for any text-based processing. The OCR is a process of recognizing text in scanned documents and making them searchable. It is essential to understand that text recognition in scanned documents is prone to errors and it is rarely 100% accurate. The number of errors depends on scanning resolution and original document quality. In most common cases, a scanned page may contain between 1 to 10 recognition errors where certain letters are incorrectly identified. For example, depending on the font, the lowercase letter l can look exactly like the numeral 1 . The uppercase letter O is often misidentified as the numeral 0, or uppercase letter S as the numeral 5 and etc. Since many alphanumeric symbols share similar, or identical, physical characteristics, differentiation often poses a challenge. This is why a similarity-based comparison comes useful to detect small differences between pages that are produced by the text recognition process. Low quality scanned documents may contain a large number of errors making them unusable for any reliable text-based comparison. See the following tutorial on how to OCR scanned documents and asses their suitability for the text-based processing. .
Prerequisites
You need a copy of the Adobe® Acrobat® along with the AutoSplit™ plug-in installed on your computer in order to use this tutorial. You can download trial versions of both the Adobe® Acrobat® and the AutoSplit™ plug-in.
Contents
Method 1 - Comparing Page Text Only ↑overview
This method compares page similarity only based on their page content. The visual appearance, text position and order is irrelevant. This method also ignores any images and graphics present on the pages. The modified cosine similarity metric is used to calculate how similar are two pages based on their text content.
Step 1 - Open a PDF File
Start the Adobe® Acrobat® application and open a PDF file using 'File > Open.' menu.
Step 2 - Open the 'Find Duplicate Pages' Dialog
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Step 3 - Specify Settings
Check the 'Compare only page text (ignore visual appearance of the pages)' option.
Using Predefined Settings
The text-based method provides a number of predefined parameters sets that are suitable for comparing different kinds of documents with a different amount of recognition errors. Each predefined set of parameters provides different conditions for similarity calculations:
  • Custom Settings - all settings are specified by user
  • Scanned Paper Document: High Quality
  • Scanned Paper Document: Medium Quality
  • Fax Document: Low Quality
  • Non-scanned PDF: exact match
  • Non-scanned PDF: fuzzy match
  • Exact match (with text order)- this method does not use cosine similarity
Settings appear below the menu after selecting a predefined parameter set.
Here are the settings used by the predefined sets:
Click 'Edit.' to customize page similarity settings:
The text comparison method uses 3 parameters to limit how different two 'similar' pages can be. By varying these parameters, it is possible to detect pages that have a different degree of similarity.
  • Minimal allowed page text similarity (in percent) - this is the value of cosine similarity metric expressed in percent. Specify minimum allowed page text similarity between 70 and 100 (in percent).
  • Maximum allowed page length difference (in characters).
  • Maximum allowed page text difference (in words).
Use these settings to experiment with processing settings when it is necessary to adjust processing algorithm for a specific document.
Use Sample Pages
Optionally, click 'Set From Page Sample.' to specify page similarity settings based on the two sample pages:
Select two pages that can be consider identical. The software will auto-calculate page similarity and the statistics will appear in the left lower corner of the dialog.
Click 'OK' to save the current similarity settings.
Specify Text Filtering Options
There are several parameters that control the page content that is being analyzed by the text comparison algorithm. Use these options when comparing scanned paper documents that may contain various text recognition errors. These options exclude certain kind of characters from processing. In many cases, it may help to compute a more accurate similarity metric.
  • Ignore text case - this option ignores text case while comparing text.
  • Ignore punctuation (,.!?-) - this options excludes all punctuation characters from comparison.
  • Ignore non-alphanumeric characters - this options ignores all characters except letters and digits.
Click 'OK' to save page similarity settings.
Click 'OK' to start searching the current PDF document for the duplicate pages:
Step 4 - Inspect Duplicate Pages
The 'Delete Duplicate Pages' dialog shows a list of duplicate or near-duplicate pages. Click on a page record to display a corresponding page in the viewer. Examine pages and select/unselect pages for deletion.
Optionally, click 'Save Report.' to create a page similarity report in HTML format. Or click 'Bookmark Pages' to create bookmarks in PDF for selected duplicate pages.
The plug-in allows to preview/compare the found duplicate or near-duplicate pages. The page similarity (in %) and the number of mismatched words is displayed for each pair of pages. Here are the examples computed for the pair of the scanned paper documents:
Note that the apperance and location of the text do not affect the results.
These two pages are considered identical despite the difference in the text color:
These two pages are considered identical despite the difference in the content layout:
These two pages are consider 94% similar despite the difference in the text order, layout and absence of the image:
Step 5 - Extract or Bookmark Duplicate Pages
Optionally, use 'Bookmark Pages' button to bookmark all checked pages. This is useful if you are not planning to delete the found duplicate pages from the document. Use checkboxes in front of the pages to select/unselect them from the processing set.
Use 'Extract Pages.' button to extract all checked pages into a separate PDF document. This operation will not remove pages from the current document.
Use 'Save Report.' button to save page similarity computation report into HTML file. It contains page similarity details, shows differences between pages and lists missing words. It can be very useful for the in-depth analysis.
Step 6 - Delete Duplicate Pages
Use checkboxes in front of the pages to select/unselect pages from being deleted. Press 'Delete Pages' button in the 'Delete Duplicate Pages' dialog to remove all checked pages from the current PDF document:
Click 'OK' button to confirm. Pages will be permanently removed.
Method 2 - Comparing Visual Appearance Only ↑overview
This method compares pages 'as images' and detects pages that look exactly the same. This method does not compare any invisible text that may be present on the page. It is not advised to use this method on scanned paper documents.
Step 1 - Open a PDF File
Start the Adobe® Acrobat® application and open a PDF file using 'File > Open.' menu.
Step 2 - Open the 'Find Duplicate Pages' Dialog
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Step 3 - Specify Settings
Check the 'Compare visual appearance for exact match (can be used to compare images)' option.
Click 'OK' to start searching for duplicate pages.
Step 4 - Inspect Duplicate Pages
The 'Delete Duplicate Pages' dialog shows a list of duplicate or near-duplicate pages. Click on a page record to display the corresponding page in the side-by-side view. Examine pages and select/unselect pages for a possible deletion.
Optionally, click 'Save Report.' to create a page similarity report in HTML format. Or click 'Bookmark Pages' to create bookmarks in PDF for selected duplicate pages.
This method is based on creating a smaller (sampled) copies of the pages and comparing them 'as images'. The following example shows two identical pages that contain only graphics and no searchable text:
If pages are visually identical, then the software detects them as duplicates:
These two pages are considered different due to the 'Approved' stamp on the one of the pages:
These two pages are consider identical by this method:
Unlike the text-based comparison method, if the color or style of the text is different, then the pages are not considered identical:
Step 5 - Delete Duplicate Pages
Click 'Delete Pages' in the 'Delete Duplicate Pages' dialog to proceed.

Duplicate Detective 1 94 – Find And Remove Duplicate Files Pdf

Click 'OK' button to delete pages from the current PDF documents. Pages will be permanently removed.
Comparing Multiple PDF Documents
This operation can be used to find and remove duplicate pages from the multiple PDF documents. The approach is to combine one or more documents into a single PDF file and run 'Find and Delete Duplicate Pages' operation on the resulting file. This will essentially produce a single document without any duplicates. Optionally, it is possible to extract all detected duplicate pages into a separate PDF document.
Step 1 - Combine Multiple PDF Documents ↑overview
Start the Adobe® Acrobat® application and select 'Tools' from the menu. Select 'Combine Files' icon from the Tools list.
Click 'Add Files.' in the 'Combine Files' menu and select PDF files to merge for comparison.
Click the 'Combine' button in the menu to merge selected PDF files.
Step 2 - Find Duplicate Pages
The combined output PDF file would appear on the screen. If not, open the combined PDF file.
Select 'Plug-Ins > Split Documents > Find and Delete Duplicate Pages.' to open the 'Find Duplicate Pages' dialog.
Check the 'Compare visual appearance for exact match (can be used to compare images)' option. Click 'OK' to start searching for duplicate pages.
Step 3 - Extract Duplicate Pages
The 'Delete Duplicate Pages' dialog will show a list of duplicate or near-duplicate pages. Click on a page record to display a corresponding page in the viewer. Examine pages and select/unselect pages.
Click 'Extract Pages.' to extract selected duplicate pages into a new PDF document.
Specify an output folder and a file name. Click 'Save' once done.
The dialog would appear showing the number of pages that have been extracted into a separate document. Now you have saved all duplicate pages into the separate PDF file before deleting them. You can examine these pages and use later if necessary.
Click 'OK' to close the dialog.
Step 4 - Delete Duplicate Pages
Click 'Delete Pages' in the 'Delete Duplicate Pages' dialog to proceed.
Click 'OK' in the dialog to delete selected duplicate pages from the current PDF document.
The selected duplicate pages would be permanently removed from the PDF document. You would need to use 'File > Save' menu to save the modified document to disk.
Click here for a list of all step-by-step tutorials available.
  • Products
    All-In-One Products
    Wise Care 365
    Data & Security Products
    Wise Anti Malware
    Wise Data Recovery
    Wise Folder Hider Free
    Wise Folder Hider Pro
    Performance Products
    Wise Registry Cleaner
    Wise Disk Cleaner
    Wise Program Uninstaller
    Wise Memory Optimizer
    Wise Game Booster
    Wise System Monitor
    Wise PC 1stAid
    Wise Plugin Manager
    Utility Tools
    Wise Windows Key Finder
    Wise Duplicate Finder
    Wise Hotkey
    Wise Reminder
    Wise JetSearch
    Wise Auto Shutdown
    Wise Force Deleter
    Multimedia Tools
    Wise YouTube Downloader
    Wise Video Converter
    Wise Video Player
  • Support
    Support Center
    FAQs
    Lost License
    Renew License
    Transfer License
    Feedback
    Product Manuals
    Forgot WFH Login Pasword
    Language
    Languages
    WiseCleaner Forum
    News & Offers
    WiseCleaner Advices & Suggestions
    Wise Care 365
    Wise Registry Cleaner
    Wise Disk Cleaner
    Other WiseCleaner Products
  • Articles
    How-to
    Restore Files with Deep Scan
    Fix Windows 10 Slow Boot-up Issue
    Remove Windows 10 Built-in Apps
    Find and Remove Duplicate Photos
    Protecting Sensitive Files
    Assign Keyboard Shortcut to Launch App
    Set a Reminder in Windows 10
    >> More How-To Articles
    Windows Tips
    Disable Preloading Microsoft Edge at Startup
    High Memory Usage Issue about EoAExperiences.exe
    Launch Emoji Keyboard in Windows 10
    Start Windows 10 in Safe Mode
    God Mode of Windows 10
    Add Windows Photo Viewer in Windows 10
    Prevent PUA with Windows Defender
    >> More Windows Tips




broken image
PreviousNext
Cookie Use
We use cookies to improve browsing experience, security, and data collection. By accepting, you agree to the use of cookies for advertising and analytics. You can change your cookie settings at any time. Learn More
Accept all
Settings
Decline All
Cookie Settings
Necessary Cookies
These cookies enable core functionality such as security, network management, and accessibility. These cookies can’t be switched off.
Analytics Cookies
These cookies help us better understand how visitors interact with our website and help us discover errors.
Preferences Cookies
These cookies allow the website to remember choices you've made to provide enhanced functionality and personalization.
Save