All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: photography and

Links on this site earn me fees or commissions.
As an Amazon Associate I earn from qualifying purchases @AMAZON

Other World Computing...
B&H Photo...
Designed for the most demanding needs of photographers and videographers.
The fastest, toughest, and most compatible portable SSD ever with speeds up to 2800MB/s.

IntegrityChecker (icj) Java Release 3.0 fc20

re: IntegrityChecker Java
re: data integrity

Consult with Lloyd to design a storage and/or backup system and/or high performance workflow.

See previous notes on IntegrityChecker 3.0fc17.

IntegrityChecker Java (icj) supports Mac, Windows, Linux, etc—anything with Java, an unrivalled cross-platform data integrity solution.

No photographer or videographer or other professional should be operating without data integrity validation. Whether bit rot or malware or software bugs or hardware problems, can you afford to remain unaware of data corruption, or to not know if your backups are intact and undamaged?

Get IntegrityChecker Java

icj version 3.0fc20 (version 3.0, final candiate 20)

As before, tested and working on macOS Ventura. Download page for existing customers.

A new utility 'jattr' is now included, a more useful/powerful tool than Apple’s xattr command, and runs on both macOS and Linux.

  • 3.0fc20
    2023-03-30 All outstanding issues with normalized file paths now resolved. No more missing files if normalization varies, no warnings should appear, cross compatibility across JDK20 vs earlier JDKs as well as APFS/HFS, and in all combinations thereof. The changes should apply across operating system foibles as well.
  • 2023-03-30 It is recommended that all macOS users move to JDK20 by running the java installer. That’s because JDK20 is the first version to not forcibly normalize file paths to NDF, a behavior that could cause issues across file system. With JDK20, file paths that icj sees are normalized the same as in the native file system.
  • 3.0fc19
    2023-03-28 This change vs fc18 fixes issues with different JVM versions and/or different file systems that encode folder or file names differently. The ANOMALY messages in 3.0fc18 should now be quieted, and using different JVMs (even alternating) should not matter.
  • 3.0fc18
    2023-03-28 NOTE: jdk 20 (and perhaps 19) process file names with 16-bit characters (accents, umlaut, etc) differently causing ANOMALY_FILENAME_CHANGED_CASE (for example: ANOMALY_FILENAME_DIFFERS: FileHashInfo: name mismatch, expecting "2019-0718-Jérôme.pdf" got "2019-0718-Jérôme.pdf") message. The names *look* identical but are actually encoded differently which results in this warning. These messages are harmless and will be emitted the first time an 'update' or other command is run with the JDK 20 or whenever this is a mismatch vs prior JDK.
    2023-03-28 macOS security bug workaround: updated script to remove quarantine flags to allow java to run, eg "openjdk-20-x866.jdk is damaged and can't be opened".
    2023-03-28 Updated script to install OpenJDK 20.
    2023-03-26 'compare' command is deprecated until/unless it can be shown to add value over status command. 'status' is more complete and also locates missing or moved files, whereas compare does not.
    2023-03-26 Fixed bug in compare command that threw exception when certain differences are present.
  • 2023-03-20 Added -XX:+AlwaysPreTouch option, can sometimes improve performance by a few percent.
    2023-03-09 upgraded icj to JNA 5.13.0.
  • 2023-03-09 jattr now suppresses warnings for inaccessible system folders eg ".Trashes".
    2023-03-03 jattr was failing for very large attributes; fixed.
    2023-03-03 jattr was failing for very large attributes; fixed.
    2023-03-03 jattr was emitting text as binary if it contained newline characters, fixed.
    2023-03-03 jattr now summarizes with "Items listed/matched" as well.
    2023-03-03 jattr remove now gives summary and does a nicer job showing status along the way
  • 2023-03-03 When showing non-writeable files with 'info' command, missing files are now listed.
    2023-03-03 Changed behavior when folders are newly 'ignored'. Now, a CLEAN_SUGGESTED warning is issued along with the commands to clean the folders in question.
  • 3.0fc17
    2023-03-02 Added jattr tool
    2023-02-25 Improved wording/output of cleaning when attributes are being cleaned.
    2023-02-25 Fixed bug when cleaning --kind=lattr ; files were not being cleaned only folders.
    2023-02-23 Fixed newly introduced bug in which end summary would say "0 existing...".
    2023-02-23 added "lattr" kind for clean as in "icj clean --kind=lattr" in order to remove legacy attribute "user.diglloyd.icj.ID#N". Code revised to use same code for that and other attribute cleaning.
  • 3.0fc16
    2023-02-22 Fixed filepath gray portion for some cases.
    2023-02-22 Moved folder with non-matching inode is now MOVED* instead of MATCHED_ID
    2023-02-21 modified output from 'matches' to give totals for matched files and folders for [output.suppressInnocuousChanges]. Also modified to not have misleading count for the items, but retained this count for other matching preference groups.
    2023-02-21 change portion of 'from' path is now grayed-out for RENAMED (eg the name part) and MOVED (whatever path part is affected).
    2023-02-21 changed ouput.* to report.* eg report.INNOCUOUS_FILE_CHANGES, report.NEW_FILES, report.INNOCUOUS_MOVED
    2023-02-20 fixed bug where output.INNOCUOUS_MOVED matches were being made on file name instead of the whole file path
    2023-02-20 renamed warn flags to output.LIST_NEW_FILES, output.INNOCUOUS_MOVED, output.MISSING_FILES_IN_MISSING_FOLDER
    2023-02-20 changed FOUND_ID to MATCHED_ID
  • 3.0fc13
    2023-02-18 Strengthened support for folder and file IDs. Originals are noted as MOVED/RENAMED, copies as MOVED*, and 2 or more copies as MOVED*_CNT.
    2023-02-18 Fixed a nasty bug due to recent change from user.diglloyd.icj.ID#N to user.diglloyd.icj.ID#S in which the ID was not being updated if an ID already existed. The (false even before) working assumption has been that user.diglloyd.icj.ID#N did not propagate. But this assumption was always false (cloning backup programs, Linux). Now the code always checks if the ID is up to date even if an ID already exists. This deals with both copying of files/folders on the same volume as well as to other volumes.
    2023-02-18 Moved folders are now handled better; exact inode match results in MOVED or RENAMED. Match by ID inode results in MOVED_ID.
    2023-02-18 Renamed warn.MOVED_BUT_SAME_PARENT_FOLDER to warn.INNOCUOUS_MOVED.
    2023-02-18 NEW files are now listed for status, update, verify, with option to suppress via warn.LIST_NEW_FILES default true.
  • 3.0fc12
    2023-02-18 'matches' command now appends a trailing "/" to folders, to distinguish from files.
    2023-02-18 For [output.suppressInnocuousChanges], fixed bug in which 'matches' was showing only matched folders, but not files
    2023-02-14 The info command now lists non-writeable files in sorted order. Ditto for empty files.
    2023-02-13 Changed user.diglloyd.icj.ID#N to user.diglloyd.icj.ID#S so as to preserve file/folder IDs for purpose of locating missing files even when copied or backed-up to another volume. To remove the old attr, use xattr -dvr "user.diglloyd.icj.ID#N" FolderName
    2023-02-12 Implemented warn.MOVED_BUT_SAME_PARENT_FOLDER defaults to false. Suppresses detailed file listings of files and folders that now have different paths, but are in the same parent folder as before. By so doing, only the top-level moves are called-out, greatly reducing clutter for very simple changes like moving a folder hierarchy elsewhere. Applies only for systems supporting file/folder IDs. Files without IDs (eg locked files) will still be noted because of matching hash values.
  • 2023-02-01 Subfolders of a missing parent folder are no longer listed.
    2023-02-01 Fixed issue where stale and missing attribute counts were not totalled, resulting in zero count in the summary section.
    2023-01-23 Fixed issue where a filename that changed character encoding would result in "AssertionError: FileHashInfo: name mismatch...". Now the code checks if the file name compares as equals() as a Path instead of a String.3.0fc10
    2023-01-23 Missing files is now red if there are missing files.
    2023-01-23 removed "*** see Missing Folders listing above ***" from Missing Folders line.
View all handpicked deals...

Pentax 8x43 Z-Series ZD ED Binoculars
$849 $699
SAVE $150 | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__