All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: diglloyd.com photography and WindInMyFace.com

Links on this site earn me fees or commissions.
As an Amazon Associate I earn from qualifying purchases @AMAZON

Consult with Lloyd: cameras, computers, backup, etc...
Lloyd’s Patreon
Designed for the most demanding needs of photographers and videographers.
The fastest, toughest, and most compatible portable SSD ever with speeds up to 2800MB/s.

IntegrityChecker (icj) Java Release 3.0 fc35

re: IntegrityChecker Java
re: DiskTester Java
re: data integrity

Consult with Lloyd to design a storage and/or backup system and/or high performance workflow.

Change history below...

You do not have a real backup until you have a verified backup.

That means all the data can be read (a physical check on the storage medium) and is cryptographicallly proven to be intact/unchanged with no files or folders missing.

No photographer or videographer or other professional should be operating without data integrity validation. Whether bit rot or malware or software bugs or hardware problems, can you afford to remain unaware of data corruption, or to not know with provable certainty whether your backups (and originals!) are intact and undamaged?

IntegrityChecker Java (icj) supports Mac, Windows, Linux, etc—anything with Java, an unrivalled cross-platform data integrity solution.

Download page for existing customers.

Get IntegrityChecker Java

Or keep guessing about whether your backups suck.

What’s changed

See also release notes for fc34.

This release makes minor changes:

  • 3.0fc35
  • 2023-11-26 icj installer now updates both .bash_profile and .zprofile to include icj_aliases.sh. The following shortcuts are now available for less typing when using icj (quit and restart terminal after installing):
            up = icj update
            ver = icj verify
            st = icj status
            sn = icj sync
            dupes = icj dupes
            mat = icj matches
            inf = icj info
            cl = icj clean
            clall = icj clean --kind = all
            clattr = icj clean --kind = attr
    2023-11-17 tj: improved fixfolders command to set dates depth-first so that parent folders whose dates are also being updated are correctly set.
    2023-11-15 tj: improved fixfolders command to ignore certain hidden files for purpose of determining newest file eg ".DS_Store", ".icjh", ".icj", ".ic"
    2023-11-05 icj: modified 'dupes' command to be smarter about picking the primary/most likely original
    2023-11-05 icj: fixed bug in 'dupes' command that would not properly remove a duplicate in all cases
    2023-11-05 icj: fixed a minor bug where the volume name was not recognized as "/"
    2023-11-05 icj: fixed a bug where 'dupes' command failed to emit any commands to deal with the duplicates
  • 3.0fc34
    2023-09-22 updated installJava.sh to use Java 21 final GA release
    2023-09-14 updated documentation for 'matches' command
    2023-09-14 Fixed 'matches' command to work for all documented flags
    2023-09-14 added 'caffeinate' command to invocation, so that icj will not allow system sleep while running.
    2023-09-06 fixed installer bug that failed to copy-over 'dtj', 'img', and 'tj'.
    2023-08-27 'matches' command now shows [missingfolders.ignore] matches.
    2023-08-26 Added missingfolders.ignore feature.
    2023-08-23 2023-08-23 install-icj.sh now preserves (does not replace) icj_prefs.txt
  • 3.0fc32
    2023-08-03 icj: added report.MATCHED_HASH_MIN_SIZE option to prefs. It applies a minimum size cutoff as to whether missing files with matching hashes are reported. Values can be of the form "1234" or "32K" or "2M", etc eg "report.MATCHED_HASH_MIN_SIZE=128K"
    2023-08-01 icj: fixed bug where missing folders would report incorrect file count, subfolder counter, byte count for missing folders in the intermediate listing.
    2023-08-01 icj: orphaned files (files whose parent folder is also missing) are now NOT listed by default.
    The --orphans option must be used to list the files.
    2023-07-29 dtj: fixed bug where --numbuf option was not being used
  • 3.0fc31
    2023-07-28 The installJava.sh script now installed JDK 21ea on Apple Silicon Macs, delivering a 4X hashing speed increase on Apple Silicon Macs.
    2023-07-25 In addition to 'jattr', new tools now include 'img' (Imaging Java), 'tj' (Tools Java), 'dtj' (DiskTester Java). Available commands include wipefiles, wipefree, fixdates (tj) fixdates from EXIF (img), aseries, stacks. These new tools are usable, but exploratory, and most lack documentation (yet).
    2023-07-23 Given the new considerations of super-fast hashing and super-fast SSDs over 20GB/sec, the --numbuf and --bufsize options have been restored for command line options.
    2023-07-22 Added optimization for super-fast SSDs on Apple Silicon via larger buffers up to 32MB on M2 Ultra (up from only 1MB). Raises peak throughput from ~17GB/sec to ~21GB/sec on M2 Ultra and from ~7GiB/sec to ~9GiB/sec on M1 Max. Optimization only applies to JDK 21 or later, which has 4X faster SHA512 hashing speed.
    023-07-22 Improved performance of initial scan by about 5-8% by using JNA native function mapping.
    2023-07-20 If a corrupted hexadecimal hash string is detected, a CorruptedHashEntryException is thrown which makes the issue more understandalone. This arose in a case where a SAN was corrupted hash entries (and other data).
  • 3.0fc30
    2023-07-19 Fixed ICJ_CLASSPATH path in icj.bat
    2023-07-18 'sha' command: added per cpu figure to command output.
    2023-07-18 'sha' command: changed to do all CPU counts up to maximum real CPU count, not every-other-past-10.
    2023-07-18 'sha' command: Increased maximum --size value to 128M, added existing --threads option to user manual
    2023-07-02 bug fix: the 'clean' command without the --kind argument was cleaning only icj_temp. Now it cleans {ic,icj,icj_temp} by default (without the --kind argument), doing what the user manual states.
    2023-07-01 Improved message behavior with non writeable folders.
    2023-07-01 Added **/private/{var,etc,tmp,tftpboot}/** to [folders.ignore] to better deal with files owned by 'root' within backups of boot drive
    2023-07-01 Removed assertion that was barfing for folders owned by root.
    2023-06-30 Improved error handling when a regular folder is replaced by a symbolic link.
  • 3.0fc28
    2023-06-30 Fixed bug where icj would fail to finish any legacy 'ic' file of 16K or smaller in size, losing an I/O buffer, thus quickly running out of buffers, halting progress and waiting forever for hashing to finish.
  • 3.0fc27
    2023-06-27 FYI early access JDK 21 offers 4X the hashing speed on Apple Silicon. For example, a single CPU can now do ~1500MB/sec on an M1 Max.
    2023-06-19 Fixed a race condition that resulted in "... java.lang.AssertionError "finalizeHashesNew: mData_SHA512_64K_CHAINED not finished...". When this happened, icj would not complete at the very end ("waiting for...").
    2023-06-19 Inserted warning when buffers are unobtainable for 15 seconds or more to help detect hanging I/O system.
    2023-06-16 Fixed bug where DATE_CHANGED was issued when date really had not changed, causing files to be unnecessarily hashed. This stemmed from mismatched milliseconds portions of file modification date.
    2023-06-16 A new utility 'jattr' is now included, a more useful/powerful tool than Apple’s xattr command, and runs on both macOS and Linux.
  • 3.0fc23
    2023-04-04 Fixed bug where prefs files were being ignored for the various flags, introduced around fc18.
    2023-04-04 File status messages are now aligned visually with mixed-length tag like RENAMED vs MOVED*
    2023-04-04 SYSTEM_SLEEP advisory rejiggered to SYSTEM_UNRESPONSIVE with exact times and the exact unexpected delay.
  • 3.0fc20
    2023-03-30 All outstanding issues with normalized file paths now resolved. No more missing files if normalization varies, no warnings should appear, cross compatibility across JDK20 vs earlier JDKs as well as APFS/HFS, and in all combinations thereof. The changes should apply across operating system foibles as well.
  • 2023-03-30 It is recommended that all macOS users move to JDK20 by running the java installer. That’s because JDK20 is the first version to not forcibly normalize file paths to NDF, a behavior that could cause issues across file system. With JDK20, file paths that icj sees are normalized the same as in the native file system.
View all handpicked deals...

Samsung 2TB T9 Portable SSD
$250 $150
SAVE $100

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | X.com/diglloyd
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__