I’m evaluating a ZFS strategy for a storage pool running on spinning HDDs that is used exclusively for CryoSPARC data processing.
Because CryoSPARC projects generate a massive amount of small metadata files alongside huge movie stacks, I am considering adding an NVMe Mirror as a special allocation class vdev (blocks=128k).
The goal is to offload all directory metadata and small files (\\le 128 KB) to the NVMe vdev, leaving the spinning disks to handle only the large bulk data files.
My questions:
-
Given that nearly ~50% of files take up less than 0.1% of our pool space, will offloading metadata and files \\le 128 KB to NVMe significantly boost GUI responsiveness and job startup times?
-
Is
blocks=128kthe right threshold for CryoSPARC, or would you recommend a different value like64kor256k?