Hi everyone,
I’ve encountered an issue where the level values in a map downloaded from the CryoSPARC website (followed picture,left) are significantly different from the original map file located in the job directory(followed picture,right).
I’m using CryoSPARC version 4.5.3 and have tried downloading the map using both Chrome and Firefox, but the issue persists. Has anyone know the reason?
It may be that the map download via the UI was truncated in a way that did not prevent the viewing program from displaying the truncated file.
Please can you post the outputs of the commands
ls -l mapfile.mrc
shasum mapfile.mrc
for both versions of the map and email us the tgz file created by the command
cryosparcm snaplogs.
Thanks for your reply and this is the output of the commands:
ls -l J595_013_volume_map.mrc
-rw-rw-r-- 1 jclabem jclabem 442369024 11 16 13:06 J595_013_volume_map.mrc
shasum J595_013_volume_map.mrc
084da8a7685f5ea5afa126c486a13c06fcf9fa22 J595_013_volume_map.mrc
ls -l cryosparc_P1_J595_013_volume_map.mrc
-rw-rw-r-- 1 spuser spuser 442369024 Nov 25 20:01 cryosparc_P1_J595_013_volume_map.mrc
shasum cryosparc_P1_J595_013_volume_map.mrc
71cb94a43dd36eea59370c29c47beedd24e552b6 cryosparc_P1_J595_013_volume_map.mrc
I have emailed the tgz file and please let me know if further infomration required.
Thanks @tosyl . The file names in your latest post differ from those in the screenshot from Inconsistent Map Level Values Between Download and Local File. Do you observe inconsistencies within each pair of map files, where one file was downloaded via the web UI, the other directly from the job directory?
Please can you post the outputs of these commands
cryosparcm cli "get_job('P1', 'J213', 'job_type', 'version', 'instance_information', 'status', 'params_spec', 'errors_run', 'created_at', 'started_at')"
cryosparcm cli "get_job('P1', 'J595', 'job_type', 'version', 'instance_information', 'status', 'params_spec', 'errors_run', 'created_at', 'started_at')"
Thanks @wtempel .
Do you observe inconsistencies within each pair of map files
Yes, the map files downloaded via the UI differ from their corresponding files in the job directory.
Here is the outputs:
$ cryosparcm cli "get_job('P1', 'J213', 'job_type', 'version', 'instance_information', 'status', 'params_spec', 'errors_run', 'created_at', 'started_at')"
{'_id': '67317b814e86c7a434a3459c', 'created_at': 'Mon, 11 Nov 2024 03:35:29 GMT', 'errors_run': [], 'instance_information': {'CUDA_version': '11.8', 'available_memory': '441.96GB', 'cpu_model': 'Intel(R) Xeon(R) Platinum 8468V', 'driver_version': '12.4', 'gpu_info': [{'id': 0, 'mem': 25386352640, 'name': 'NVIDIA GeForce RTX 4090', 'pcie': '0000:49:00'}], 'ofd_hard_limit': 51200, 'ofd_soft_limit': 1024, 'physical_cores': 96, 'platform_architecture': 'x86_64', 'platform_node': 'gpu1', 'platform_release': '5.15.0-94-generic', 'platform_version': '#104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024', 'total_memory': '503.52GB', 'used_memory': '57.65GB'}, 'job_type': 'nonuniform_refine_new', 'params_spec': {'compute_use_ssd': {'value': False}, 'crg_do_anisomag': {'value': True}, 'crg_do_spherical': {'value': True}, 'crg_do_tetrafoil': {'value': True}, 'refine_ctf_global_refine': {'value': True}}, 'project_uid': 'P1', 'started_at': 'Mon, 11 Nov 2024 04:11:26 GMT', 'status': 'completed', 'uid': 'J213', 'version': 'v4.6.0'}
$ cryosparcm cli "get_job('P1', 'J595', 'job_type', 'version', 'instance_information', 'status', 'params_spec', 'errors_run', 'created_at', 'started_at')"
{'_id': '673807dc4c41e373a06f1395', 'created_at': 'Sat, 16 Nov 2024 02:47:56 GMT', 'errors_run': [], 'instance_information': {'CUDA_version': '11.8', 'available_memory': '947.48GB', 'cpu_model': 'Intel(R) Xeon(R) Platinum 8468V', 'driver_version': '12.4', 'gpu_info': [{'id': 0, 'mem': 47576711168, 'name': 'NVIDIA L40', 'pcie': '0000:29:00'}], 'ofd_hard_limit': 51200, 'ofd_soft_limit': 51200, 'physical_cores': 96, 'platform_architecture': 'x86_64', 'platform_node': 'gpu31', 'platform_release': '5.15.0-94-generic', 'platform_version': '#104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024', 'total_memory': '1007.52GB', 'used_memory': '53.52GB'}, 'job_type': 'nonuniform_refine_new', 'params_spec': {}, 'project_uid': 'P1', 'started_at': 'Sat, 16 Nov 2024 02:49:10 GMT', 'status': 'completed', 'uid': 'J595', 'version': 'v4.5.3'}
@tosyl Thanks for these details. Please could you share one inconsistent pair of map files with us so that we may inspect more closely. Ideally, one of the pair’s files should be have bee downloaded recently/now and the files should be accomapined by a freshly created tgz logs archive from the command
cryosparcm snaplogs
to ensure the logs include the time window when the file was downloaded via the web UI.
If you can, please share the files with us privately via your institution’s file sharing service. Otherwise, we can make arrangements for you to upload the files. Please let us know.
I have prepared these three files and sent you a private message with the details, please find the link and download them. Thanks for your help @wtempel !
@tosyl May we ask for additional information that may help us troubleshoot. Please:
uname -a
free -h
cat /proc/1/sched | head -1
cat /sys/fs/cgroup/memory.high
grep "$(df $(cryosparcm cli "get_project_dir_abs('P1')") | tail -n 1 | awk '{print $NF}') " /proc/mounts
$ uname -a
Linux login 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
$ free -h
total used free shared buff/cache available
Mem： 503Gi 72Gi 132Gi 36Mi 298Gi 427Gi
Swap： 31Gi 61Mi 31Gi
$ cat /proc/1/sched | head -1
systemd (1, #threads: 1)
$ cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
486843453440
$ grep "$(df $(cryosparcm cli "get_project_dir_abs('P1')") | tail -n 1 | awk '{print $NF}') " /proc/mounts
netdata_hdd_hdd_fs /public storage rw,noatime 0 0
Hope these can help!
Thanks @tosyl for sending the job report and posting the information.
Please can you also post