Hello, I have tried correcting the same movies in cryoSPARC and cryoSPARC live. Interestingly, the movies correct just fine (without “Low Memory Mode”) in standard cryoSPARC, but fail even with “Low Memory Mode” turned on in cryoSPARC live.
Movie information:
Parameter | Value
-------------- | -----
Raw Pixel Size | 0.43 A/pix
F-crop | 2
Frames | 65
Total Dose | 65 e/A2/frame
Error from cryoSPARC live:
[CPU: 12.27 GB] Traceback (most recent call last):
File "cryosparc_worker/cryosparc_compute/jobs/rtp_workers/run.py", line 376, in cryosparc_compute.jobs.rtp_workers.run.rtp_worker
File "cryosparc_worker/cryosparc_compute/jobs/rtp_workers/run.py", line 439, in cryosparc_compute.jobs.rtp_workers.run.process_movie
File "cryosparc_worker/cryosparc_compute/jobs/rtp_workers/run.py", line 585, in cryosparc_compute.jobs.rtp_workers.run.do_patch_motion
File "cryosparc_worker/cryosparc_compute/jobs/rtp_workers/run.py", line 590, in cryosparc_compute.jobs.rtp_workers.run.do_patch_motion
File "cryosparc_worker/cryosparc_compute/jobs/motioncorrection/patchmotion.py", line 255, in cryosparc_compute.jobs.motioncorrection.patchmotion.unbend_motion_correction
File "cryosparc_worker/cryosparc_compute/jobs/motioncorrection/patchmotion.py", line 606, in cryosparc_compute.jobs.motioncorrection.patchmotion.unbend_motion_correction
File "cryosparc_worker/cryosparc_compute/engine/cuda_core.py", line 353, in cryosparc_compute.engine.cuda_core.EngineBaseThread.ensure_allocated
File "/home/cryosparc/cryosparc_worker/deps/anaconda/envs/cryosparc_worker_env/lib/python3.7/site-packages/pycuda/gpuarray.py", line 210, in __init__
self.gpudata = self.allocator(self.size * self.dtype.itemsize)
pycuda._driver.MemoryError: cuMemAlloc failed: out of memory
Installation information:
cryosparc@troll:/home/users/posert$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/troll/scratch/cryosparc/cryosparc_master
Current cryoSPARC version: v3.3.1+211214
----------------------------------------------------------------------------
CryoSPARC process status:
app RUNNING pid 3144, uptime 7 days, 22:05:20
app_dev STOPPED Not started
command_core RUNNING pid 2562, uptime 7 days, 22:05:30
command_rtp RUNNING pid 2790, uptime 7 days, 22:05:27
command_vis RUNNING pid 2752, uptime 7 days, 22:05:28
database RUNNING pid 2121, uptime 7 days, 22:05:33
liveapp STOPPED Not started
liveapp_dev STOPPED Not started
webapp RUNNING pid 3116, uptime 7 days, 22:05:22
webapp_dev STOPPED Not started
----------------------------------------------------------------------------
License is valid
----------------------------------------------------------------------------
global config variables:
export CRYOSPARC_LICENSE_ID={id}
export CRYOSPARC_MASTER_HOSTNAME="troll"
export CRYOSPARC_DB_PATH="/troll/scratch/cryosparc/cryosparc_database"
export CRYOSPARC_BASE_PORT=39000
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_INSECURE=false
export CRYOSPARC_CLICK_WRAP=true
cryosparc@troll:/home/users/posert$ nvidia-smi
Tue Mar 8 14:04:18 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.29.05 Driver Version: 495.29.05 CUDA Version: 11.5 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:05:00.0 Off | N/A |
| 20% 35C P8 2W / 215W | 876MiB / 7974MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... On | 00000000:06:00.0 Off | N/A |
| 20% 34C P8 1W / 215W | 10MiB / 7982MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce ... On | 00000000:09:00.0 Off | N/A |
| 20% 33C P8 6W / 215W | 10MiB / 7982MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA GeForce ... On | 00000000:0A:00.0 Off | N/A |
| 20% 28C P8 19W / 215W | 10MiB / 7982MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+