I am trying to run cryosparc on simulated images, and have successfully converted them from png to mrc, and tif to mrc (using EMAN2). I also can successfully import the images and see them in the workspace. However, when I try to do a manual pick (I turn off the CTF, because there is no CTF in the images), I get an error that it could not get interactive information to set up the job, and that it could not get the image for the micrograph. Can anyone help me understand exactly what kind of info needs to be present in my simulated images? I am imagining it has something to do with the mrc headings and columns, but I am not quite sure how to find out what is required and what is missing from my images.
Can’t help with what might be missing from your simulated images, but I like TEM Simulator to simulate micrographs. Only issue is that it’s pretty slow.
As a sanity check, can the simulated micrographs be viewed correctly by EMAN2, or RELION (or any other package)?
@renee Please can you confirm that you were able to manually pick particles in other projects/on other inputs on this particular CryoSPARC installation. Also, did you try a CTF job just for the diagnostic information that non-interactive job could provide about the “suitability” of the image format?
Is the image you’ve simulated a particle? If you are manually picking, then a particle is not useful. You would need to supply micrographs. What you might be trying to do is pick particles from normal input micrographs using a simulated 2D template of a model of your protein? For that, 1) import 3D volume, 2) create templates, 3) 2D select a few templates 4) template pick your normal micrographs.
Just a guess from supplied info… Try to be very clear about what “run cryosparc on simulated images” means, and what you are giving as inputs to the job that failed.
========= monitor process now starting main process at 2023-08-04 11:59:46.586112
MAINPROCESS PID 2182824
========= monitor process now waiting for main process
MAIN PID 2182824
interactive.run_manual_picker_v2 cryosparc_compute.jobs.jobregister
INTERACTIVE JOB STARTED === 2023-08-04 11:59:50.204982 ==========================
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: off
[EXTERN got get_interactive_info 2023-08-04 18:59:52.044885 ]
[2023-08-04 11:59:52,046] ERROR in app: Exception on /get_interactive_info [POST]
Traceback (most recent call last):
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 2447, in wsgi_app
response = self.full_dispatch_request()
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/_compat.py”, line 39, in reraise
raise value
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1950, in full_dispatch_request
rv = self.dispatch_request()
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/interactive_common.py”, line 36, in wrapper
res = func(*args, **kwargs)
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/run_manual_picker_v2.py”, line 374, in get_interactive_info
thumb_shape = get_thumb_shape()
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/run_manual_picker_v2.py”, line 304, in get_thumb_shape
clipx = (nx//binfactor)
ZeroDivisionError: integer division or modulo by zero
[EXTERN got get_micrograph_data 2023-08-04 18:59:52.104264 ]
/home/user/cryosparc/cryosparc_master/cryosparc_compute/micrographs.py:400: RuntimeWarning: divide by zero encountered in true_divide
return trim_mic(arr_zp_lp, out_shape) / trim_mic(ones_zp_lp, out_shape)
[2023-08-04 11:59:52,155] ERROR in app: Exception on /get_micrograph_data [POST]
Traceback (most recent call last):
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 2447, in wsgi_app
response = self.full_dispatch_request()
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/_compat.py”, line 39, in reraise
raise value
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1950, in full_dispatch_request
rv = self.dispatch_request()
File “/home/user/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.8/site-packages/flask/app.py”, line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/interactive_common.py”, line 52, in wrapper
res = func(*args, **kwargs)
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/run_manual_picker_v2.py”, line 435, in get_micrograph_data
return read_mrc_image(path_rel, psize, lp_filter_A, vmin, vmax)
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/jobs/interactive/run_manual_picker_v2.py”, line 341, in read_mrc_image
arr_bin = mg.bin_mic(arr, binfactor)
File “/home/user/cryosparc/cryosparc_master/cryosparc_compute/micrographs.py”, line 533, in bin_mic
clipx = (nx//binfactor)
ZeroDivisionError: integer division or modulo by zero
get_micrograph_data key micrograph_blob_denoised
read_mrc_image /mnt/hdd2/P/_projections_for_cryoSparc/CS-projections/J7/imported/009798482795944560034_10_object_rot_angle_0.0_0.0_degrees_Inf_eV.mrc
[EXTERN got get_intensity 2023-08-04 18:59:52.156381 ]
[EXTERN done get_intensity 2023-08-04 18:59:52.156397 0.00s ]
[EXTERN got get_picks 2023-08-04 18:59:52.156718 ]
[EXTERN done get_picks 2023-08-04 18:59:52.156734 0.00s ]
========= sending heartbeat at 2023-08-04 11:59:58.551209
========= sending heartbeat at 2023-08-04 12:00:08.564282
Hi @CryoEM2, is it possible to generate volumes from simulated particles? What I want to do is start after the particle extraction step with my simulated particles, and do a reconstruction.
No I don’t think that is possible. If you had 10’s of thousands (say 50,000 images) and they represented many different unique views of the 3D volume, then it might be possible to have them generate a volume. But angular assignment of particles is not easy, even in ideal cases.
I would guess there is a misunderstanding, or you should find some literature precedence for the exercise, at least to help guide troubleshooting
If you know the parameters you simulated with (defocus, astigmatism, angle of projection), “dumb” reconstructions can be carried out (e.g. in RELION with relion_reconstruct) by treating it like a pre-refined particle stack, but I’m not sure how current cryo-EM packages handle noise-free simulated data.
Again, it might be better to simulate micrographs using TEM Simulator, as I suggested above, then you can work through the whole pipeline as a data sanity check.
I can generate as many unique views of the 3D volume as required, so I am not limited in that case. I am tripping up on uploading these particles to cryosparc. I think I should not have been attempting to upload them as images, but perhaps can find a way to upload them as particles, but am unsure of how to proceed. I can certainly look for literature to help guide me.
relion_star_loopheader and relion_star_datablock_stack, will allow you to create a metadata file describing your particles. But you will need to be able to complete defocus and other critical parameters yourself, potentially on a per-particle basis.
But they’re basically just bash scripts, you could write it by hand if necessary.
