Extract from micrographs shows neither error nor output

[Diane]

Dear all,

I recently had an issue with one extraction job and was wondering if someone could help.

I did process two of my datasets until ab-initio reconstruction and wanted to merge the datasets. But because of a mistake in the pixel size of one of those datasets it was impossible. I started reprocessing the dataset (import, motion correction and CTF estimation) with the correct pixel size. And here comes my problem. I want to extract only the particles from my last round of 2D classification (saving a lot of time and computational resources). My extraction job does run and finish (marked completed, no errors) but doesn’t display any output . Does anyone know what could be the issue?

Thank you in advance for your help.

Diane

[spunjani]

Hi @Diane,

Thanks for posting. If I understand correctly, you would like to extract the particles using their original locations that were determined from the previous workflow, from the newly motion-corrected CTF-estimated movies. To do so, you can create a new ‘Extract from Micrographs’ job, and connect:

• the ‘exposures’ output from your new CTF estimation job, and
• the ‘particles’ output from your old 2D classification job

Can you please confirm this was the workflow used, and please provide any screenshots that might help us diagnose the missing outputs issue? Thanks!

[Diane]

Yes this is correct, it is the path I used.
On my console I also see that the errors file is empty (job.log.err).

[spunjani]

Hi @Diane,
It looks like the particles are not actually being extracted/written to disk. Can you please post the job log?
You can get it from the command line interface:
cryosparcm joblog PX JX

[Diane]

I think you are correct the particles are not extracted. Nothing happens.

[spunjani]

Hi @Diane,
Do you mind providing a screenshot of the Inputs and Parameters tab for this job?

[Diane]

Sure. No problem. I only used default parameters.

[spunjani]

Hi @Diane, I’m sorry I should have been more clear, I am interested to see the low-level inputs of the J392.micrographs and J360.particles input, can you please open the dropdowns and then paste another screenshot? This is definitely a strange issue that we haven’t encountered before so I apologize for the lengthy troubleshoot!

[Diane]

Sorry I didn’t understand your request. I hope this is what you need.
Thank you for taking care of it.

[Diane]

[stephan]

Hi @Diane,

Is it possible that the micrographs you’re trying to extract particles from don’t actually have any valid particle locations? I noticed you’re connecting inputs from two separate jobs, presumably at different parts from your workflow (are you trying to re-extract particles?). Can you make sure that the particles input matches up properly with the exposures input (i.e. make sure the locations of the particles you’re trying to extract actually came from the micrographs you’ve connected to the job)?

[Diane]

Dear Stephan,

As mentioned previously: I reprocessed my dataset (import, motion correction and CTF estimation) with a new pixel size and then tried to extract particles from the 2D classes of the same dataset but previous pixel size. So the particles come from the same micrographs but reprocessed. As Spunjani was saying, it is:

• the ‘exposures’ output from your new CTF estimation job, and
• the ‘particles’ output from your old 2D classification job

“The micrographs you’re trying to extract particles from don’t actually have any valid particle locations?”

-> I don’t know and I don’t know how to check it. I haven’t done any picking on those micrographs.

“Are you trying to re-extract particles?”

-> More or less, yes.

“Make sure the locations of the particles you’re trying to extract actually came from the micrographs you’ve connected to the job”

-> they are the same, but they do have different pixel size.

[MHB]

I see a similar problem. A rescaled (slightly altered pixel size) data set of images does not output any valid particles from a prior run of the same images with a slightly different pixel size. I presume particle positions are in pixel (x,y) so unclear why this would fail.

[MHB]

I found a work around.

You can covert your .cs particles to a .star file. Then import particles from the .star file and extract from the mics with the new pixel size. Patch ctf and all downstream jobs work fine.

[CleoShen]

Hi Setphan,
I have a similar problem. Shown as below, my job has paused here for an overnight, no any updates, the status is running. I’m wondering if it is because I have too many micrographs?

[joybeiyang]

Hi Diane,

did you find a solution for this? I hit into a similar problem.

Many thanks,

Bei

[wtempel]

Welcome to the forum @joybeiyang.
Given the considerable time that has passed since the prior most recent post under this topic, please can you describe your specific situation (please replace P123, J99 with the the actual project and job identifiers, respectively):

1. Output of these commands on the CryoSPARC master computer:

   cryosparcm cli "get_job('P123', 'J99', 'job_type', 'version', 'instance_information', 'params_spec', 'input_slot_groups')"
   cryosparcm eventlog P123 J99 | tail -n 20
   cryosparcm joblog  P123 J99 | tail -n 20
   

2. Output of these commands on the relevant CryoSPARC worker computer:

   free -g
   cat /sys/kernel/mm/transparent_hugepage/enabled 
   nvidia-smi
   ps -eo user,pid,ppid,rsz,vsz,start,command | grep -E "cryosparc_.+run "