Hi Mark,
Thank you very much! Definitely a lot to test.
I also see the reason for processing data separately now, will do it once I get more allocation, my directory is getting full!
Hi Mark,
Thank you very much! Definitely a lot to test.
I also see the reason for processing data separately now, will do it once I get more allocation, my directory is getting full!
Same, it’s really convenient.
are you using 16 Floating points for patch motion and Extract Particles from micrographs ? default = 32 which take a lot more space.
Uh, I have been using default. What is the difference between 32 and 16? Will using 16 compromise the result by any chance?
I collected 0 and 40 in two sessions, so it should be pretty easy to process them separately if I have enough space…lol
Does not compromise the result at all - just takes up half the disk space - I would always use fp16 (should be the default IMO)
Great! Thank you very much, Oli!
Something else to consider (as an alternative to rebalance 2D classes): Job: Rebalance Orientations | CryoSPARC Guide
Post 5. Great minds?
Whoops sorry I missed it!!
@olibclarke is correct, stay with 16-floating points. On my system, if users do not enable 16-floating points by default and I find out, I will then administer discipline to them.
also see https://guide.cryosparc.com/processing-data/tutorials-and-case-studies/tutorial-float16-support
The jobs to enable this are Patch Motion Correction and Extract Particles From Micrographs.
@Jiarui, you very likely you have massive particle stacks that take up space on disk and SSD scratch, also much longer + vRAM to compute 32 floating points vs. 16.
Hi Mark,
Thank you so much, this is very helpful! Will implement it from now on. Do you know if it is possible to redo my completed jobs without affecting my current results?
As far as I know, both CryoSPARC and RELION both internally convert 16-bit to 32-bit before any kind of processing (I think @hsnyder confirmed a while back, but I am failing to find the relevant post on the forum). Certainly RELION does, because if loading a full particle stack (16-bit MRC) into RAM, it will take slightly more than double the space of the stack on disk, which would be consistent with conversion to 32-bit with a margin of overhead for other metadata.
Confirmed. fp16 is (currently) used as a storage format only, we convert to/from fp16 during I/O to/from disk. We may leverage it in other ways in the future but this is true as of v4.6.2
Thank you @rbs_sci and @hsnyder. I should pay more attention to this.
When I installed/configured RELION with cmake, I followed https://relion.readthedocs.io/_/downloads/en/release-5.0/pdf/ for the cmake install with floating points.
@Jiarui you can always clear a job and re-do it, but there is all of that metadata. If all you care about are the particles locations you can always reassign particles to micrographs. In some cases I feel a good TOPAZ model is enough to move onto a new data (but similar) and make some good picks.