When I attempt to initiate a local refinement from volume/particles from an ab initio job, it fails with the attached error. I get the same error if I convert the final_particles.cs file from ab initio to a star file, and then use those particles to start local refinement (even though in general this works, if the particles.cs file comes from a homogeneous refinement, for example). The star file seems to have the eulers and offsets present, so I’m a little puzzled as to what the issue is.
The reason to try this is that for certain datasets, stochastic gradient descent in ab initio seems to give a better result than branch-and-bound in homogeneous refinement. I’d like to try local refinement from the ab initio orientations, based on the idea that the orientations from ab initio may be approximately right, but improveable by local refinement.
Ah I think I see the issue - the RandomSubset column only has one value - half sets haven’t been assigned - maybe if I manually fix that in the star file it’ll work…
Regarding this - would it be possible to have random half sets assigned automatically if they don’t already exist so this workflow could be performed entirely within cryoSPARC?
I just tried this again and it doesn’t work any more. After exporting a particle.cs file from an ab initio job to a star and re-importing, it imports fine and appears to convert the pose information, but when I set up a local refinement it fails with the attached error, which I cannot interpret.
Thank you for the feedback, we’ve recorded this as an issue and are now working on a way to accommodate this workflow. With random assignments of particle splits and accommodating differences in volume/particle box shapes, it would be possible to do this workflow entirely in cryoSPARC without having to manually assign particle splits – however, the latest issue is interesting.
It looks like the error is related to an issue with the alignments3D/split field, which should be assigned the values of the rlnRandomSubset field. The blob information in the particles also seems to be improperly written in the import particles job. What was the input to the ‘Particle data path’ field in the original job?
By the “original job”, do you mean the import particles job? The particle data path is set to the extract directory of the corresponding Extract job. But importing solely from the star file without explicitly setting the data path gives the same results.
Apologies for the delay – thanks for clarifying, I believe the issue is in the particle importing. When you manually reassigned the rlnRandomSubset field, did you assign values in the set {1,2} or something else? Import particles currently assumes that this field is indexed starting from 1, so a split into two half-maps must correspond to rlnRandomSubset taking values in {1,2} only. Import particles then subtracts 1 from each entry in this field and sets that as the alignments3D/split field, to match our convention of split values being either 0 or 1.
Ah! That makes sense now! I had set rlnRandomSubset to {0,1}…
Works now, thank you! Maybe it would be better to just have a check for if there are two and only two values in that column, and if so split accordingly?
Sorry if this has been addressed elsewhere on the forum, but is there any update to the workflow going from ab initio into local refinement? And perhaps more importantly, Oli, did this seem to help at all for your processing?
It works, but I haven’t found a case yet where it helps a lot - still may be worth a try in cases where ab initio looks a lot better than refinement though
Thanks Oli, and a preemptive happy new year to you!
Indeed my ab initio models look great (nice secondary structure - looks as expected based on homologues), while the refinements seem to have endless trouble with alignment. That being said, this is a heavily glycosylated 50 kDa protein, so I’m not too sure it will work out (it is supposed to be dimeric and therefore hopefully more feasible. I’m still working on sample prep on that end of the problem, but I figured I might as well not let these monomer micrographs sit idle).
In the new local refinement job, you should see a parameter called “Force re-do GS split” – this will randomly split the particles into equal sized half-sets, overriding what was present in the split column before. Alternatively, if you are importing particles from RELION and the rlnRandomSubset field is not present, there is a parameter in Import Particles called “Ignore half-set split if missing” and it will randomly split particles at the start of your workflow.
Thanks very much for your suggestion about using the new local refinement job. I will try to update the software to the latest version and use the new job.
Even if you re-do the GS split, if the starting angles for local search were not independent, then the continued local refinement remains non-GS. It’s fine if you just need a better reference to restart a normal GS refinement later on, but otherwise there can still be overfitting.
