If a Class P Filter job is queued up using ‘particles_all_classes’ from a not-yet-completed Heterogeneous Refinement (say, using only class 0), it will fail saying ‘particles.alignments_class_0’ is not connected. It used to work so it might be related to changing how the alignments are stored recently.
Hey @DanielAsarnow – apologies for the delayed response! Is the Hetero Refine job a clone of a pre-v4.5 job? When you build the Class Probability filter job, what do the low level inputs look like? (see below)
@vperetroukhin Sorry I forgot about this! So the issue is that the Class P-Filter job is looking for particles.alignments3D and if that isn’t there (because the input Het. Refine was run on newly extracted particles with no previous alignments), then it fails. If the Het Refine used particles that were previously aligned, say by Homo. Refine, then the error doesn’t occur.
Further, if alignments3D_multi is present then this field will go into the passthrough, and subsequent jobs that depend on the new name will be OK. But if it isn’t (for various reasons) then there can also be the situation where Class P-Filter works but produces output without the new alignments field, then subsequent jobs will output warnings (most jobs) or actually fail (at least Het. Refine).
So: at least Class P-Filter should be looking for and working if only alignments3D_multi is present, e.g. new Het. Refine jobs with no previous 3D alignments, I think this is legitimately a bug right now.
Not sure if the 2nd situation warrants a workaround, I think that only applies to old jobs.
Hey @DanielAsarnow,
Thanks for reporting this! This job is indeed confusing in its current state. We’ve noted this for a future release. A couple notes:
So the issue is that the Class P-Filter job is looking for particles.alignments3D and if that isn’t there (because the input Het. Refine was run on newly extracted particles with no previous alignments), then it fails. If the Het Refine used particles that were previously aligned, say by Homo. Refine, then the error doesn’t occur.
Class P-filter will only fail in this case (i.e., particles.alignments3D is missing) if the ‘3D Class Indexes’ parameter is not set. The job was designed to filter particles based on their probability mass in one or more fixed classes – not based on the probability within the per-particle assigned/‘argmax’ class. The job assumes that either:
-
The class indices parameter is None and you’ve provided particles from one upstream class (with the assigned class probability written out in
particles.alignments3D). In this case, it will ignorealignments3D_multiif it is present and just filter based onalignments3D/class_posterior. Note that homogeneous/non-uniform refinements will write out 1s toalignments3D/class_posterior, so if particles are not from a single class but this column exists because it is passed through from an upstream refinement then Class P-filter will not fail but no particles will ever be filtered out. This latter situation is definitely confusing. -
You’ve provided particles with
particles.alignments3D_multi(or pre v4.5alignments_class_X) and set the '3D Class Indexes` parameter. The job sums the probability mass for each particle in the selected class indices, and rejects all particles that have a sum lower than the specified threshold.
The only way currently to filter particles by their maximum probability mass is to filter each of the class particles outputs by the same threshold (a la case 1 above) and then combine particles. This will lose the full classification posterior however.
Further, if alignments3D_multi is present then this field will go into the passthrough, and subsequent jobs that depend on the new name will be OK. But if it isn’t (for various reasons) then there can also be the situation where Class P-Filter works but produces output without the new alignments field, then subsequent jobs will output warnings (most jobs) or actually fail (at least Het. Refine).
Could you post a screenshot/text of the warning that these subsequent jobs produce?
Thanks,
Valentin