In using the Rebalance 2D classes job in the 2.16 beta, initially it worked fine and as expected. however in other similar runs, it just assigns zero particles to each class for no reason I can figure out. I have 300k particles, and matching templates from the same Select 2D job, and I have selected 10 superclasses with the default rebalancing factor of 0.1. Thoughts? I notice that it says the “maximum superclass size is set at zero particles”, this seems like a problem…
Thanks for reporting this, we will look into it! A question:
Were the templates and particles generated from the same 2D classification job, followed immediately by a Select 2D job?
The job relies on all particles in the particle stack being assigned to one of the templates present in the templates dataset, through a correspondence between the particles alignments2D/class field and the template idx. It looks like the templates are getting recognized correctly, since each of the 47 templates are properly assigned to a superclass.
But, the corresponding particles don’t all seem to be recognized. The 300k particles are loaded in at the start, but the sum of the particles assigned to each superclass is only around 136k – initially this should add up to the total number of loaded particles. The expected behaviour is that first all particles corresponding to a given superclass will be counted (in the “There are x templates in superclass y; Adding z particles to superclass y” lines in the stream log), and removal happens only after the maximum superclass size is set.
It would also be good if there was an option to manually select (or edit) the superclasses… the spectral clustering mostly works very well, but occasionally makes visually obvious errors that I don’t have a way to correct.
In any of the runs that produced no particles_selected in the end, could you confirm that every template that was input into the Rebalance 2D Classes job had a non-zero number of particles assigned to them? I believe this may occur when one or more templates have no particles assigned to them. You can obtain this information by taking the particles_excluded output from the Rebalance 2D job, together with the templates_all output, and connecting these to a Select 2D job, which will show the number of particles occupying each template.
That would make sense, although it seems like something else is going wrong with the correspondence between particles and 2D classes – the total number of particles added in the “Adding x particles to superclass y” lines doesn’t add up to the 322k particles passed in. I haven’t been able to find the underlying problem with this, but in the next release there will be an override parameter to the (maximum) superclass size, which should allow you to specify the desired size of superclasses. This will at least prevent the issue happening where one of the superclasses being empty causes all of them to be empty.
Thanks for your time in helping me look into this issue!