Hi,
In using the Rebalance 2D classes job in the 2.16 beta, initially it worked fine and as expected. however in other similar runs, it just assigns zero particles to each class for no reason I can figure out. I have 300k particles, and matching templates from the same Select 2D job, and I have selected 10 superclasses with the default rebalancing factor of 0.1. Thoughts? I notice that it says the “maximum superclass size is set at zero particles”, this seems like a problem…
Cheers
Oli
Hi @olibclarke,
Thanks for reporting this, we will look into it! A question:
The job relies on all particles in the particle stack being assigned to one of the templates present in the templates dataset, through a correspondence between the particles alignments2D/class field and the template idx. It looks like the templates are getting recognized correctly, since each of the 47 templates are properly assigned to a superclass.
But, the corresponding particles don’t all seem to be recognized. The 300k particles are loaded in at the start, but the sum of the particles assigned to each superclass is only around 136k – initially this should add up to the total number of loaded particles. The expected behaviour is that first all particles corresponding to a given superclass will be counted (in the “There are x templates in superclass y; Adding z particles to superclass y” lines in the stream log), and removal happens only after the maximum superclass size is set.
Yes, they were all from the same Select 2D job (Job 51), which was from a single 2D classification job, Job 36.
It would also be good if there was an option to manually select (or edit) the superclasses… the spectral clustering mostly works very well, but occasionally makes visually obvious errors that I don’t have a way to correct.
Thanks for the suggestion – we will also add this to the list of improvements to rebalance 2D classes!
Best regards,
Michael
i have the same issue here. template are nicely recognised but there are no particle selected at the end.
Hi @olibclarke and @ehanssen,
In any of the runs that produced no particles_selected in the end, could you confirm that every template that was input into the Rebalance 2D Classes job had a non-zero number of particles assigned to them? I believe this may occur when one or more templates have no particles assigned to them. You can obtain this information by taking the particles_excluded output from the Rebalance 2D job, together with the templates_all output, and connecting these to a Select 2D job, which will show the number of particles occupying each template.
Best,
Michael
Hi @mmclean - it is difficult to confirm now because that was six months ago, but I will check the next time I run into this issue - thanks!
Oli
actually looking at the screenshot in my original post, you can see that zero particles have been assigned to superclass 2, so maybe your hunch is right… How does the user avoid this though?
Hi @olibclarke,
That would make sense, although it seems like something else is going wrong with the correspondence between particles and 2D classes – the total number of particles added in the “Adding x particles to superclass y” lines doesn’t add up to the 322k particles passed in. I haven’t been able to find the underlying problem with this, but in the next release there will be an override parameter to the (maximum) superclass size, which should allow you to specify the desired size of superclasses. This will at least prevent the issue happening where one of the superclasses being empty causes all of them to be empty.
Thanks for your time in helping me look into this issue!
Best,
Michael
