Topaz extract throws error when more micrographs are added

Hello Community,
I have just started working with topaz on cryosparc. out of around 14000 micrographs i manually curated about 600 micrographs. did blob pickin, template picking, 2d classification. i used these particle in topaz train job (on about 600 micrographs). the graph are


I am very new to these neural network. It will be great help if you guys look at it and give me some suggestions.

Thank you,
Prabin

they look great. you can use that trained model to “topaz extract” = pick particles on any number of micrographs. Best performance using it to pick those in the same dataset. The number of particles it picks will strongly depend on the expected number of particles you provided in the training. If you are looking for a rare particle, you may still want to provide a number of expected particles close to the full particle set per micrograph to get every rare particle, but if the number reflects the expected number of that rare particle then the proportion of rare will be higher.

in general I just look for ever increasing->plateauing training precision and it should be good.

best practices are to use your very best particles to train and a tiny amount of particles (1000) can be plenty.

BTW, Topaz extract will likely fail with “argument list too long” for 14000 micrographs. You’ll have to do it in batches of 5000 micrographs per job. Exposure Sets can split easily

That is a nice model from Topaz Train.

We usually go in cycles.

When applying the model to the whole data set for TOPAZ extract, you need to use Exposure Sets Tool to split the 14k micrographs into groups of 2-3 so each one is ~5k micrographs. Then input these to their own Topaz Extract job with your model from TOPAZ train.

Unless you denoised in TOPAZ, ensure the slider for denoised micrographs is off in both train and extract. I then use cryosparc’s own extract job to extract particles + micrographs, but I combine all 2-3 Topaz extract jobs into this job so everything is together.

the new TOPAZ v0.3 will give errors with cryosparc, you need to only use the recommended v0.25

Very helpful, Thank you so much

1 Like

Thank you for insightful explanation.