Delete rejected exposures

Data Management
1

Hi,
Is there a way to delete just rejected exposures, in either cryoSPARC, or cryoSPARC Live? If not, would the team considering adding this feature?

Thanks

2 Likes
2

Hi @ccgauvin94, the only way do this right now is with a custom Python script. In CryoSPARC v4.1, you can use the cryosparc-tools Python library on a machine with access to the exposures.

Here’s an example script:

from pathlib import Path
from cryosparc.tools import CryoSPARC

cs = CryoSPARC(license="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", email="nick@example.com", password="password123", host="localhost", base_port=39000)
project = cs.find_project('P#')
job = project.find_job('J#')
exposures_rejected = job.load_output('exposures_rejected')
project_dir = Path(project.dir())

for group in ('movie_blob', 'micrograph_blob', 'micrograph_blob_non_dw', 'background_blob', 'micrograph_thumbnail_blob_1x', 'micrograph_thumbnail_blob_2x'):
    field = group + "/path"
    if field in exposures_rejected:
        print(f"Removing {len(exposures_rejected)} blobs in field {field}")
        for rel_path in exposures_rejected[field]:
            print(f"Removing {rel_path}...")
            abs_path = project_dir / rel_path
            abs_path.readlink().unlink()

Substitute your license, user account credentials, instance hostname and port in the cs = CryoSPARC(... initialization.

Substitute P# and J# with the project and job numbers respectively. J# should be a Curate Exposures job. For Live, use a Live Exposure Export job (create from Session > Details > Actions > Export exposures) and substitute exposures_rejected with rejected_exposures or manual_rejected_exposures.

Hope that helps, let me know if you have any trouble with it.

5 Likes
3

Great, thanks. I saw the cryosparc-tools announcement and thought that might provide this functionality. I’ll give this a go with a test dataset.

1 Like
4

Hi @nfrasser

I am playing around with this script and I’m not sure what the following line is doing exactly:

            abs_path = project_dir / rel_path

It seems to be taking my relative path, and somehow producing the absolute path of the micrograph with it. But I’m not familiar with the operand, nor the type. Could you point me toward some documentation of this?

Thanks

5

@ccgauvin94 this is a Python pathlib navigation operation for joining two directories, it’s equivalent to the following:

import os.path
abs_path = os.path.join(project_dir, rel_path)

Hope that helps!

1 Like
6

I came up with a Jupyter notebook based on the above and cryosparc-tools to delete movies based on the rejected exposures from the Curate Exposures job. Just putting it here in case anyone else finds it useful:

If you stop before the last block, it will just print out a list of the filenames, which you can then use to move the files to a different folder or whatever, instead of deleting.

2 Likes
7

Dear nfrasser:
Do we have an easier way to delete the rejected images by the “Curate exposures” job in the new version?
Thanks,
Lan

8

Are you referring to

Are you referring to files that were written by CryoSPARC job, or to files that were imported to CryoSPARC and subsequently used for further processing, like motion correction, etc.?

9

Yes, I would like to know if we have an updated method to delete the rejected images from the row source file, motion-collected images, which are identified by the curation exposures step of a normal process. Thanks,
Lan