Applying a defect file at the Reference-Based Motion Correction stage?

Import and Export
1

Hello cryoSPARC team,

We encountered a hot pixel issue during our data processing. Initially, the hot pixels persisted through motion correction and affected downstream processing. We later found that when a defect file is provided at the import stage, the hot pixel issue is resolved after motion correction.

At this point, however, we already have several datasets that have been fully processed, and we would prefer not to recompute everything from scratch.

We are therefore wondering whether it is possible to apply a defect file at a later stage, specifically during Reference-Based Motion Correction.

  • Does reference-based motion correction make use of a defect file if one is provided?

  • How is the defect file applied in this step?

  • Is it possible to add or apply a defect file retroactively using cryoSPARC tools?

Any advice on how to handle this situation would be greatly appreciated.

Thanks
Hao

2

Hi @pey123456! The defect file is applied in the same way as in Patch Motion Correction. You can add one using cryosparc-tools by adding a mscope_params/defect_path column to the dataset that contains the path to the defect file, relative to the project directory. For example:

from cryosparc.tools import CryoSPARC
import json
import numpy as np
from pathlib import Path

with open(Path('~/instance-info.json').expanduser(), 'r') as f:
    instance_info = json.load(f)

cs = CryoSPARC(**instance_info)
assert cs.test_connection()

puid = "P392"
project = cs.find_project(puid)
exposure_juid = "J466"
exposure_wuid = "W17"
exposure_job = project.find_job(exposure_juid)
exposure_title = "exposures_rejected"
exposures = exposure_job.load_output(exposure_title)

defects_filename = "defects.txt"
# create a defect file with just one pixel
with open(project.dir / defects_filename, "w") as f:
    f.write("100 100 1 1\n")
exposures["mscope_params/defect_path"] = defects_filename

ext_job = project.create_external_job(
    workspace_uid = exposure_wuid,
    title = "Added defect file"
)
ext_job.add_input("exposure", "exposures", slots=exposures.prefixes())
ext_job.connect("exposures", exposure_juid, exposure_title)
ext_job.add_output(
    type="exposure",
    name="exposures",
    slots=["mscope_params"],
    passthrough="exposures"
)
with ext_job.run():
    ext_job.save_output("exposures", exposures)
1 Like