Migrating the project and re-assigning the raw data for Import job

Import and Export
1

Dear Colleagues,

I have moved my project to another location and re-attached it.

I also have raw data removed to the same new location.

Using the general CS migration procedure, I have relocated and re-attached my project to the New location. Additionally, I have moved the raw data (raw data + gain reference + XML) to the same New location.

Could someone please clarify how to update the path for the J1 import job to point to the New raw data location (including raw data, gain reference files, and XML)?

To ensure I understand correctly:

  1. One needs to update the symlink files in the J1/imported folder to reflect the New raw data location.
    I tried the following advice - Edit symlinks in the Import job after project migration - Data Management - CryoSPARC Discuss It did not work for me so far.

cryosparcm icli
cli.job_import_replace_symlinks(‘P47’, ‘J1’, ‘/data/dmitry/test/data/’, ‘/mnt/mfs/cryoem1_data/dmitry/Raw_data/test/data/’)

  1. Is there a specific way to update the path within the J1 import job itself?

My dataset is quite sized, and I don’t want to rerun the motion correction step.

Thank you for your assistance.

Kind regards,
Dmitry

2

Please can you describe the state of the links after attempting the symlink replacement:

  1. Are the symlinks not updated?
  2. Or: Are the updated symlinks not pointing to the correct target(s)?
1 Like
3

Hello @wtempel,

Thank you for your previous questions.

I’d like to describe my current situation and seek your guidance.

I needed to re-attach and relink a project along with its original data after moving it to a new location.
Here’s what I’ve done so far:

  1. Detached the project inside CryoSPARC.
  2. Copied the CryoSPARC project folder to the new location.
  3. Reattached the project in CryoSPARC (Project: P48).
  4. Created a symlink for the new original data location for the import files (P48, J1):

cryosparcm icli
In [1]: cli.job_import_replace_symlinks(‘P48’, ‘J1’, ‘/data/dmitry/test’, ‘/mnt/mfs/cryoem1_data/raw_data/test/’)
Out[1]: 0

In [2]: exit.

When I tested a new motion correction run with the imported images, I encountered the following error:
image

The step completed, but I’m unsure if everything was executed correctly.

Could you please:

  1. Confirm if my steps are correct.
  2. Advise on how to resolve the error I encountered.
  3. Clarify if any additional steps are needed for previously processed motion correction protocols, now that the project is reattached in its new location.

Thank you for your assistance.

Kind regards,
Dmitry

4

@Dmitry I cannot quite tell whether this is a problem with the link(s) or with the underlying files. Please can you provide additional context:

  1. Replace J199 with the patch motion job’s job ID in the following command, run the command and post the output
    cryosparcm eventlog P48 J199 | grep -B 8 "Can not read TIFF directory count" | tail -n 20
  2. Post the output of the command
    find $(cryosparcm cli "get_job_dir_abs('P48', 'J1')")/imported -ls | tail -n 5
1 Like