Hi,
I am moving a couple of cryopsparc instance from one instance to another one. Both instances see the same storage so I simply imported the project directory into the new instance. The process said complete but sadly I dont see any workspace in the imported project. Does anyone have an idea?
Best,
David
I tried other projects, that worked fine its only this one that seems to have troubles oddly the thumbnails of the results are visible but the workspace is not
I have more issues. I was able to import one other project just fine but when importing a third project the important never really finishes and every time I log in I still see the import status bar. What can I do?
Best,
David
Hi @david.haselbach,
Hmm, we havenât come across this issue before. Were there any errors in the logs?
cryosparcm log command_core
Because the import didnât complete successfully, youâll have to manually clear the notification from showing. You can do this by clicking on the âResource Managerâ, then âNotification Managerâ. There should be a clear button next to the active notification.
Regards,
Suhail
[EXPORT_PROJECT] : Exporting project P23...
[EXPORT_PROJECT] : Exported project P23 to /groups/haselbach/181114_Ribo_6hpf/cryosparc/P40/project.json in 0.01s
[JSONRPC ERROR 2020-03-18 12:17:18.112486 at get_result_download_abs_path ]
-----------------------------------------------------
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 114, in wrapper
res = func(*args, **kwargs)
File "cryosparc2_command/command_core/__init__.py", line 4075, in get_result_download_abs_path
d = dataset.Dataset().from_file(os.path.join(proj_dir_abs, metafile_rel))
File "cryosparc2_compute/dataset.py", line 536, in from_file
with open(path, 'rb') as file:
IOError: [Errno 2] No such file or directory: u'/groups/haselbach/181114_Ribo_6hpf/cryosparc/P40/J24/mask.cs'
The file really doesnât exist but there is a file P40_J24_imported_mask.csg in the folder.
Hey @david.haselbach,
The traceback you posted doesnât seem to be related to the issue you initially posted about- I think @sdawood helped you with the notification problem, though can you please let us know what happened with the empty workspaces issue?
Was the instance that originally created the project still running while you did this? Were users actively using the project/were any modifications made to the project in the other instance? Since the instances are not able to sync with each otherâs databases, errors may arise when you import a project that is being used by another instance. The best thing to do is to make a copy of the project into another directory and import it from that location.
If this is not the case, can you provide more details about how you did the import itself? Screenshots would be very helpful!
the clear buttons dont have an effect. I am also pretty sure that not two instances work on it as the old instance is not accessible at the moment.
What would be the best step now? If I delete the project in the new instance will it delete the data?
The empty workspace is actually still an issue. However, this is a project that is rather old and was made in early version of cryosparc 2. Intresting the preview looks fine and it looks like the data is there. 
But once I click on it, it says no workspaces found.
Yes, the data will be deleted. Please make a backup of your project directory before you do anything else.
This is probably the reason youâre seeing the discrepancy. Only projects that were taken from instances that were on version v2.11+ would have the functionality to export the project properly so that they can be imported into other instances. Is it possible if you can get the old instance back up and running so that you can update it to a more recent version and complete a database migration?
To see some options for reviving your old instance, see this post here: https://cryosparc.com/docs/tutorials/migration/
Sorry this was a misunderstanding. Both versions are the new version. But the project was just made qith a quite early version that has been updated in the meanwhile and I was wondering whether this might be a problem.
I think also the other problem is more problematic as I cant really go on with unstopped progress and the clear button has sadly no effect. Can I do something else to stop the import?
As long as the instance that originally created this project was updated to a newer version, the project would have been properly exported (continuously).
Does this mean the notification is still appearing, even after you go to the Notification Manager and clear the active notification?
As a last-resort option, you can restart your cryoSPARC instance. cryosparcm restart This will stop any running background processes, including the import process.
Does this mean the notification is still appearing, even after you go to the Notification Manager and clear the active notification?
yes indeed
As a last-resort option, you can restart your cryoSPARC instance.
cryosparcm restartThis will stop any running background processes, including the import process.
I tried this also had no effect
Is it possible if you can send me any screenshots? Can you also send me logs of command_core again, like you did earlier?
Hi, meanwhile I restarted the master once again and got all these problems solved after the third restart. But the missing workspace is still an issue. and I cant really find that.
Hi @david.haselbach,
Is it possible if you can send us a screenshot/copy+paste the output of the âlsâ command executed inside the project directory? Just to make sure it has the required documents inside it.
Hey @david.haselbach,
Is it possible if you can turn on the instance where this project was created? That way youâll be able to re-export the data required by modifying something in the database, for example changing the name of the Workspace. Iâm not quite sure why your workspaces.json file is empty.
Sorry for bump the topic up. I have met the problem while export the project from v2.15.0 to v3.0.1 (since I failed to start cryosparc v3 while directly update cryosparc, I installed a new instance with a new port from cryosparc v3 and remains old cryosparc v2).
Our users want to continue running the project in cryosparc v3 so they import it directly to v3.0.1 and get an empty project.
I confirmed that both the cryosparc v2 and cryosparc v3 could work.
Now we have run cryosparc v3 for a new project and I can figure out to find the error message
using cryosparcm log command_core. Instead, the cryosparcm log for command_core seems fairly normal about those running jobs.
for cryosparc v2 command_core log, it seems all the export log are normal and jobs end to run.
We have tried to restart both cryosparc v2 and cryosparc v3 but only import empty workspace.
Simply import one or two job didnât work either.
Hi all,
we have a somewhat similar issue.
In CS 2.xy we have in the past moved some of our projects onto new disk locations without much hassle. Copied the project folder and imported it from the new location as per instructions: https://guide.cryosparc.com/processing-data/tutorials-and-case-studies/tutorial-data-management-in-cryosparc#1-self-consistent-self-contained-project-directories-i-e-no-need-to-export-projects
After an upgrade to CS 3.0.1 the import doesnât seem to be working.
We just get an empty workspace with the correct name and 0 jobs.
Here is log output from: cryosparcm log command_core
**custom thread exception hook caught something
**** handle exception rc
set status to failed
Indexing /xyz/P49 ----
Base path: /xyz/P49
[IMPORT_PROJECT] : Importing project from /xyz/P49
[IMPORT_PROJECT] : Created project P84
[IMPORT_PROJECT] : Created workspace W1 in P84
[IMPORT_PROJECT] : Inserting jobs into project...
[IMPORT_PROJECT] : Uploading project image data...
[IMPORT_PROJECT] : Done. Uploaded 0 files in 0.00s
[IMPORT_PROJECT] : Inserted job document in 0.00s...
[IMPORT_PROJECT] : Inserting streamlogs into jobs...
[IMPORT_PROJECT] : Done. Inserted 0 streamlogs in 0.00s...
[IMPORT_PROJECT] : Imported J1 into P84 in 0.00s...
[IMPORT_PROJECT] : Uploading project image data...
[IMPORT_PROJECT] : Unable to import project from /xyz/P49: could not find MARK
[POST-RESPONSE-THREAD ERROR 2021-01-06 17:38:47.701724 at import_project_run ]
-----------------------------------------------------
Traceback (most recent call last):
File "/xyzxyz/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 3402, in import_project_run
gridfs_data = pickle.load(openfile)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 0: ordinal not in range(128)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/xyzxyz/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 146, in run
self.target(*self.args)
File "/xyzxyz/cryosparc/cryosparc2_master/cryosparc_command/command_core/__init__.py", line 3412, in import_project_run
gridfs_data = pickle.load(openfile, encoding='bytes')
_pickle.UnpicklingError: could not find MARK
-----------------------------------------------------
**custom thread exception hook caught something
**** handle exception rc
set status to failed
The folder was copied with: rsync -av --progress --partial
The ownership was changed to cryosparc_user with: chown -R cryosparc_user /xyz/P49
Any ideas?
There is a bug in Cryosparc 3.0.1 (not sure if it existed in 3.0.0) when importing Cryosparc v2 projects.
At lines 3303 and 3412 in command_core/__init__.py, Cryosparc attempts to read a v2 gridfs file from the project after a failed attempt to interpret that file as a v3, but it does not reset the fileâs buffer position before the second reading attempt. This causes the could not find MARK error (itâs an internal Python error).
Hopefully they fix it in the next patch, since importing v2 to v3 is completely broken right now, but you can easily fix it yourself right now (as I did) by editing the Python script and adding openfile.seek(0) before each of the two lines that raises the error. The import works fine for me after this.