We have access to a government-funded cluster. However, due to the security limitations, we cannot share our filesystem directly with the cluster. How can we take advantage of the cluster with CryoSPARC? We have the ability to mount all of our folders under separate directories, just not the exact same paths as our local machines.
So the scenario is something like:
- Cluster runs worker
- Your machines have files
And you can mount the files on the cluster, but not at the same location they are on your machines?
Make a new directory structure on your machines that mimics the government cluster where the worker will be. Then just symlink your existing files into the appropriate spot on your local machine in the directory structure. CryoSPARC Master and Worker should both think they’re in the same spot.
Let’s say you have to put the files in /home/user/files on the cluster, but on the local machine running CryoSPARC Master, they’re in /mnt/shared/user/files
On the local machine, make /home/user/files (if it doesn’t exist). Then symbolically link (ln -s /mnt/shared/user/files /home/user/files) your directory with the files into the new directory structure, on the local machine. Then make sure the files are accessible at that path on the cluster.
If that doesn’t work, we’ll probably need more details about how things are laid out. If you’re using network shares, links can get messy.
1 Like