Remote connection to a virtual machine guide

Install, Configure and Update
1

I have installed cryosparc(both master and worker nodes) on my local machine but i want to run/queue the jobs on a remote virtual machine how can I do this?
P.S. I am new to cryosparc…

2

Does your remote virtual machine meet the prerequisites?
If it does, you would then need to follow installation and connection instructions.
You will probably also need to consult documentation for your virtual machine platform for

  • GPU management
  • network configuration
3

Do I need to install the cryosparc worker package on the remote machine as well?

4

It needs to be available on the remote machine, either via

  • installation
  • or sharing of an existing, installed, cryosparc_worker/ directory. When sharing, the absolute path to cryosparc_worker/ on all clients mounting the share must match the absolute path that was in effect when the cryosparc_worker/install.sh command was run.
5

I want to share the cryosparc_worker directory to my remote machine,but I never ran the cryosparc_worker/.install.sh command also once worker directory has been transferred what I have to do to run the jobs on remote machine

6

If you ran
cryosparc_master/install.sh with the --standalone option, cryosparc_worker/install.sh would have run automagically.

You might want to run
/path/to/cryosparc_worker/bin/cryosparcw connect [..] with the appropriate options (see the links I posted earlier). For this command to work, you need to ensure that the remote machine can access relevant network ports on the CryoSPARC master (details also linked above). When establishing that connectivity, be sure to protect network ports on the CryoSPARC master from illegitimate access attempts.

7

Thanks for your help. I am transferring the worker directory to my VM. Also I did not get the part "you need to ensure that the remote machine can access relevant network ports on the CryoSPARC master "
earlier the cryosparc was working fine in my local machine but I had to move it to VM due to resource constrints.Does this point guarantee the remote machine can access relevant ports on master?
also what is the master hostname in the cryosparcw connect command?

8

@aggsh This is how I understood your situation and request:

  1. You installed and successfully used CryoSPARC on machine “A”
  2. You wish to continue running CryoSPARC on machine A, but would like to also submit for processing some jobs that were built on machine A to (virtual) machine “B”. This would require expanding the CryoSPARC “instance” to a two-machine instance where machine A is a combined master/worker and machine B a worker that is managed by machine A.

Please correct my understanding and provide additional details as needed.

9

Yes, apparently machine A does not have the required RAM/compute to run all the jobs.

10

or to simplify I want to run all jobs from remote only. making machine A the master node and machine B the worker

11

In this case

  • machine B needs to

    • correctly resolve (via DNS or /etc/hosts entry) the hostname defined on machine A inside the cryosparc_master/config.sh file for the CRYOSPARC_MASTER_HOSTNAME variable
    • access the database, command_core and command_rtp ports on machine A

  • machine A needs to

    • correctly resolve (via DNS or /etc/hosts entry) to machine B the hostname defined by the cryosparcw connect --worker parameter, as specified when you run cryosparcw connect on machine B
    • run commands remotely on machine B via ssh

Your IT support, whom you may also want to point to the installation guide, may be able to help ensure these prerequisites.

12

… or migrate your entire master/worker installation to machine B?

13

I have migrated the entire worker to machine B is there anything left to do?

14

Did you

  1. connect worker B to the CryoSPARC instance
  2. ensure ssh access from worker A to worker B
  3. share raw data and project directories between worker A and worker B

(as described in the links above)?

15

I did not understand point 2 and point 3

16

You may want to consult the applicable sections (re:2, re:3) of the guide and your IT support, as needed.
… or sidestep points 2 and 3 by moving all master and worker activity to machine B, assuming that virtual machine B fulfills minimum worker requirements.

17

my machine B satisfies all the worker requirements, so should I move both the master and worker directories there? also do I need to move that database folder? also what steps should I follow once I have moved both the folders.

18

also while I was trying to transfer the file using rsync command to my remote machine it prompts
“some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]”

19

@aggsh Does this topic concern the same CryoSPARC installation as Curl command dependency? Should this topic be closed and discussion continued in Curl command dependency?