User issue when updating from CryoSparc 2.15 to 3.0

prash · December 9, 2020, 8:24pm

Hi folks

Machine info: Standalone machine running with SBgrid setup.

CryoSPARC System master node installed at
/home/singhpk/software/cryosparc/cryosparc_master
Current cryoSPARC version: v3.0.0
----------------------------------------------------------------------------

cryosparcm process status:

app                              RUNNING   pid 23617, uptime 0:15:30
app_dev                          STOPPED   Not started
command_core                     RUNNING   pid 23400, uptime 0:15:54
command_rtp                      RUNNING   pid 23470, uptime 0:15:44
command_vis                      RUNNING   pid 23463, uptime 0:15:45
database                         RUNNING   pid 23306, uptime 0:15:58
liveapp                          RUNNING   pid 23646, uptime 0:15:29
liveapp_dev                      STOPPED   Not started
watchdog_dev                     STOPPED   Not started
webapp                           RUNNING   pid 23603, uptime 0:15:32
webapp_dev                       STOPPED   Not started

----------------------------------------------------------------------------

global config variables:

export CRYOSPARC_LICENSE_ID="abc123"
export CRYOSPARC_MASTER_HOSTNAME="abc"
export CRYOSPARC_DB_PATH="/home/singhpk/software/cryosparc/cryosparc_database"
export CRYOSPARC_BASE_PORT=61000
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_INSECURE=false
export CRYOSPARC_CLICK_WRAP=true

I tried updating to 3.0 and got some error. Decided to do a clean install. Noticed that the new instruction for CS 3.0 as new names for the folders (i.e. cryosparc_master instead of cryosparc2_master). Now I think I have 2 installs. CS 2.15 is on port 39000 and CS 3.0 is on 61000.

Issue: Unable to login using my credentials. Error: User not found

How can I fix this? Also, how can I access my projects from CS 2.15?

Excited to start using 3.0!

Thank you

Prash

nfrasser · December 9, 2020, 8:36pm

@prash you just need to make sure that you’re using the same database for your new installation. You can do so by replacing the value of CRYOSPARC_DB_PATH in cryosparc_master/config.sh with the same value it had in the previous cryosparc2_master/config.sh. Then it will use the same database and projects. Let me know if you need additional clarification on this.

Do you still have the error logs you got during the update? Can you send them over to me?

prash · December 9, 2020, 8:50pm

Thank you @nfrasser for quick response. I updated the path for the database but now I am getting a new error when I start CS.

bash-4.2$ cryosparcm start
Starting cryoSPARC System master process..
CryoSPARC is already running.
If you would like to restart, use cryosparcm restart
bash-4.2$ cryosparcm restart
CryoSPARC is running.
Stopping cryosparc 
command_core: stopped
command_rtp: stopped
command_vis: stopped
database: stopped
Shut down
Starting cryoSPARC System master process..
CryoSPARC is not already running.
database: started
command_core: started
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

command_vis: started
command_rtp: started

For your second question, I don’t have the exact error log but its something I have encountered in the past and one way to fix it was to do a clean install. Here is the message I would receive:

supervisorctl: command not found

nfrasser · December 9, 2020, 8:54pm

Can you get the last few logs from cryoSPARC’s command_core module with this command?

cd cryosparc_master
bin/cryosparcm log command_core | tail -n 1000

And send over the output

prash · December 9, 2020, 8:58pm

Its a long output. It wont let me send all of it.

    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 189, in background_worker
        scheduler_run_core() # sets last run time
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1736, in scheduler_run_core
        prune_scheduler_queued_jobs()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1721, in prune_scheduler_queued_jobs
        all_queued_jobs = list(mongo.db['sched_queued'].find({}, {'queued_job_hash' : 1, 'last_scheduled_at' : 1}))
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Scheduler Failed **** 
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 194, in background_worker
        check_heartbeats()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1972, in check_heartbeats
        'heartbeat_at' : {'$lt' : deadline} }, {'project_uid' : 1, 'uid' : 1}))
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Heartbeat check failed ****
    [JSONRPC ERROR  2020-12-09 15:42:17.526504  at  get_num_active_licenses ]
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1608, in get_num_active_licenses
        for j in jobs_running:
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 200, in background_worker
        concurrent_job_monitor()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1615, in concurrent_job_monitor
        current_concurrent_licenses_deque.append(get_num_active_licenses())
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 124, in wrapper
        raise e
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1608, in get_num_active_licenses
        for j in jobs_running:
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Concurrent job monitor failed ****
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 205, in background_worker
        heartbeat_manager()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1659, in heartbeat_manager
        active_jobs = get_active_licenses()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1624, in get_active_licenses
        for j in jobs_running:
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Instance heartbeat failed ****
    [JSONRPC ERROR  2020-12-09 15:42:18.529468  at  get_config_var ]
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 481, in get_config_var
        res = mongo.db[colname].find_one({'name' : name})
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/collection.py", line 1319, in find_one
        for result in cursor.limit(-1):
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    -----------------------------------------------------
    [JSONRPC ERROR  2020-12-09 15:42:18.530041  at  get_running_version ]
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 438, in get_running_version
        version = get_config_var('running_version')
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 124, in wrapper
        raise e
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 481, in get_config_var
        res = mongo.db[colname].find_one({'name' : name})
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/collection.py", line 1319, in find_one
        for result in cursor.limit(-1):
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    -----------------------------------------------------
    [JSONRPC ERROR  2020-12-09 15:42:18.530496  at  get_system_info ]
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 462, in get_system_info
        'version' : get_running_version(),
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 124, in wrapper
        raise e
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 438, in get_running_version
        version = get_config_var('running_version')
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 124, in wrapper
        raise e
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 115, in wrapper
        res = func(*args, **kwargs)
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 481, in get_config_var
        res = mongo.db[colname].find_one({'name' : name})
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/collection.py", line 1319, in find_one
        for result in cursor.limit(-1):
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    -----------------------------------------------------
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 189, in background_worker
        scheduler_run_core() # sets last run time
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1736, in scheduler_run_core
        prune_scheduler_queued_jobs()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1721, in prune_scheduler_queued_jobs
        all_queued_jobs = list(mongo.db['sched_queued'].find({}, {'queued_job_hash' : 1, 'last_scheduled_at' : 1}))
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Scheduler Failed **** 
    Traceback (most recent call last):
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 194, in background_worker
        check_heartbeats()
      File "/home/singhpk/software/cryosparc/cryosparc_master/cryosparc_command/command_core/__init__.py", line 1972, in check_heartbeats
        'heartbeat_at' : {'$lt' : deadline} }, {'project_uid' : 1, 'uid' : 1}))
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
        if len(self.__data) or self._refresh():
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
        self.__send_message(q)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
        address=self.__address)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
        exhaust=exhaust)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
        return func(session, server, sock_info, slave_ok)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
        unpack_res)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/server.py", line 137, in run_operation_with_response
        first, sock_info.max_wire_version)
      File "/home/singhpk/software/cryosparc/cryosparc_master/deps/anaconda/envs/cryosparc_master_env/lib/python3.7/site-packages/pymongo/helpers.py", line 140, in _check_command_response
        raise NotMasterError(errmsg, response)
    pymongo.errors.NotMasterError: node is not in primary or recovering state, full error: {'ok': 0.0, 'errmsg': 'node is not in primary or recovering state', 'code': 13436, 'codeName': 'NotMasterOrSecondary'}
    ****** Heartbeat check failed ****

nfrasser · December 9, 2020, 9:07pm

A restart might fix this first make sure the other cryoSPARC instance is not running

cd /path/to/cryosparc2_master
bin/cryosparcm stop

Then restart

cd /path/to/cryosparc_master
bin/cryosparcm stop
bin/cryosparcm start

prash · December 9, 2020, 9:19pm

Still getting the same error.

CryoSPARC is not already running.
database: started
command_core: started
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

command_vis: started
command_rtp: started

Also, before the restart when I tried to stop cryosparc2 it gave me this error:

Could not find conda environment: cryosparc_master_env
        You can list all discoverable environments with `conda info --envs`.
            Warning: Could not activate conda environment; this indicates that a cryoSPARC installation is either incomplete or in progress
            bin/cryosparcm: line 133: supervisorctl: command not found
            CryoSPARC is running.
            Stopping cryosparc 
            bin/cryosparcm: line 737: supervisorctl: command not found

nfrasser · December 9, 2020, 9:30pm

Ah full system restart should help with this, can you try that before trying those steps?

prash · December 9, 2020, 9:40pm

Sorry @nfrasser. Full restart and then the commands didn’t help. They gave the exact same errors as above.

prash · December 9, 2020, 9:42pm

Is this a feasible option?
remove the cs2 and cs3, reinstall cs3 and then copy the cs2 database folder to the cs3 database?

prash · December 9, 2020, 9:56pm

Hey @nfrasser

I think I got it to work. I think your idea to change the db in the config file was genius. In the same config file I changed the port from 61000 to 39000 to match the cryosparc2_database setting and it worked!!!

It now running CS 3.0. I am so excited to get started on it!

Thank you CS team for working hard to get the CS 3.0 out.

Prash

nfrasser · December 9, 2020, 9:57pm

Ahhh I missed that you changed your base port, glad you got this working!

prash · December 9, 2020, 10:11pm

Just to summarize (so other users don’t have to go through the complete conversation)

Issue: Installed CS 3.0, but encountered ‘User not found’ error on the platform (probably because I installed while CS 2.15 was still on the system on a different port).

Other issue: Needed to access project from CS 2.15

Solution:

Edit cryosparc database path. Replace 3.0 (default) path to 2.15 (previous version) path in the 3.0 in

cryosparc_master/config.sh
In the same file (config.sh) update the port to the previous port to which the database was connected. In my case that was replacing 61000 to 39000.
Save the file
Restart the workstation.
cd into the cryosparc2_master and stop cryosparc instance:

cd /path/to/cryosparc2_master
bin/cryosparcm stop
cd into the cryosparc_master. Stop and then start the instance

cd /path/to/cryosparc_master
bin/cryosparcm stop
bin/cryosparcm start
Open web browser and go to localhost:39000

ehanssen · December 10, 2020, 1:04am

slightly different issue here. after the update to 3.0 and at the end odf the restart i get
command_core: ERROR (spawn error)
Thoughts?

prash · December 10, 2020, 2:36am

Sometimes this can happen if there is an orphan process that needs to be killed.
You can check by running:
ps -ax | grep "supervisord"