Master direct jobs cannot require any licenses

open

#1

Hi All,

We’ve recently had a problem with jobs that are sent to the queue staying there. This happens to jobs running on GPU’s or CPU’s (including interactive jobs).

The error is “Master direct jobs cannot require any licenses”. 12%20PM

We’ve recently updated to V2.11 but it ran fine for a week or so before we got this error so don’t think it’s directly related to this.

Any help would be greatly appreciated.

Jeremy


#2

Hi @Jeremy,

Thanks for reporting. Is it possible if you can send us the data found in the “Metadata” tab of the job you’re getting these errors from? Can you also send us the output of cryosparcm joblog <project_uid> <job_uid> and the output of cryosparcm log command_core, right after you get this error message?


#3

Hi Stephen,
Below is the output from the commands you sent and below that is the metadata log. Apologies for the text dump.
Cheers,
Jeremy

/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/bin/cryosparcm joblog 112 82

Traceback (most recent call last):

  File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 174, in _run_module_as_main

    "__main__", fname, loader, pkg_name)

  File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 72, in _run_code

    exec code in run_globals

  File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/cryosparc2_compute/client.py", line 83, in <module>

    print eval("cli."+command)

  File "<string>", line 1, in <module>

  File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/cryosparc2_compute/client.py", line 57, in func

    assert False, res['error']

AssertionError: {u'message': u"OtherError: argument of type 'NoneType' is not iterable", u'code': 500, u'data': None, u'name': u'OtherError'}
/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/bin/cryosparcm log command_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

 File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

Traceback (most recent call last):

  File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker

    scheduler_run_core() # sets last run time

  File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core

    assert not license_needed, "Master direct jobs cannot require any licenses"

AssertionError: Master direct jobs cannot require any licenses

---------- Scheduler running ---------------

Lane  default node : Jobs Queued (nonpaused, inputs ready):  [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']

Total slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}

Available slots:  {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}

Available licen:  10000

Now trying to schedule J42

  Need slots :  {u'GPU': 0, u'RAM': 0, u'CPU': 0}

  Need fixed :  {u'SSD': False}

  Need licen :  True

  Master direct :  True

****** Scheduler Failed ****

#4

The metadata file is too large to post but I can send it through offline if you like.