Hi All,
We’ve recently had a problem with jobs that are sent to the queue staying there. This happens to jobs running on GPU’s or CPU’s (including interactive jobs).
The error is “Master direct jobs cannot require any licenses”. 
We’ve recently updated to V2.11 but it ran fine for a week or so before we got this error so don’t think it’s directly related to this.
Any help would be greatly appreciated.
Jeremy
Hi @Jeremy,
Thanks for reporting. Is it possible if you can send us the data found in the “Metadata” tab of the job you’re getting these errors from? Can you also send us the output of cryosparcm joblog <project_uid> <job_uid> and the output of cryosparcm log command_core, right after you get this error message?
Hi Stephen,
Below is the output from the commands you sent and below that is the metadata log. Apologies for the text dump.
Cheers,
Jeremy
/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/bin/cryosparcm joblog 112 82
Traceback (most recent call last):
File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/deps/anaconda/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/cryosparc2_compute/client.py", line 83, in <module>
print eval("cli."+command)
File "<string>", line 1, in <module>
File "/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/cryosparc2_compute/client.py", line 57, in func
assert False, res['error']
AssertionError: {u'message': u"OtherError: argument of type 'NoneType' is not iterable", u'code': 500, u'data': None, u'name': u'OtherError'}
/gpfs0/apps/strubi/cryosparc/2.2-gcc5.4.0/cryosparc2_master/bin/cryosparcm log command_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
Traceback (most recent call last):
File "cryosparc2_command/command_core/__init__.py", line 182, in background_worker
scheduler_run_core() # sets last run time
File "cryosparc2_command/command_core/__init__.py", line 1302, in scheduler_run_core
assert not license_needed, "Master direct jobs cannot require any licenses"
AssertionError: Master direct jobs cannot require any licenses
---------- Scheduler running ---------------
Lane default node : Jobs Queued (nonpaused, inputs ready): [u'J42', u'J20', u'J79', u'J202', u'J78', u'J81', u'J82', u'J43', u'J201', u'J26', u'J52']
Total slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6,}
Available slots: {u'compG000': {u'GPU': set([0, 1, 2, 3, 4, 5, 6, 7]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])}, u'compG001': {u'GPU': set([0]), u'RAM': set([0, 1, 2, 3, 4, 5, 6, 7]), u'CPU': set([0, 1, 2, 3, 4, 5}
Available licen: 10000
Now trying to schedule J42
Need slots : {u'GPU': 0, u'RAM': 0, u'CPU': 0}
Need fixed : {u'SSD': False}
Need licen : True
Master direct : True
****** Scheduler Failed ****
The metadata file is too large to post but I can send it through offline if you like.