I’ve identified two errors with the Particle Set Intersection tool.
The first is fairly trivial, which is that particle set A must be larger than particle set B, or the job fails.
I can get around that by making sure that Set A is larger, and the job progresses further, but fails at the final step of output checking due to some mis-match in the B minus A set.
That was my thought too. Unfortunately, cryosparc is running on a cluster and I’m not admin, so this is not immediately under my control. But I will pursue updating.
Thanks
RJ
Got the sysadmin to update and still having the same problem. Note that the last line “outputs are still usable,” appears not to be true either. I can see the output there in the little box, but when I try to pass it to the next job, that job sits there waiting for input.
help?
THanks
RJ
License is valid.
Running job on master node
[CPU: 85.8 MB] Project P11 Job J513 Started
[CPU: 85.9 MB] Master running v2.14.2, worker running v2.14.2
[CPU: 86.0 MB] Running on lane dcc
[CPU: 86.0 MB] Resources allocated:
[CPU: 86.0 MB] Worker: dcc-cryosparc-02.dscr.duke.local
[CPU: 86.0 MB] --------------------------------------------------------------
[CPU: 86.0 MB] Importing job module for job type particle_sets…
[CPU: 86.0 MB] Job ready to run
[CPU: 86.0 MB] ***************************************************************
[CPU: 86.1 MB] Operating on particle. Action is intersect. Slot that will be used during output is blob.
[CPU: 137.8 MB] Inputted 76942 items as set A
[CPU: 137.8 MB] Action is Intersect/Difference
[CPU: 175.9 MB] Inputted 69294 items as set B
[CPU: 156.4 MB] Computing set operations…
[CPU: 207.8 MB] Intersection: 59750 items
[CPU: 207.8 MB] A-minus-B: 17192 items
[CPU: 207.8 MB] B-minus-A: 9544 items
[CPU: 207.8 MB] Creating output files…
[CPU: 207.9 MB] Done!
[CPU: 207.9 MB] Done in 1.69s
[CPU: 205.2 MB] --------------------------------------------------------------
[CPU: 205.2 MB] Compiling job outputs…
[CPU: 205.2 MB] Passing through outputs for output group intersect from input group particles_A
[CPU: 211.2 MB] This job outputted results [‘blob’]
[CPU: 211.2 MB] Loaded output dset with 59750 items
[CPU: 211.2 MB] Passthrough results [‘ctf’, ‘location’, ‘alignments3D’, ‘alignments2D’, ‘pick_stats’]
[CPU: 223.1 MB] Loaded passthrough dset with 76942 items
[CPU: 223.3 MB] Intersection of output and passthrough has 59750 items
[CPU: 223.3 MB] Passing through outputs for output group A_minus_B from input group particles_A
[CPU: 219.2 MB] This job outputted results [‘blob’]
[CPU: 219.2 MB] Loaded output dset with 17192 items
[CPU: 219.2 MB] Passthrough results [‘ctf’, ‘location’, ‘alignments3D’, ‘alignments2D’, ‘pick_stats’]
[CPU: 236.3 MB] Loaded passthrough dset with 76942 items
[CPU: 236.3 MB] Intersection of output and passthrough has 17192 items
[CPU: 236.3 MB] Passing through outputs for output group B_minus_A from input group particles_B
[CPU: 234.8 MB] This job outputted results [‘blob’]
[CPU: 234.8 MB] Loaded output dset with 9544 items
[CPU: 234.8 MB] Passthrough results [‘ctf’, ‘location’, ‘alignments3D’, ‘alignments2D’, ‘pick_stats’]
[CPU: 232.6 MB] Loaded passthrough dset with 69294 items
[CPU: 232.6 MB] Intersection of output and passthrough has 7357 items
[CPU: 218.3 MB] Checking outputs for output group intersect
[CPU: 229.8 MB] Checking outputs for output group A_minus_B
[CPU: 223.6 MB] Checking outputs for output group B_minus_A
[CPU: 220.1 MB] Traceback (most recent call last):
File “cryosparc2_master/cryosparc2_compute/run.py”, line 87, in cryosparc2_compute.run.main
File “cryosparc2_compute/jobs/runcommon.py”, line 843, in check_outputs
assert output_len == target_len, “Result %s had the wrong length (%d, should be %d). Note that outputs are still usable.” % (resname, output_len, target_len)
AssertionError: Result ctf had the wrong length (7357, should be 9544). Note that outputs are still usable.
Alright, we found a bug in the Particle Sets tool code that was causing this. We’ll have a fix out for it in the next release.
In the meantime, I’ve uploaded a patched copy of the Python file for you to use.
Download the file from our AWS S3 bucket here: https://structura-assets.s3.amazonaws.com/run_sets_v2.14_intersection_bugfix/run_sets.py
Download directly or use wget via the command line:
wget https://structura-assets.s3.amazonaws.com/run_sets_v2.14_intersection_bugfix/run_sets.py
Replace the file in the following folder on the cryoSPARC master node:
cryosparc2_master/cryosparc2_compute/jobs/utilities
Then try running the same job again, and you should be good to go!
This fix is now out in the latest cryoSPARC v2.15