Hi,
I have imported several projects into a new cryoSPARC instance.
After importing, for some projects, the job dependencies (parents, children) information is lost.
Below is an example.
Is there any way to restore the lost dependencies?
Thanks,
Kotaro
I noticed that the projects whose job dependencies were lost were all created before any patch was applied to the cryoSPARC instance. They are created by v3.2.0 (without any patch).
The other projects’ job dependencies are all intact, and they were created after at least patch 210629 was applied.
I hope this information helps.
Thanks,
Kotaro
Hi @kttn
Have you perhaps solved this yet?
We have cryosparc v3.2.0+210511 and the patch was applied but the database from the patched cryosparcv3.2 instance was corrupted and I had to import the project from the existing intact project directory into a new project in the old database (created using cryosparcv3.1). The parent/child information is now missing as in your case.
Hi!
I have a similar issue… all my projects lost the job dependencies and I have no idea what triggered it.
It surely cannot be magic, and even if I am the administrator of this machine I have no clue what happened… all was good before Christmas break (as far as I remember), the first thing I did once I arrived was to start cryoSPARC and found it this way.
I applied the latest patch today and it did not solve the problem 
Current version: v3.3.1+220118
Is anyone experiencing the same?
Thanks,
André
@AndreGraca @lizellelubbe @kttn @ALBA @HeatherQ Please can you identify a job on your workspace for which you are missing expected connections up- and downstream. With the job opened, navigate to the Metadata tab and push Copy to clipboard. Paste and save the clipboard contents to a text file and attach that file to a forum direct message to me. Please also briefly note in the direct message:
Thanks.
@wtempel
Thanks for the prompt reply!
I have attached below the metadata of one of the jobs. It was imported as part of a project, and i have successfully using the outputs from this job for downstream jobs.
{"_id":{"_str":“61e70511cecb8d97246ff0dd”},“created_at”:“2021-12-07T18:03:49.612Z”,“project_uid”:“P1”,“status”:“completed”,“title”:“New Job J94”,“uid”:“J94”,“workspace_uids”:[“W3”],“children”:[“J242”,“J237”,“J241”],“cloned_from”:null,“deleted”:false,“parents”:[],“ui_tile_height”:1,“ui_tile_images”:[{“name”:“imported_small”,“fileid”:“61e70511cecb8d97246ff0d1”,“num_rows”:1,“num_cols”:1},{“name”:“imported_smaller”,“fileid”:“61e70511cecb8d97246ff0d3”,“num_rows”:1,“num_cols”:1},{“name”:“imported_smallest”,“fileid”:“61e70511cecb8d97246ff0d5”,“num_rows”:1,“num_cols”:1}],“ui_tile_width”:3,“job_type”:“import_movies”,“ui_layouts”:{“P1W3”:{“show”:true,“floater”:false,“top”:0,“left”:1648,“width”:444,“height”:192,“groups”:[{“start”:“J94”,“endrow”:1,“conns”:[{“start”:“J94”,“end”:“J237”,“startrow”:0,“startcol”:945,“endrow”:1,“endcol”:315,“subtree”:0,“coloridx”:2,“origin”:[1240,192],“extent”:[630,100],“coords”:[[630,0],[630,80],[0,80],[0,100]]}],“startcol”:945,“startrow”:0,“leftendcol”:315,“rightendcol”:315,“single”:true,“len”:1,“subtree”:0,“lane”:3}]}},“completed_at”:“2021-12-07T18:06:25.374Z”,“created_by_job_uid”:null,“created_by_user_id”:“61e702b1b5e301d4728faa51”,“description”:“Grid 4088851”,“failed_at”:null,“interactive”:false,“job_dir_size”:3519924,“killed_at”:null,“last_accessed”:{“name”:“qianqiao”,“accessed_at”:“2022-01-21T17:29:51.049Z”},“launched_at”:“2021-12-07T18:05:47.935Z”,“priority”:0,“queued_at”:“2021-12-07T18:05:37.831Z”,“started_at”:“2021-12-07T18:05:49.212Z”,“waiting_at”:null,“version”:“v3.1.0”,“type”:“import_movies”,“run_as_user”:null,“params_secs”:{“import”:{“title”:“Movies”,“desc”:“Metadata and raw data for movies.”,“order”:0},“compute_settings”:{“title”:“Compute settings”,“desc”:"",“order”:1}},“params_base”:{“blob_paths”:{“type”:“path”,“value”:null,“title”:“Movies data path”,“desc”:“Absolute path, wildcard-expression (e.g. /mount/data/somewhere/.mrcs) that will be imported. MRC (mrc, mrcs, stk) and TIFF format supported.",“order”:0,“section”:“import”,“advanced”:false,“hidden”:false,“path_dir_allowed”:false,“path_file_allowed”:true,“path_glob_allowed”:true},“gainref_path”:{“type”:“path”,“value”:null,“title”:“Gain reference path”,“desc”:“Absolute path to a single gain reference for all the raw data, in MRC format. Leave blank if data is already gain-corrected.”,“order”:1,“section”:“import”,“advanced”:false,“hidden”:false,“path_dir_allowed”:false,“path_file_allowed”:true,“path_glob_allowed”:false},“defect_path”:{“type”:“path”,“value”:null,“title”:“Defect file path”,“desc”:“Absolute path to a defect file for all the raw data. This should be a .txt file. Leave blank if not applicable.”,“order”:2,“section”:“import”,“advanced”:false,“hidden”:false,“path_dir_allowed”:false,“path_file_allowed”:true,“path_glob_allowed”:false},“gainref_flip_x”:{“type”:“boolean”,“value”:false,“title”:“Flip gain ref & defect file in X?”,“desc”:“Flip gain ref and defect file left-to-right (in X axis)”,“order”:3,“section”:“import”,“advanced”:false,“hidden”:false},“gainref_flip_y”:{“type”:“boolean”,“value”:false,“title”:“Flip gain ref & defect file in Y?”,“desc”:“Flip gain ref and defect file top-to-bottom (in Y axis)”,“order”:4,“section”:“import”,“advanced”:false,“hidden”:false},“gainref_rotate_num”:{“type”:“number”,“value”:0,“title”:“Rotate gain ref?”,“desc”:“Rotate gain ref counter-clockwise by 90 degrees this many times”,“order”:5,“section”:“import”,“advanced”:false,“hidden”:false},“psize_A”:{“type”:“number”,“value”:null,“title”:“Raw pixel size (A)”,“desc”:“Pixel size of the raw movie data in Angstroms”,“order”:6,“section”:“import”,“advanced”:false,“hidden”:false},“accel_kv”:{“type”:“number”,“value”:null,“title”:“Accelerating Voltage (kV)”,“desc”:"",“order”:7,“section”:“import”,“advanced”:false,“hidden”:false},“cs_mm”:{“type”:“number”,“value”:null,“title”:“Spherical Aberration (mm)”,“desc”:"",“order”:8,“section”:“import”,“advanced”:false,“hidden”:false},“total_dose_e_per_A2”:{“type”:“number”,“value”:null,“title”:“Total exposure dose (e/A^2)”,“desc”:"",“order”:9,“section”:“import”,“advanced”:false,“hidden”:false},“negative_stain_data”:{“type”:“boolean”,“value”:false,“title”:“Negative Stain Data”,“desc”:“If Negative Stain Data is on, this indicates that there are light particles on dark background. If it’s off, this indicates the movies have dark particles on light background (cryo-em data).”,“order”:10,“section”:“import”,“advanced”:false,“hidden”:false},“phase_plate_data”:{“type”:“boolean”,“value”:false,“title”:“Phase Plate Data”,“desc”:"",“order”:11,“section”:“import”,“advanced”:false,“hidden”:false},“override_exp_group_id”:{“type”:“number”,“value”:null,“title”:“Override Exposure Group ID”,“desc”:"",“order”:12,“section”:“import”,“advanced”:false,“hidden”:false},“skip_header_check”:{“type”:“boolean”,“value”:false,“title”:“Skip Header Check”,“desc”:“Skip reading of every header file to increase import speed. WARNING: this assumes exposure shapes and extensions are consistent across the entire dataset.”,“order”:13,“section”:“import”,“advanced”:false,“hidden”:false},“eer_num_fractions”:{“type”:“number”,“value”:40,“title”:“EER Number of Fractions”,“desc”:“Number of fractions to make out of the EER input data.”,“order”:14,“section”:“import”,“advanced”:false,“hidden”:false},“eer_upsamp_factor”:{“type”:“number”,“value”:2,“title”:“EER Upsampling Factor”,“desc”:"Upsampling factor when decoding EER input data. Note that the pixel size you provide should be the raw pixel size at the nominal 4k sensor, not the pixel size after EER upsampling. “,“order”:15,“section”:“import”,“advanced”:false,“hidden”:false},“compute_num_cpus”:{“type”:“number”,“value”:4,“title”:“Number of CPUs to parallelize”,“desc”:“Use this many CPUs to read headers in parallel”,“order”:16,“section”:“compute_settings”,“advanced”:false,“hidden”:false}},“params_spec”:{“blob_paths”:{“value”:”/sdf/group/cryoem/exp/202112/20211201-CT10_TEMALPHA/20211201-CT10_33frames/raw/20211201-CT10-dIgAPLUSARP4_slot5_tilted/Images-Disc1/GridSquare_4088851/Data/.mrc”},“psize_A”:{“value”:0.95},“accel_kv”:{“value”:300},“cs_mm”:{“value”:2.7},“total_dose_e_per_A2”:{“value”:49}},“input_slot_groups”:[],“output_result_groups”:[{“uid”:“J94-G0”,“type”:“exposure”,“name”:“imported_movies”,“title”:“Imported movies”,“description”:“Imported movies.”,“contains”:[{“uid”:“J94-R0”,“type”:“exposure.movie_blob”,“group_name”:“imported_movies”,“name”:“movie_blob”,“passthrough”:false},{“uid”:“J94-R1”,“type”:“exposure.mscope_params”,“group_name”:“imported_movies”,“name”:“mscope_params”,“passthrough”:false}],“passthrough”:false,“num_items”:1581},{“uid”:“J94-G1”,“type”:“exposure”,“name”:“failed_movies”,“title”:“Failed movies”,“description”:“Movies that failed to import successfully.”,“contains”:[{“uid”:“J94-R2”,“type”:“exposure.movie_blob”,“group_name”:“failed_movies”,“name”:“movie_blob”,“passthrough”:false}],“passthrough”:false,“num_items”:0}],“output_results”:[{“uid”:“J94-R0”,“type”:“exposure.movie_blob”,“group_name”:“imported_movies”,“name”:“movie_blob”,“title”:“Raw movie data”,“description”:"",“min_fields”:[[“path”,“O”],[“shape”,“3u4”],[“psize_A”,“f4”],[“is_gain_corrected”,“u4”],[“format”,“O”],[“has_defect_file”,“u4”]],“versions”:[0],“metafiles”:[“J94/imported_movies.cs”],“num_items”:[1581],“passthrough”:false},{“uid”:“J94-R1”,“type”:“exposure.mscope_params”,“group_name”:“imported_movies”,“name”:“mscope_params”,“title”:“Exposure parameters”,“description”:"",“min_fields”:[[“accel_kv”,“f4”],[“cs_mm”,“f4”],[“total_dose_e_per_A2”,“f4”],[“phase_plate”,“u4”],[“neg_stain”,“u4”],[“exp_group_id”,“u4”],[“defect_path”,“O”]],“versions”:[0],“metafiles”:[“J94/imported_movies.cs”],“num_items”:[1581],“passthrough”:false},{“uid”:“J94-R2”,“type”:“exposure.movie_blob”,“group_name”:“failed_movies”,“name”:“movie_blob”,“title”:“Raw movie data”,“description”:"",“min_fields”:[[“path”,“O”],[“shape”,“3u4”],[“psize_A”,“f4”],[“is_gain_corrected”,“u4”],[“format”,“O”],[“has_defect_file”,“u4”]],“versions”:[0],“metafiles”:[“J94/failed_movies.cs”],“num_items”:[0],“passthrough”:false}],“output_group_images”:{“imported_movies”:“61e70511cecb8d97246ff0d7”},“errors_build_params”:{},“errors_build_inputs”:{},“errors_run”:[],“running_at”:“2021-12-07T18:05:56.033Z”,“token_acquired_at”:null,“tokens_requested_at”:null,“last_scheduled_at”:null,“resources_needed”:{},“resources_allocated”:{},“run_on_master_direct”:true,“queued_to_lane”:"",“queue_index”:null,“queue_status”:null,“queue_message”:null,“queued_job_hash”:null,“interactive_hostname”:"",“interactive_port”:null,“PID_monitor”:3411,“PID_main”:3412,“PID_workers”:[],“cluster_job_id”:null,“is_experiment”:false,“job_dir”:“J94”,“experiment_worker_path”:null,“enable_bench”:false,“bench”:{},“project_uid_num”:1,“uid_num”:94,“last_exported”:“2022-01-20T01:09:56.843Z”,“queued_to_hostname”:false,“queued_to_gpu”:false,“no_check_inputs_ready”:false,“num_tokens”:0,“job_sig”:“9481775061155261968555886467287970803830534813212076261460352349626788101807333665321045673607719038006240352509805466590001702007926530181407787098384572713616812326811774078055251374363145109097187581392462226282492077556005609010568251119578861724199620718448954717344340558966855461242018244598211249718752059567319813583297456684913337972096877329336578059078448679320763557800846912350156360131861451512915842617759712709734298381296393674559716142982259786194003470401973299558879292730817843477116190922923415501737823353639529112564575459299320702294841346560232946939469321930281143540923364564930238039444”,“tokens_acquired_at”:1638900347.9314334,“status_num”:35,“imported”:true}
@wtempel thanks again for your help.
Below is the metadata for a “Select 2D classes” job, which was later used for ab-initio reconstruction and so on. Up and downstream connections are missing.
{"_id":{"_str":“6089f1e181fe32b74193b850”},“children”:[],“cloned_from”:null,“created_at”:“2021-03-11T18:21:23.209Z”,“deleted”:false,“parents”:[],“project_uid”:“P2”,“queue_message”:null,“status”:“completed”,“title”:“New Job J214”,“type”:“select_2D”,“ui_tile_height”:1,“ui_tile_images”:[{“name”:“templates_selected”,“fileid”:“6089f1e181fe32b74193b844”,“num_rows”:1,“num_cols”:2}],“ui_tile_width”:2,“uid”:“J214”,“workspace_uids”:[“W1”],“job_type”:“select_2D”,“completed_at”:“2021-03-11T18:22:33.072Z”,“created_by_job_uid”:null,“created_by_user_id”:“6089eec6b1609c86b6cd09e2”,“description”:“Enter a description.”,“failed_at”:null,“interactive”:true,“job_dir_size”:192674367,“killed_at”:null,“last_accessed”:{“name”:“alba”,“accessed_at”:“2022-01-21T17:43:40.596Z”},“launched_at”:“2021-03-11T18:21:30.995Z”,“priority”:0,“queued_at”:“2021-03-11T18:21:30.673Z”,“started_at”:“2021-03-11T18:21:31.547Z”,“waiting_at”:“2021-03-11T18:21:34.756Z”,“version”:“v3.1.0”,“run_as_user”:null,“params_secs”:{“general_settings”:{“title”:“General Settings”,“desc”:"",“order”:0},“auto_thresholds”:{“title”:“Auto Thresholds”,“desc”:“Automatically apply thresholds and skip the interactive process”,“order”:1}},“params_base”:{“transpose_templates”:{“type”:“boolean”,“value”:false,“title”:“Transpose templates”,“desc”:"",“order”:0,“section”:“general_settings”,“advanced”:true,“hidden”:true},“selected_templates”:{“type”:“string”,“value”:null,“title”:“Selected templates (comma sep)”,“desc”:"",“order”:1,“section”:“general_settings”,“advanced”:true,“hidden”:true},“resolution_better_than”:{“type”:“number”,“value”:null,“title”:“Classes where resolution better than”,“desc”:"",“order”:2,“section”:“auto_thresholds”,“advanced”:false,“hidden”:false},“particle_count_above”:{“type”:“number”,“value”:null,“title”:“Classes where particle count higher than”,“desc”:"",“order”:3,“section”:“auto_thresholds”,“advanced”:false,“hidden”:false}},“params_spec”:{},“input_slot_groups”:[{“type”:“particle”,“name”:“particles”,“title”:“Particles”,“description”:“Particles”,“count_min”:0,“count_max”:1,“repeat_allowed”:false,“slots”:[{“type”:“particle.blob”,“name”:“blob”,“title”:“Particle raw data”,“description”:"",“optional”:false},{“type”:“particle.alignments2D”,“name”:“alignments2D”,“title”:“Particle 2D alignments”,“description”:"",“optional”:false}],“connections”:[{“job_uid”:“J213”,“group_name”:“particles_selected”,“slots”:[{“slot_name”:“blob”,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“blob”,“result_type”:“particle.blob”,“version”:“F”},{“slot_name”:“alignments2D”,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“alignments2D”,“result_type”:“particle.alignments2D”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“ctf”,“result_type”:“particle.ctf”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“location”,“result_type”:“particle.location”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“alignments3D”,“result_type”:“particle.alignments3D”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“pick_stats”,“result_type”:“particle.pick_stats”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“alignments_class_0”,“result_type”:“particle.alignments3D”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“alignments_class_1”,“result_type”:“particle.alignments3D”,“version”:“F”},{“slot_name”:null,“job_uid”:“J213”,“group_name”:“particles_selected”,“result_name”:“alignments_class_2”,“result_type”:“particle.alignments3D”,“version”:“F”}]}]},{“type”:“template”,“name”:“templates”,“title”:“2D Class Averages”,“description”:“Class averages”,“count_min”:1,“count_max”:1,“repeat_allowed”:false,“slots”:[{“type”:“template.blob”,“name”:“blob”,“title”:“Template raw data”,“description”:"",“optional”:false}],“connections”:[{“job_uid”:“J213”,“group_name”:“templates_selected”,“slots”:[{“slot_name”:“blob”,“job_uid”:“J213”,“group_name”:“templates_selected”,“result_name”:“blob”,“result_type”:“template.blob”,“version”:“F”}]}]}],“output_result_groups”:[{“uid”:“J214-G0”,“type”:“particle”,“name”:“particles_selected”,“title”:“Particles selected”,“description”:"",“contains”:[{“uid”:“J214-R0”,“type”:“particle.blob”,“group_name”:“particles_selected”,“name”:“blob”,“passthrough”:false},{“uid”:“J214-R1”,“type”:“particle.alignments2D”,“group_name”:“particles_selected”,“name”:“alignments2D”,“passthrough”:false},{“uid”:“J214-R2”,“type”:“particle.ctf”,“group_name”:“particles_selected”,“name”:“ctf”,“passthrough”:true},{“uid”:“J214-R3”,“type”:“particle.location”,“group_name”:“particles_selected”,“name”:“location”,“passthrough”:true},{“uid”:“J214-R4”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments3D”,“passthrough”:true},{“uid”:“J214-R5”,“type”:“particle.pick_stats”,“group_name”:“particles_selected”,“name”:“pick_stats”,“passthrough”:true},{“uid”:“J214-R6”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_0”,“passthrough”:true},{“uid”:“J214-R7”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_1”,“passthrough”:true},{“uid”:“J214-R8”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_2”,“passthrough”:true}],“passthrough”:“particles”,“num_items”:241081},{“uid”:“J214-G1”,“type”:“template”,“name”:“templates_selected”,“title”:“Templates selected”,“description”:"",“contains”:[{“uid”:“J214-R9”,“type”:“template.blob”,“group_name”:“templates_selected”,“name”:“blob”,“passthrough”:false}],“passthrough”:false,“num_items”:76},{“uid”:“J214-G2”,“type”:“particle”,“name”:“particles_excluded”,“title”:“Particles excluded”,“description”:"",“contains”:[{“uid”:“J214-R10”,“type”:“particle.blob”,“group_name”:“particles_excluded”,“name”:“blob”,“passthrough”:false},{“uid”:“J214-R11”,“type”:“particle.alignments2D”,“group_name”:“particles_excluded”,“name”:“alignments2D”,“passthrough”:false},{“uid”:“J214-R12”,“type”:“particle.ctf”,“group_name”:“particles_excluded”,“name”:“ctf”,“passthrough”:true},{“uid”:“J214-R13”,“type”:“particle.location”,“group_name”:“particles_excluded”,“name”:“location”,“passthrough”:true},{“uid”:“J214-R14”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments3D”,“passthrough”:true},{“uid”:“J214-R15”,“type”:“particle.pick_stats”,“group_name”:“particles_excluded”,“name”:“pick_stats”,“passthrough”:true},{“uid”:“J214-R16”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_0”,“passthrough”:true},{“uid”:“J214-R17”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_1”,“passthrough”:true},{“uid”:“J214-R18”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_2”,“passthrough”:true}],“passthrough”:“particles”,“num_items”:1537},{“uid”:“J214-G3”,“type”:“template”,“name”:“templates_excluded”,“title”:“Templates excluded”,“description”:"",“contains”:[{“uid”:“J214-R19”,“type”:“template.blob”,“group_name”:“templates_excluded”,“name”:“blob”,“passthrough”:false}],“passthrough”:false,“num_items”:5}],“output_results”:[{“uid”:“J214-R0”,“type”:“particle.blob”,“group_name”:“particles_selected”,“name”:“blob”,“title”:“Particle raw data”,“description”:"",“min_fields”:[[“path”,“O”],[“idx”,“u4”],[“shape”,“2u4”],[“psize_A”,“f4”],[“sign”,“f4”]],“versions”:[0],“metafiles”:[“J214/particles_selected.cs”],“num_items”:[241081],“passthrough”:false},{“uid”:“J214-R1”,“type”:“particle.alignments2D”,“group_name”:“particles_selected”,“name”:“alignments2D”,“title”:“Particle 2D alignments”,“description”:"",“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/particles_selected.cs”],“num_items”:[241081],“passthrough”:false},{“uid”:“J214-R2”,“type”:“particle.ctf”,“group_name”:“particles_selected”,“name”:“ctf”,“title”:“Passthrough ctf”,“description”:“Passthrough from input particles.ctf (result_name)”,“min_fields”:[[“type”,“O”],[“exp_group_id”,“u4”],[“accel_kv”,“f4”],[“cs_mm”,“f4”],[“amp_contrast”,“f4”],[“df1_A”,“f4”],[“df2_A”,“f4”],[“df_angle_rad”,“f4”],[“phase_shift_rad”,“f4”],[“scale”,“f4”],[“scale_const”,“f4”],[“shift_A”,“2f4”],[“tilt_A”,“2f4”],[“trefoil_A”,“2f4”],[“tetra_A”,“4f4”],[“anisomag”,“4f4”],[“bfactor”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R3”,“type”:“particle.location”,“group_name”:“particles_selected”,“name”:“location”,“title”:“Passthrough location”,“description”:“Passthrough from input particles.location (result_name)”,“min_fields”:[[“micrograph_uid”,“u8”],[“exp_group_id”,“u4”],[“micrograph_path”,“O”],[“micrograph_shape”,“2u4”],[“center_x_frac”,“f4”],[“center_y_frac”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R4”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments3D”,“title”:“Passthrough alignments3D”,“description”:“Passthrough from input particles.alignments3D (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R5”,“type”:“particle.pick_stats”,“group_name”:“particles_selected”,“name”:“pick_stats”,“title”:“Passthrough pick_stats”,“description”:“Passthrough from input particles.pick_stats (result_name)”,“min_fields”:[[“ncc_score”,“f4”],[“power”,“f4”],[“template_idx”,“u4”],[“angle_rad”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R6”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_0”,“title”:“Passthrough alignments_class_0”,“description”:“Passthrough from input particles.alignments_class_0 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R7”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_1”,“title”:“Passthrough alignments_class_1”,“description”:“Passthrough from input particles.alignments_class_1 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R8”,“type”:“particle.alignments3D”,“group_name”:“particles_selected”,“name”:“alignments_class_2”,“title”:“Passthrough alignments_class_2”,“description”:“Passthrough from input particles.alignments_class_2 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_selected.cs”],“num_items”:[241081],“passthrough”:true},{“uid”:“J214-R9”,“type”:“template.blob”,“group_name”:“templates_selected”,“name”:“blob”,“title”:“Template raw data”,“description”:"",“min_fields”:[[“path”,“O”],[“idx”,“u4”],[“shape”,“2u4”],[“psize_A”,“f4”],[“res_A”,“f4”]],“versions”:[0],“metafiles”:[“J214/templates_selected.cs”],“num_items”:[76],“passthrough”:false},{“uid”:“J214-R10”,“type”:“particle.blob”,“group_name”:“particles_excluded”,“name”:“blob”,“title”:“Particle raw data”,“description”:"",“min_fields”:[[“path”,“O”],[“idx”,“u4”],[“shape”,“2u4”],[“psize_A”,“f4”],[“sign”,“f4”]],“versions”:[0],“metafiles”:[“J214/particles_excluded.cs”],“num_items”:[1537],“passthrough”:false},{“uid”:“J214-R11”,“type”:“particle.alignments2D”,“group_name”:“particles_excluded”,“name”:“alignments2D”,“title”:“Particle 2D alignments”,“description”:"",“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/particles_excluded.cs”],“num_items”:[1537],“passthrough”:false},{“uid”:“J214-R12”,“type”:“particle.ctf”,“group_name”:“particles_excluded”,“name”:“ctf”,“title”:“Passthrough ctf”,“description”:“Passthrough from input particles.ctf (result_name)”,“min_fields”:[[“type”,“O”],[“exp_group_id”,“u4”],[“accel_kv”,“f4”],[“cs_mm”,“f4”],[“amp_contrast”,“f4”],[“df1_A”,“f4”],[“df2_A”,“f4”],[“df_angle_rad”,“f4”],[“phase_shift_rad”,“f4”],[“scale”,“f4”],[“scale_const”,“f4”],[“shift_A”,“2f4”],[“tilt_A”,“2f4”],[“trefoil_A”,“2f4”],[“tetra_A”,“4f4”],[“anisomag”,“4f4”],[“bfactor”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R13”,“type”:“particle.location”,“group_name”:“particles_excluded”,“name”:“location”,“title”:“Passthrough location”,“description”:“Passthrough from input particles.location (result_name)”,“min_fields”:[[“micrograph_uid”,“u8”],[“exp_group_id”,“u4”],[“micrograph_path”,“O”],[“micrograph_shape”,“2u4”],[“center_x_frac”,“f4”],[“center_y_frac”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R14”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments3D”,“title”:“Passthrough alignments3D”,“description”:“Passthrough from input particles.alignments3D (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R15”,“type”:“particle.pick_stats”,“group_name”:“particles_excluded”,“name”:“pick_stats”,“title”:“Passthrough pick_stats”,“description”:“Passthrough from input particles.pick_stats (result_name)”,“min_fields”:[[“ncc_score”,“f4”],[“power”,“f4”],[“template_idx”,“u4”],[“angle_rad”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R16”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_0”,“title”:“Passthrough alignments_class_0”,“description”:“Passthrough from input particles.alignments_class_0 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R17”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_1”,“title”:“Passthrough alignments_class_1”,“description”:“Passthrough from input particles.alignments_class_1 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R18”,“type”:“particle.alignments3D”,“group_name”:“particles_excluded”,“name”:“alignments_class_2”,“title”:“Passthrough alignments_class_2”,“description”:“Passthrough from input particles.alignments_class_2 (result_name)”,“min_fields”:[[“split”,“u4”],[“shift”,“2f4”],[“pose”,“3f4”],[“psize_A”,“f4”],[“error”,“f4”],[“error_min”,“f4”],[“resid_pow”,“f4”],[“slice_pow”,“f4”],[“image_pow”,“f4”],[“cross_cor”,“f4”],[“alpha”,“f4”],[“alpha_min”,“f4”],[“weight”,“f4”],[“pose_ess”,“f4”],[“shift_ess”,“f4”],[“class_posterior”,“f4”],[“class”,“u4”],[“class_ess”,“f4”]],“versions”:[0],“metafiles”:[“J214/P1_J214_passthrough_particles_excluded.cs”],“num_items”:[1537],“passthrough”:true},{“uid”:“J214-R19”,“type”:“template.blob”,“group_name”:“templates_excluded”,“name”:“blob”,“title”:“Template raw data”,“description”:"",“min_fields”:[[“path”,“O”],[“idx”,“u4”],[“shape”,“2u4”],[“psize_A”,“f4”],[“res_A”,“f4”]],“versions”:[0],“metafiles”:[“J214/templates_excluded.cs”],“num_items”:[5],“passthrough”:false}],“output_group_images”:{“templates_selected”:“6089f1e181fe32b74193b842”,“templates_excluded”:“6089f1e181fe32b74193b84a”,“particles_selected”:“6089f1e181fe32b74193b84c”,“particles_excluded”:“6089f1e181fe32b74193b84e”},“errors_build_params”:{},“errors_build_inputs”:{},“errors_run”:[],“running_at”:“2021-03-11T18:22:19.309Z”,“token_acquired_at”:null,“tokens_requested_at”:null,“last_scheduled_at”:null,“resources_needed”:{},“resources_allocated”:{},“run_on_master_direct”:true,“queued_to_lane”:"",“queue_index”:null,“queue_status”:null,“queued_job_hash”:null,“interactive_hostname”:"",“interactive_port”:57037,“PID_monitor”:32311,“PID_main”:32312,“PID_workers”:[],“cluster_job_id”:null,“is_experiment”:false,“job_dir”:“J214”,“experiment_worker_path”:null,“enable_bench”:false,“bench”:{},“project_uid_num”:1,“uid_num”:214,“ui_layouts”:{“P2”:{“show”:true,“floater”:true,“top”:0,“left”:24807,“width”:298,“height”:192,“groups”:[]},“P2W1”:{“show”:true,“floater”:true,“top”:0,“left”:24807,“width”:298,“height”:192,“groups”:[]}},“last_exported”:“2022-01-19T12:53:40.784Z”,“queued_to_hostname”:false,“queued_to_gpu”:false,“no_check_inputs_ready”:false,“num_tokens”:0,“job_sig”:“9156006096563139227933048378638513853576709495228464893220546361670027053646819036849339310573097103719483364147646049127606327605119339367362400056168692355583510404600515625505742102321286068581264602358542326622551242240696008130414544596855260353510206015354677217410644251313893308838298746510486064451368478523830832855657966805078514203511483867944068394620547532761241649232781617951579986199181783943417299609910063867671028059053232535466319717024551950949037722545719359199401175599826908353713283677826598773153015757245873076926497301466424780203356889772131022768415855288363731958956833590048294108251”,“tokens_acquired_at”:1615486890.9939601,“status_num”:35,“imported”:true}
@wtempel, I was wondering if progress was made to fix this issue. The error messages possibly linked to these broken relationships (please see Error in new or old jobs on imported project), particularly " AssertionError: Unsupported MRC datatype: 1067200494", are keeping me from moving forward with the data processing at a critical moment.
I thank you again for your attention and support.
Following with interest. I also have a similar problem. The dependencies are still in the individual jobs (I can see the inputs) but I lose the ability to visualise the interconnections by tree view.
Hi!
Yes, indeed, as @ALBA mentioned this job dependencies issue is connected with certain jobs that stopped working at the same time. Although not job dependencies showing correctly, I attempted the more cumbersome way to try to continue the workflow of processing a dataset and so I found myself stuck with several errors, like the one described by @ALBA.
So processing with cryoSPARC has been unfortunately on hold for some time. Did anyone try to do a clean installation of the same version of cryoSPARC?
I might try to do that next week when I have more time, unless if anyone knows it won’t help.
Another option is to downgrade, although I would be happy to use the latest functionalities of cryoSPARC.
Hi all. We have identified a likely cause and are working on a patch.
@wtempel thank you for working on this issue; hopefully, the patch will fix it.
@AndreGraca The major problem for me was the “unsupported mrc datatype” error, which kept me from doing anything, for real, with the particle stack. I ended up walking backward, step by step, to the beginning of my data processing tree (tens of jobs). I identified that several motion-corrected micrographs were somehow corrupted, though they still appeared just fine during manual curation. Thus, I have cloned and repeated every job, including patch motion correction again, and faithfully reproduced the volume and statistics obtained previously.
All the best,
The Patch is out and it works! Thanks @wtempel and everyone in the team that worked on this!
So so so happy to see the tree view again! =D
Don’t forget to cryosparcm cli "update_parents_and_children_for_project(project_uid='P1')" (example for P1) after patch installation.
Best of luck,
André
