Hi! I’m a relatively new user and I often need to redo processes when I make mistakes. It would be really helpful if the amount of time a job took were available in the metadata. (If it’s already available somewhere else, let me know!) Though I know that no two instances of the same kind of job will take the same amount of time, I’d like to be able to use that information roughly to optimize my use of my own time. I currently use time data from active jobs to estimate this, but sometimes there’s enough variability per process that it’s tough to get a good estimate. Thank you!
Hi @kmradford ,
Thanks for the suggestion! We definitely have plans to make this information more accessible in an upcoming update to the interface. For now, you can still gather these stats by querying the cryoSPARC database (MongoDB) directly.
First, enter the MongoDB shell: cryosparcm mongo
Here’s an example query that finds all completed jobs this year and reports the total and average runtime in minutes grouped by job type:
db.jobs.aggregate([
{ $match: { status: 'completed', deleted: false, completed_at: { $gte: new Date('01/01/2022'), $lt: new Date() } } },
{ $project: { job_type: 1, running_at: 1, completed_at: 1, duration: {$divide: [{$subtract: ["$completed_at", "$running_at"]}, 60000]} } },
{ $group: { '_id': '$job_type', count: { $sum: 1 }, totalRunimeMin: { $sum: '$duration' }, averageRunimeMin: { $avg: '$duration' } } },
{ $sort: { totalRunimeMin: -1 } }
])
You should see a list of outputs similar to this:
{
"_id" : "class_2D",
"count" : 172.0,
"totalRunimeMin" : 2642.05511666667,
"averageRunimeMin" : 15.3607855620155
}
(172 completed 2D Classification jobs, averaging ~15 minutes for a total runtime of 44 hours)
If you’re unsure about what the job type is referring to, you can inspect a job and refer to the job_type field within the ‘Metadata’ tab:
In the above example, 2D Classification is class_2D.
You can modify this query based on your preference. For example, if you would like to see stats for a single project, add project_uid: 'P1' to the $match object
To exit the MongoDB shell, type exit
Hope this helps!
- Suhail
1 Like