How to access FSC output in v4

Features and Functionality
1

Through Scipion, I downloaded the .txt file where the values of the FSC are saved in some jobs to then create the graphs:
I accessed using the “request” python package and passed a url (eg http://localhost:39000/api/files/633d8c7486073b4db2127751) with the file id. With previous versions of cryoSPARC there were no problems because authentication was not required, now it asks me to authenticate in order to access. Is there a way for me to be able to download this file without authentication?

2

Hi @cfonseca ,

Apologies for the delay! Thank you for your post. In v4 we’ve added some extra steps to ensure API endpoints and queries/commands made to the CryoSPARC MongoDB are not made accidentally. As part of this effort the web application endpoints require a valid session token. You can read more about these changes on our guide.

To access these files, I would recommend writing a script to query the database directly (fs.files collection with _id from the API url /api/files/<ID>) and download the result. Here’s an example of how you can connect to MongoDB from an external script:

eval $(cryosparcm env)
mongo_uri=$(python -c "from cryosparc_compute import database_management; print(database_management.get_mongo_uri('meteor'))")
echo $mongo_uri

Hope this helps!

- Suhail

3

Thanks @sdawood ! Here is our situation:

We are using Cryosparc from the command line (from Scipion). So far all is migrated to use CS4 without any issue except the FSC.

Your suggestion could be an option but it implies to have a MongoDB client in our Scipion environment. We are trying to move to the other direction (not installing many packages in scipion core environment) to avoid dependency problems or a heavy environment. Thus this will fore us to create a dedicated environment that could have the mongo client and also pwem.

Since this is an option, I’d like to make some other suggestions:

1.- implement app tokens. We could try to mimic a “browser” and get the session token, but this will imply to ask the user to his/her user and password. And we do no want this. Nor probably you nor the user. With an app token we should be able to reach those secured urls.

2.- Free FSC data (not just the image but the numbers) from being secure. I can see the is sensible information in CS, but the FSC alone might not be that important?

Cheers, all the best, Pablo.

4

Hi @pconesa1,

Thanks for your feedback. In a future release, we will be adding the ability to retrieve any image stored in the database (FSC plots, orientation distributions, micrograph thumbnails, etc.,) via the HTTP API provided by the command_vis service (CRYOSPARC_BASE_PORT+3)

In order to use the API, you need to add License-ID to your request’s header. The License-ID value must match the value of CRYOSPARC_LICENSE_ID inside cryosparc_master/config.sh.
For example:

headers = {'License-ID': 'xxx-xxx-xxx'}
...
r = requests.get(url, data, headers=headers, timeout=timeout)

We will update this post with more details (including the url to use) once this feature is deployed.

5

Hi @pconesa1 ,

In CryoSPARC v4.1 (released December 12, 2022) we’ve added functionality to the command_vis service (CRYOSPARC_BASE_PORT +3) that allows for downloading job event log data stored in MongoDB GridFS via an HTTP call.

In Python, you’ll need the following packages imported:

import requests
import uuid
from IPython.display import Image, display # to view images in a notebook

To list all files stored by the job in GridFS, you can use the following endpoint:

job_db_files = requests.post(
    'http://localhost:62003/api',
    json={
        'method': 'list_job_files',
        'params': {'project_uid': 'P135', 'job_uid': 'J550'},
        'jsonrpc': '2.0',
        'id': uuid.uuid4().hex
    },
    headers={'License-ID': '<CRYOSPARC_LICENSE_ID>'}
)
job_db_files = r.json()

To extract the binary data from any MongoDB file _id (whether it be text or images):

fsc_img = requests.post(
    'http://localhost:62003/get_job_file',
    json={'fileid': '639742e85a144297ce7714ce'}, # MongoDB _id of File
    headers={'License-ID': '<CRYOSPARC_LICENSE_ID>'}
)
display(Image(fsc_img.content))

Here’s how it looks all together:

image
image1260×1898 436 KB

Hope this helps!

- Suhail

3 Likes
6

Thanks sdawood!!! Now, I have access to the fsc data!!! :partying_face:

1 Like