Raw Data Set Download from EMPIAR

Cryo-EM Data Processing
1

I am encountering issues while downloading raw datasets from the EMPIAR database. Despite having a stable internet connection, the download speed is consistently limited to around 10–11 MB/s. Additionally, large file downloads frequently fail, disrupting the data retrieval process.

I have tried using Aspera for data transfer, but it fails multiple times, making it unreliable for large downloads. Any guidance on best practices for handling large dataset downloads from EMPIAR would be greatly appreciated.

2

EBI EMPIAR appears to be limited to 100 Mbps (per user). At least I’ve never seen faster than that from it.

PDBJ EMPIAR is faster if in Asia.

I usually just fire up a tmux session and use wget overnight. With the exception of the really monster datasets, most are downloaded in 24 hours or so.