3D Flex Data prep - RAM unavailable


3D Flex was released exactly at the right time for us, as it is precisely what our project needs right now!
Unfortunately, I, like some others, have trouble with the RAM requirements. I tried running 3D Flex data prep, but it does not start (“RAM not available”). The workstation I am using has 24 CPU cores, 32 GB RAM, 1 GPU, so it is unsurprising RAM is limiting.

I therefore tried decreasing the number of particles from 460,000 particles first to 100,000 then to 10,000 particles. Neither job did start. My original particle size is 256 pixels, I had specified in the data prep jobs to crop to 220px and then resize to a training box size of 50px. I have also tested re-extracting to a box size of 50px using an extract particles job and using that as input for the data prep job. This also did not start, not even when additionally limiting the number of 50px particles to 10,000. Is this because of the 32 GB RAM? It seems quite extreme to me, that not even the jobs with severely limited input data would start.

How can I tell the required memory for the job (and the other 3D flex jobs)? I am mainly asking to see if it would be worth to upgrade our RAM, and if so, by how much. Are there any workarounds to limit memory usage, ie by splitting the job somehow?

I would appreciate some feedback on this. Many thanks in advance.

Welcome to the forum @katsie
Please see the response to a related question here: 3DFlex train job - RAM unavailable - #2 by apunjani.