Answer:
The 32-bit equation solver is selected in Calculation Parameters (see the image). You have probably used an older file as a template. The 32-bit solver can use a maximum of 2 GB of RAM, no matter how much RAM the computer actually has.
If you select the 64-bit solver, the calculation will run smoothly.