Is this in vCloud 5.1.2 or 5.1.0/5.1.1? If you are on 5.1.2 and using a single cluster for allocation pools (non-elastic) you might want to force it into elastic mode.
explanation is here for the Memory usage:
Allocation Pool Organization VDC Changes in vCloud Director 5.1.2