By keeping the number of loaded and running apps to a minimum, Apple is hoping to avoid getting into a tight memory situation.
It's an easy solution for the OS writers. Not so much for developers, who might need to handle saving their state and reloading it the next time you click their icon and the app is reloaded into RAM from flash.
Not the best for battery life either, since extra cycles are used for saving, then reloading / restoring data.
All that said, 128MB is a fair amount. Most handhelds, with only 64MB, do run into tight memory. Those with 128MB are much better off.