I hope you got this solved, but in case you did not...
I would systematically begin troubleshooting by eliminating ALL plugins. Then, activate Gantry. See how that goes. If you get the same performance issue with just Gantry enabled, you may wish to backup your data and try a fresh install.
If the WordPress and Gantry only works great, then systematically enable and test each additional plugin. Do the same thing for the template. This will at least get you to a point of finding where the problem originates. I find it hard to believe it is originating with Gantry itself. I've been using it for WordPress and Joomla! for developing custom templates and RIA's and it is killer for base speed optimizations.
Good luck and keep us updated to your progress. If it is another plugin that is not playing nice it would be great to log it here.
Jeff
When someone is impatient and says, "I haven't got all day," I always wonder, How can that be? How can you not have all day?