Yes it probably is the heat shield. Go to a big box store and find
some solid aluminum wire - say 12 gauge and then jack her up when she
is cold - use jack stands to protect you from getting crushed - whack
the exhaust pipe - heat shields till you find the rattle - wrap some
wire around the rattling heat shield and twist it tight.

Removing the heat shield will work but it increases the danger of fire
from parking over dry grass or leaves - not a smart thing to do even
with an intact heat shield.