While 2nd homeowners and investors should pay more than those in a primary residence (which they do if they have no homestead exemption), any attempts 'stick it to the interlopers' as it were could backfire. Whether it is a primary home or not, a luxury or not, increasing taxes and insurance on those folks to unreasonable levels does not do anyone any good either. Many of the 2nd homeowners have owned in the area for a long time and are no more speculating that someone with a primary residence. They may have a reasonable mortgage and enough income to support the property, but if you tax everyone out of the market, which home do you think is going to go up for sale or into foreclosure? Not the primary residence, wherever it may be. And of course while that would not be as serious as someone losing their primary residence, a bunch of foreclosures/defaults/properties for sale doesn't help the local economy either.
Face it, the area is a tourist area and is supported to a great deal by 2nd homeowners and renters of those properties.