I like your ideas to have two different types down votes.

I agree that having a payment down vote "to be used when you feel that a given post (or comment) has received a bigger payout than it deserves" will result in more accurate curation.

It also makes sense that there should be a separate spam/abuse flagging/down vote as well.

Separating the two eliminates confusion as to the reason for the down vote.
I think your idea, and @djleo's idea could be combined. He suggested that we gamify downvoting: https://steemit.com/steemit/@djleo/proposal-drastically-improve-steemit-quality-control-with-incentives