I am in similar shoes as you.
I accidentally downvoted posts or comments once or twice. I remember removing at least one downvote, so I also very rarely (almost never) downvoted anything on the Steem blockchain so far, but after HardFork 21 I will try to use my downvotes in a similar way as you.
Downvoting scam and spam posts/comments is also a good idea, not just/only bidbotted posts with questionable quality.
I will use my downvotes responsibly, and I will downvote only scam/spam posts/comments, and bidbotted posts with bad quality. Maybe I will join the FlagTrail.
I would like to encourage everyone to do the same, and make the Steem blockchain a better place.