RE: Do you ever get that feeling?
I would suggest that you have a deeper issue. Perhaps parents or teachers or siblings or just your own psyche. Even the most loving parents can cause you to have a 'need to prove' edge in your personality. An achieving sibling can create insecurity. Teachers with the 'can do better' phrase. If they encouraged you and praised you when you did a good thing but chastised you when bad - which is a normal way to be with children, and if your psyche / brain was over sensitive to the put down then you feed on the need to prove yourself.
You have to understand that your wife wouldn't have married you if she didn't see the good in you - unless it was a shotgun wedding. You have friends who like you as you or they wouldn't be a friend. And as far as I know it is said that God made you as you are and loves you as you are. So cheer up and be happy that you are the individual that you are, even if that means being sensitive more than some. It certainly doesn't make you less worthy, if anything it makes you more - more likeable, more lovable and more worth it.
And for the record I spent 50 of my 54 years on this world thinking everyone else was better than me. Now I know different.