I think the way graphics matter is by making something visually appealing. it gives you that first step into something. Its like a first impression. so in that sense it does matter.

I personally care more about the context of the visuals. If the game is going for hyper realism then thats what I expect. If the game is being more stylized then thats how I judge it. I wouldn't compare a Link to the pasts visuals with say battlefield 1 because each is trying to achieve something different with their graphics.

I would compare battlefield 1 with Mass effect Andromeda because both are trying to look realistic. BF1 does a good job while Mass Effect fails miserably.