I have a matching numbers Black Shadow, and a Rapide assembled from parts so that nothing matches but the case halves. Certainly the numbers have no bearing on the riding experience, but that's not the appeal - it's the history, which has been better preserved in the matching numbers bike. It's a time capsule from the date of production.
Does intangible history matter? To many, not at all. But the idea is common in the antiques world - Napoleon's sword is worth more than its identical counterpart owned by an anonymous lieutenant, even if the latter slices better.
If I were buying a Vincent today, I would probably not pay the (large) premium matching numbers demand. Cost vs. benefit, always a subjective call. But it may be different if you care about future value - I expect (without having studied it) that matching numbers vintage bikes and cars will tend to appreciate more than others.