I think almost anyone who spends effort to put vinyl on equivalent digital footing would approach this quesiton like this:

• the equivalent bit depth of vinyl “samples” is not even 16 bits – the noise floor is higher on vinyl than on a redbook CD. A vinyl “sample” might be 14 bits, at best.
• The sampling rate is not infinite, either, that which you referred to as “infinite bitrate because it is analog”. The reason is that your stylus is not one atom wide point sampler, but rather a kind of gigantic pyramid in atom scale, and the material and its cutting process results in grooves of specific minimum size that is practically achievable before the features are either too thin, or the recording head too brittle, or the material heating too much due to friction of the playback, or whatever. This actually affects the maximum frequency that can be stored and played back, which can be used to compute its sampling rate. I have heard that vinyl might go up to 50 kHz, but that just means that digital sampling rate 100 kHz and above is superior, then.

If you understand digital audio, these are actually 100% valid arguments that put vinyl on equal footing with digital, and suggest that any 16-bit hi-res file is already likely to be technically superior. There remains the question of whether it matters that you get ultrasonic content to play back, because you can’t hear it, our amplifiers may have trouble amplifying it, and our speakers are not likely to play those frequencies back well in any case due to being inaudible to humans.