I used to design broadcast audio gear for a living. How balanced inputs and outputs are designed is more important that the difference between single ended RCA and balanced XLR.
Putting aside the internal balanced component architecture for a minute.....
A well designed balanced input will have significant common mode noise rejection, expressed as the Common Mode Rejection Ratio (CMRR). CMRR is a measure of how well a balanced input stage rejects noise, usually powerline noise. A good input stage will have 60 - 80 dB of CMRR @ 60 Hz. I don’t see a lot of consumer gear manufacturers specifying CMRR. Another advantage of using XLR connectors is they lock, have high pressure contacts, and are just generally physically more robust that RCA.
By contrast, a single ended RCA interconnect has no common mode noise rejection. Worse that that, the cable shield is the signal ground and so you are connecting different component signal grounds together, which, depending on their design, especially power supplies, is inviting hum and ground loops into your system.
On the output side, if a balanced output is truly balanced and floating you can short one side to ground and have the level on the other side double. This is easy to do with transformers, much harder to do with solid state or tube output sections. I suspect most consumer gear won’t pass this test.
As far as sound, well that depends on how well the gear is engineered. Balanced interconnects may require more circuitry, which could degrade the sound. But there are exceptions, like the McIntosh MC 275, where an additional gain section is used for the single ended inputs as opposed to the balanced inputs.
On balance, even for short runs, the advantage of balanced interconnects not running signal ground on the cable shield between components is significant enough that I will default to balanced unless I can prove to myself that unbalanced sounds better or unbalanced is the only alternative.
