It's possible this is like the physical diffrences in brains that stop people beng able to perceive faces, or objects, or movement.. all different parts of the brain process these and are separate constructs.

could it be a problem of pitch recognition i wonder? as tiny hairs in our ears pick up the frequencies, but then the transmission of those through neurons could be disorganized somehow.
though perhaps there are a range of different problems that can cause amusia with different effects on the perception of sound...

imagine having this in conjunction with synaesthesia..!