It does affect it. See it like this. A mob dies, now the system calculates the number of loots, if it says 1 rare 1 blue 2 white it will drop for you. IF you're wearing a high amount of magic find, the system decides which items will "upgrade". So you have a chance that the whites turn into blue/rare/legendary or the blues into rares..nobody knows how it changes it, but it does