Magnetism question

Therefore, if for whatever reason, you decide to measure the velocity at which a uniform magnetic field moves with respect to a fixed reference point and the velocity of the charged particle with respect to the same reference point, you must subtract the values from each other to obtain the velocity of the charged particle with respect to the magnetic field.

See I kind of understand that, and if I ever got a problem like that I would do that, and none of the textbook problems are like that because the fields are assumed to be stationary. But here, I'm kind of wondering for a personal "... this doesn't quite make sense" kind of thing

Say, there's a field something like this


If the field isn't moving, then it has a value of --> (whatever that is) everywhere. If it is moving, it still has a value of --> everywhere. You can't tell the difference just by measuring the field anywhere, but it affects particles in different ways, so what is making it different?