Technically, impedance is resistance plus a bit extra that only shows up with alternating current, and since music is alternating current, the extra matters. You don't need the physics. Lower ohms mean the speaker draws more current for the same volume, so the amplifier works harder. That's genuinely all you need.
The one rule
Amplifier specs state the lowest impedance they're rated for, usually 4 Ω. Keep the speaker's rating at or above that number and you're fine:
- 8 Ω speakers on any amp: always fine.
- 4 Ω speakers on an amp rated down to 4 Ω: fine.
- 4 Ω speakers on an amp rated only to 8 Ω: works at low volume, overheating territory when driven hard. Avoid.
Nominal versus reality
The number on the box is an average across the whole audible range. A speaker's actual impedance changes with the tone being played, so a nominally 8 Ω speaker might briefly present 4 Ω somewhere in the bass. When people call a speaker a difficult load, they mean those dips, not the label.
In practice
Modern solid-state amps and ordinary 4 to 8 Ω speakers are a solved combination; stop worrying. Electrostatics and speakers that dip under 3 Ω deserve a proper look at the amplifier's ratings. Old tube amplifiers are the real exception, because they have output taps for 4, 8 and 16 Ω that must match the speaker, or both the sound and the output transformer suffer.
Impedance also says nothing about sound quality. A 4 Ω speaker isn't better or worse than an 8 Ω one; it's an electrical matching question and nothing more.