You run 4 cache servers. The obvious way to pick one is server = hash(key) % N. It spreads keys evenly and needs no lookup table.
Traffic grows, so you add a fifth server. N changes from 4 to 5, and almost every remainder changes with it. Watch how many keys land somewhere new.