Unlike most usage pairs, this one has no fixed answer — the correct word changes depending on who is speaking and where they are. That is why it feels slippery. It works exactly like "come" and "go": movement toward the reference point takes "bring," movement away takes "take," and the reference point shifts with the speaker.
Movement toward the speaker or the place being discussed.
Movement away from the speaker or the place being discussed.
Quick way to remember
Ask where the thing ends up relative to you. Coming here is "bring," going there is "take" — the same pairing as "come" and "go." If you are unsure, work out which of those two you would use, and match it.