How can retro-reflective materials be used for invisibility?

Unlike other matierals, which refract light and ter it in different directions, retro-reflective matieral reflects light in the exact same direction it came from, giving it the appearence that nothing is really there.