504 - Impeached from the Hall of Fame?

What would it take to get impeached from any of the Halls of Fame - how could it even happen and how might that affect sports card values?