Answer:   Because they'd enslave the black M&M's, steal all the red M&Ms' land, hunt the blue M&M's to extinction, accuse the yellow M&M's of obstructing trade, start a panic that the little green M&M's were invading the Earth, and complain that the damn brown M&M's were taking all their jobs.

Recognize any of those....?