Because it takes more force to accelerate a mass, and gravity exerts more force on a greater mass. So every object winds up with the same acceleration.
And THAT, of course, says that all objects in the same gravity will pick up the same speed in the same time, and fall the same distance in the same time.
