Because there is no force of suction or attraciton from Vacuum side. The only force responsible is the pressure force which is because of gas moving from high pressure to low pressure.
And there is no change in kinetic energy because of collision from walls of container as the nature of collision is Elastic in Nature.