Britain, France, the Netherlands and Spain didn't peaceably let go of their colonies. They were all defeated in colonial wars.
Britain: remember the American Revolution, the Boer War, Mau-Mau in Kenya, revolts in Malaya and Burma, mass murder in Amritsar in 1919 and then Gandhi in India?
France: Algeria and Indochina.
Spain: Bolivar in South America, revolts in Cuba and Mexico.
The Netherlands: Indonesia.
Portugal: failed colonial wars brought down the fascist regime in 1974.