The airport will be shut all Monday and all flights cancelled, affecting up to 16,000 passengers, the BBC quoted an airport spokeswoman as saying.
The bomb was discovered at on Sunday morning at the George V Dock during pre-planned work at the airport in east London, the Met Police said.
The airport was shut at 10 p.m., and the Met said that it was working with the Royal Navy to remove the device.
Passengers have been asked not to travel to the airport as the terminal is closed and to contact their airline, the BBC reported.
Airport CEO Robert Sinclair apologised and said: “I recognise this is causing inconvenience for our passengers, and in particular some of our local residents.
“The airport is cooperating fully with the Met Police and Royal Navy and working hard to safely remove the device and resolve the situation as quickly as possible.”
Airlines using London City Airport include British Airways, Flybe, CityJet, KLM and Lufthansa, with flights to domestic and European city destinations.
The Met Police said a 214-metre exclusion zone had been set up and properties inside were evacuated.