The building is in the upmarket Salamanca area of the city.
Three storeys of the building at the centre of the blast are said to be affected.
It is not yet known if anyone has been injured or killed, but people may have been trapped according to local reports.
Two schools are near the spot where the explosion occurred.
In a message posted on Twitter, civil protection officials said an “incident” was in progress in the General Pardiñas area.
“Please remain attentive to the indications of the emergency services and facilitate access to the area for them,” it said.
In January last year an explosion occurred in a building in Madrid which caused it to partially collapse.
It killed four people and wounded ten others.