HDRI images used to light the scene (especially when downloaded from a free source) can often be way over or even underexposed, causing unnecessary sky issues for the final render - and the need to replace with a static image.
With 32bit colour photo editors such as Adobe PhotoShop, you can edit the exposure (using the exposure tool) to end up with useable HDRI sky images in the final render which also maintains lighting consistency between sky and building.
This example is from Noemotion - available for free download under a creative commons license. I have deliberately overexposed the first example and the second is a correct exposure without blown highlights or black shadows.
The second image does look dingy, but is fine as an HDRI - you can increase the light intensity significantly in render software - they are not meant to look like a normal 8 or 16bit colour image.
On the topic of window reflections, another option is to fine-tune this in your renderer and saving as a global setting (if possible). I prefer a touch of interior visibility myself with a strong reflection of the sky.
My trees are a little below your standard -