Your model is on the edge of the tiny face problem, too many segments in your curves for it to perform well at that size. Perfect candidate for The Dave Method.
One of the benefits of this is you could leave all your tags turned on and simply copy the component outside the building to work on it in open space.
As you see the first try with Upright causes some holes and a mixture of faces, make the path and profile a component, make a copy, scale the copy up, (in this case I only needed to scale up by 10 but sometimes you need to go 100 or even 1000 to get a good result), edit the large one and then you can delete it as the edits are retained in the smaller one.
