It looks as if there may be quite a few funny things going on with your model. One of the first things to learn about SU is its inherent “stickiness”, as Aaron has demonstrated. Moving one thing can have an effect on other geometry. So an early lesson is to group things. In your model, you might make the external walls one group, the internal walls another, the floor a third, and so on. If you did that, you would find that altering an internal wall would have no effect on an external one.
Also, it would be normal to make things like windows and doors into components. Components are just like Groups except that if you change one, all instances change. It is not clear what the difference is between your window in blue and the others but I presume there is one. If the ones already in the walls are components, you can just select and delete. If they are all instances of the same component, swopping them with another component is even easier. But we’re getting quite sophisticated now!