When I am making walls, I can push the outer walls just fine, but when I do a wall in the middle of the floor, there ends up being a hole underneath. How can I fix this?
Give the floor some thickness before you outline the walls. As it is, your floor has no thickness at all and you are seeing what Push/Pull is supposed to do.
Alternatively, you could erase the face you are calling the floor before pulling the walls up. Or you could hit Ctrl before pulling the walls up. That is likely to cause you problems later if you leave the floor with no thickness so not the best solution.
Deleting the face works best for me. Ill just draw it back in later