Take this picture for reference:
Usually, I use the outer_loop of a face to get the geometrie of a “wall” (blue and white faces combined). In this case, there is a recess in the loop due the other face splitting the edge on the side. The BoundingBox I get after combining the faces on that “wall” to a group does only work in very specific situations. Any pointers?