Your model can be pretty much as big as you want it. I wouldn’t worry about it for your game. However, the view may occasionally get clipped, but that can be fixed by re-scaling the view. The underlying geometry is not a problem.
Here’s a scale model of Hell’s Kitchen (about 2.5 mi. x 3 mi.):