What you’re talking about is Prompt-based image generation.
Like Diffusion. The use case for that is limited, though diffusion at least references a model (really an image…lame)..So there’s hope.
I think the better use case (for now) is for AI tools to generate assets and make adjustments WITHIN a rendering engine (Eg enscape, vray), or within SketchUp. This means I can quickly change the scene from summer to winter, or add rocks, or generative/procedural background environments… So in this case the control and repeatability is possible.
An AI helper using parameteric tools similar to Live Components would replace the 3d warehouse, for example. I can say " I want a 1982 Honda Civic in mustard yellow with a convertible roof" and it should be able to produce that accurately.
AI hasn’t been trained to use Software yet. Right now the LLMs and Image gens are procedurally creating content based on sampling. Once AI agents get exposed to more software tools, they’ll be able to create more stuff we can use.
For Arch Viz and other rendering, there are a few areas like adding traffic or moving people where AI can really shine, because those aspects take a LOT of time and processing power to do in the traditional manner. Same with backgrounds - we might model a great building, but we’re not being paid by our client to model half a city block around it.
Going forward, I’d like to see FAR more integration between real design data/parameters (traffic flows actually being realistic, people moving in real ways), aerial photos used to generate places, with AI agents (NPCs) simulating real activity within those places. EG a person sits down at a seat and watches a movie, or waits for traffic before crossing the road.. As a design tool this will allow us designers to create simulations of the real world.
The data behind this is pretty heavy though - a few companies (eg Arup) have created crowd simulation models, plus we have traffic models, weather models, etc… Those databases are all significantly complex and draw from data that’s held within locked commercial servers. AI might make “lite versions” of them accessible to typical commercial projects. The AI grunt needed to create models is also high and currently subsidised (OpenAI loses $100billion or so, a year).
Gaming is what’s moving this forward (I cant wait to see the new Grant Theft Auto VI game, not for the gameplay but the city environment that it creates, complete with roaming A.I. npcs)