How to Build 3D Worlds with Astra

The core architecture is straightforward. You force the model to anchor on real-world reference imagery (or images generated by something like GPT-Image-2) rather than its imagination. You have Astra dispatch dedicated sub-agents to sculpt those assets headlessly in Blender, assemble the scene in Three.js or Unreal Engine, and place an unyielding, blind critic between the render and the photo to kick flawed work back down the line. You keep the loop spinning until you love it.

Here is the exact kickoff prompt from the neighborhood build (note that I’ve improved my process since I wrote this one, more details below):

I want you to build a photorealistic, 1:1 recreation of [my childhood neighborhood] (REDACTED FROM THE PROMPT, OBVIOUSLY). It should be utterly perfect, visually indistinguishable from reality, with every single thing done at photoreal quality, from the street layout to the houses to the trees, lawns, mailboxes, road signage, lighting, and anything you could think of.

First, gather reference: pull Google Maps and Street View data, satellite imagery, real-estate listings for homes in the neighborhood (current and sold; their photos show the houses close-up from multiple angles, and their lot dimensions and square footage keep the scale accurate), and any other photos of the neighborhood online. The street layout, lot placement, and house positions must match the real neighborhood 1:1. Someone who grew up there should recognize every corner.

Fan out sub-agents and have sub-agents tackle each element individually so that the neighborhood is utterly perfect. Loop on each item and have a separate sub-agent check it visually against the real reference imagery to ensure it looks photoreal. That separate sub-agent should be a really harsh critic, and if it doesn't look like the real place, it should keep going.

Don't stop until each sub-agent is utterly wowed when compared with actual photos and Street View. It should literally compare renders and real imagery side by side blind and say which one looks real. Do this in Three.js, and use Blender against the real reference photos to make the assets and the world look photorealistic. Loop until it's utterly perfect.

The model pulled street grids from OpenStreetMap, building footprints from county records, elevation profiles from USGS lidar, and house angles from historical real estate listings. It then established an automated Blender bridge that exported runtime assets, staged daylight environments, and matched camera frustums directly against the real photos for review.

1. Anchor on a Reference

If you prompt for “a photorealistic suburban house,” you get a generic video-game asset with sterile textures and uncanny proportions. The target must be an uncompromising, specific photograph.

Before any geometry gets drafted, the model’s entire focus is collecting and cataloging ground truth. For the neighborhood, that meant parsing aerial views, street-level captures, and home listing tours. For Manhattan, the agent locked onto street photos of the Flatiron looking south from 23rd Street, estimated the camera focal lengths and sun elevation, and solved the world outward from that exact perspective. If you are starting fresh without personal files, make finding references the explicit primary task before any 3D generation begins.

If you want to build something entirely new (say, a sci-fi game world), you can use images generated by an image model (my recommendation is GPT-Image-2) as the reference.

2. Drive Blender Headlessly for Assets

Directing Astra to generate raw 3D code in Three.js or OpenGL yields pretty unimpressive assets. So just prompt it to use Blender! Something simple, like: “use Blender against the real reference photos to build and iterate each asset.”

Astra excels at orchestrating Blender via headless Python. It can construct difficult assets like procedural foliage with natural leaf clustering, bake ambient occlusion maps, produce believable surface imperfections etc. On the NYC build, I enforced a rule: every single mesh, texture, and rig had to be generated dynamically through Blender/code without downloading third-party assets. It takes longer, but the entire scene remains coherent, lightweight, and fully regenerable. Conversely, in Unreal, I lean heavily into Epic’s ecosystem (especially MetaHumans, because generating believable anatomy and facial hair from raw scripts/Blender remains insanely difficult).

3. Fan Out Across Specialized Sub-Agents

A single agent attempting to hold an entire build in working context will cut a lot of corners. You must split it up.

Instruct the agent to spin up dedicated workers (sub-agent) for distinct parts. In the neighborhood project, separate sub-agents independently owned the architectural shells, ground textures, asphalt aging, driveway gravel, porch railings, and vehicle geometry. Each worker iterates in isolation, answering only for the fidelity of its assigned domain before passing assets to the main branch.

4. Enforce Blind Side-by-Side Audits

The builder must never grade its own homework (see my Gauntlet Loop article for more details on why). Realism emerges only when a fresh-context evaluator is forced to choose between the work-in-progress render and the reference photograph.

Prompt snippet:

THE PHOTO TEST: real photos of actual locations vs our renders of the same spots, same angle, same light, same weather, shuffled. Fresh-context critics answer “which is the photograph?” A location ships only when critics can’t reliably pick.

5. Three.js or Unreal? Both, for different jobs

I bounce between both environments constantly, choosing purely based on distribution targets and asset requirements.

The decision comes down to the final format. When the goal is a friction-free link you can text a friend to drop them into a synchronized multiplayer map, choose Three.js and Blender. If you require believable cinematic crowd simulations, atmospheric volumetric lighting, and massive open-world scales, deploy Astra through Unreal’s Model Context Protocol (MCP).

6. The Production Loop

Put it in goal mode (/goal) right away, or it’ll just stop way before the task is done.

Make sure your prompt explicitly tells it to keep a live HTML page deployed so you can check it on your phone, or have it spin up a simple web page that dumps side-by-side screenshots every time it changes something. Trust me, this is insanely helpful.

These worlds get heavy fast. The fix is to open a second agent session purely for optimization while the first one keeps building. Let thread one make it pretty, and let thread two make it run fast. You can even spin up a third agent to grade the build every hour... if the world isn’t visibly better or the FPS is tanking, it needs to stop screwing around and fix the bottleneck.

The Starter Prompt

The consolidated starter prompt below incorporates these production constraints. Update the bracketed terms, select your engine, load it into Codex with ultra reasoning enabled, and let the loop take over:

/goal Build a photorealistic 1:1 recreation of [PLACE] in [Three.js with Blender for assets / Unreal Engine with Free Assets, and MetaHumans]. It should be indistinguishable from reality.

First, gather reference: (ex: map data for the street layout and footprints, satellite and aerial imagery, Street View, listing photos, and these photos I'm attaching).

Fan out sub-agents and have each one own a single element (buildings, vegetation, ground surfaces, vehicles, lighting (adjust these to your elements!)). Make assets in Blender. Loop on each element with a separate fresh-context critic that compares our render to the real photo of the same spot, blind, and says which is the photograph. Prepare packets so blind means blind: neutral filenames, matched resolution, stripped metadata, matched compression. Freeze the critic prompt now. Every tell the critic finds is a work order.

Keep a build that always runs and publish a link I can open on my phone on here.now. Every hour, judge this version against last hour's and adjust if you're not making progress. Don't stop until the critics can't tell which is real.

– Matt

Questions people ask about making 3D games with Astra

How do you make a 3D game with GPT-6 Astra?

Give Astra a real reference (photos, map data, or generated images), tell it to build the assets in Blender, fan out sub-agents so each owns one element, assemble in Three.js or Unreal Engine, and put a separate blind critic between every render and the reference photo. Run it in goal mode and keep looping until the critic can't tell which image is real. The full starter prompt is at the end of this guide.

Should I use Three.js or Unreal Engine with Astra?

Three.js plus Blender when the deliverable is a link anyone can open in a browser: a block, a house, a multiplayer mini-game. Unreal Engine when you need photoreal humans, city-scale assets, weather, and cinematic capture, since Astra can lean on City Sample, Megascans, and MetaHumans through Unreal MCP.

Why use Blender instead of having Astra write 3D code directly?

Astra generating raw geometry in Three.js code produces flat, generic assets. Driving Blender headlessly through Python gives it real tools for foliage, materials, baking, and surface imperfections, so assets can be iterated against the reference photo until they read as real.

Can Astra build a photorealistic world from one prompt?

Yes, if the prompt sets a reference bar, asks for Blender assets, fans out sub-agents, and adds a blind critic. The Manhattan build in Unreal Engine and the neighborhood build in Three.js both started from a single kickoff prompt in goal mode, with only occasional steering afterward.

What does the blind critic actually do?

A fresh-context agent sees the render and the real photo of the same spot, shuffled and stripped of metadata, and answers one question: which is the photograph? Every tell it lists becomes a work order for the builder, and a location only ships when the critic can no longer pick reliably.