Grand Treasure Island

Grand
Treasure
Island

The downloadable version looks a lot nicer: shadows, better lighting, sunsets.

Every street and building on Treasure Island, modeled by AI.

About the game

Location

Treasure Island, San Francisco

Map data © OpenStreetMap contributors · OpenMapTiles

What's in the game?
0.63
square miles
16
miles of road
199
buildings
540
street signs and furniture
35
kinds of plants
458
cars

All playable.

All modeled with AI.

  • Modeled by Claude Opus 5.5
  • Some buildings modeled with GPT-6 Astra
  • Textures generated with GPT Image 2
  • Sound effects generated with ElevenLabs Sound Effects v3
  • Built from public and private photographic sources
  • Map data from OpenStreetMap, terrain from USGS geodata

Real and modeled

Photo: Maceo May ApartmentsReal
In the game: Maceo May ApartmentsModeled
Maceo May Apartments
Photo: The new park by Isle HouseReal
In the game: The new park by Isle HouseModeled
The new park by Isle House
Photo: The west shore, looking at the Bay BridgeReal
In the game: The west shore, looking at the Bay BridgeModeled
The west shore, looking at the Bay Bridge
Photo: The marsh, looking toward the cityReal
In the game: The marsh, looking toward the cityModeled
The marsh, looking toward the city

Character

Step into the shoes of Chester Schatz. You spent five years in the hole for a crime you didn't commit, because you had dirt on the wrong people. Now you're back where it all started: Treasure Island.

Old black-and-white mugshot of Chester, a treasure chest with legs, holding a sign: Chester Schatz, inmate 94130

Activities

Explore all of Treasure Island, San Francisco, and live a life of leisure, exploration and light trespassing.

How we built this

How we built this

The goal was to model an entire island using AI at the level of a modern video game. We're not there yet, but we're a whole lot closer than when we started.

My role as an engineer in this was to raise the bar. I playtested a lot.

  • 3 months
  • a lot of tokens
  • 16 miles of road
  • almost 200 buildings
  1. 01

    Start from data, not from prompts

    The terrain comes from USGS lidar and the road layout from OpenStreetMap. That gave us every road, path and fence, and the footprint of every building. Code bakes all of it into the island, and agents fix the parts that don't look good.

    USGS lidar terrain, OpenStreetMap roads and building footprints
  2. 02

    An agent team

    I talk to one agent, the foreman. The foreman doesn't build anything itself. It hands each job to a team of agents and reviews what comes back. The whole team is in the repo.

  3. 03

    Agents check agents

    Agents are still very willing to pass mediocre work as “fits the spec.” So every change gets looked at in the actual game by agents that didn't make it. One of them is the bar raiser: it compares the game with photos of the real place and sends work back until it gets closer. The dev box runs up to four copies of the game at once, so checking keeps up with building.

    The shed in the game with two roll-up doors on its long wall

    Sent back

    “Two kept doors on photographed faces contradict the dated photos.”

    The same shed with the doors fixed to match the photos

    Passed

    “All three must-fixes are done, and nothing from the earlier round has come back.”

    The shed at 899 13th St, same camera, two rounds of the same review on 2026-11-15
  4. 04

    Breadth and depth

    We had to model every building, and some buildings deserve a lot more. So each building has a level of fidelity: first the shape, then doors, windows and roofs, then details, then materials. The agents finish the whole island at a low level first, and then we keep pushing the buildings that matter most.

    L1ShapeL2Doors, windows, roofsL3DetailsL4MaterialsL5FinishWhole island at L1 first, then deeper where it matters
  5. 05

    Systems, not hand placement

    We didn't let agents hand-place every tree. Procedural systems spread plants across beds, lay out mulch and scatter vegetation. Same for buildings: a family is a group of buildings that share one design, like the townhouses with garages on Sturgeon Street. We designed each family once, and every building in it gets its own variation: colors, shutters, roofs. Windows and doors go on a grid that a system fills in, instead of an agent placing them by eye.

    Four houses on Sturgeon Street, all built from the same housing family with different colors, shutters and roofs
    One housing design, four instances on Sturgeon Street
    490 Avenue of the Palms with its window grid drawn over the facade
    490 Avenue of the Palms: the window grid, measured from photos, drawn over the facade
    Housing family: 113 · Midrise family: 61 · One-offs: 25
  6. 06

    A mix of methods

    Photos never make it into the game. They're only a source. The inputs are measured open data and photos, public and my own. Then it's a mix of methods: code turns the data into the island, agents model things like the pelicans, and procedural systems vary the trees, the flower beds and the buildings. The output is the island.

    INPUTSOpen dataOSM, USGS lidarPhotospublic and minePhotos never shipMETHODSCodebakes the layoutAgent artworkthe pelicansProceduraltrees, beds, buildingsOUTPUTThe island
    Brown pelicans and gulls on the breakwater
    Agent artwork: pelicans on the breakwater
    A double row of young blue gums in mulch beds in Cityside Park, each a little different
    Procedural: blue gums, each one varied
    The whole island in the game from high above the north-east
    The output: the island
  7. 07

    Better input, better output

    Buildings modeled only from public photos came out mediocre. Once I added my own video footage, they got a lot better.

    Maceo May's city side in the game in September: a blank grey wall

    Before: public imagery only

    My photo of Maceo May's city side: dark cladding, cream frames, glazed ground floor

    My photo

    Maceo May's city side in the game in November, matching the photo

    After: built with my photos

    Maceo May, city side
  8. 08

    Chester

    Claude animated Chester, the main character, with a rubber-hose animation system. That worked shockingly well. It was the biggest surprise of the project.

    Chester's rubber-hose arms on the jetpack
  9. 09

    Why Godot

    We used an established game engine instead of hand-rolling a web renderer. Godot gave us physics for the player and the cars, a modern renderer, a game camera, and builds for Mac, Windows, Linux and the browser from one project. All of that would have been pretty hard to build ourselves.

    GodotPhysicsCars on rigid bodiesForward+ rendererGame cameraAudioMac · Windows · Linux · Web
  10. 10

    How far can this go?

    I still don't know. Right now we're at a GTA 3, GTA 4 level. But every big jump so far came from better input and a higher bar, not a better model: Maceo May went from a grey box to a building you recognize once it had my photos. What runs out is time, tokens and photos. When my agents say “the models aren't there yet”, they never show it. So I haven't seen anything that says we can't push further.

    GTA 3GTA 4We're here?No sign yet of where the limit is

What's next

There are several ways to take this.

Chester scores a goal on the island's soccer pitch, confetti in the air

Build out the game

Add missions and make it more fun. Right now there isn't a whole lot to do.

Aerial photo of Treasure Island as a Navy base in 1944, the exposition halls still standing
U.S. Navy photo, 30 March 1944. Public domain.

Tell the island's history

Treasure Island was built for the 1939 Golden Gate International Exposition, then spent decades as a Navy base. From public domain photos, we could model the island as it looked back then, and let you walk through it as it changed. 3D worlds are a great way to tell history.

Chester jetpacks over the Bay toward the San Francisco skyline

Model the whole Bay Area

The Bay Area is a pretty wild place, and a great setting for a Grand Theft-style game. One of the dreams is a racing game set in all of San Francisco. That sounds huge, but with the right models and the right harness, it's possible.

An early WorldKit test world: grey buildings, a road and a red car

Turn the tools into a toolkit

Everything we built to model this island can model other places. That's WorldKit. See the early preview →

See you on Treasure Island

Questions or feedback? Post to @phppco on X · or open an issue on GitHub