Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license
This commit is contained in:
@@ -0,0 +1,256 @@
|
||||
# Advanced workflows (live-verified on UE 5.8)
|
||||
|
||||
Everything here was executed against a running 5.8 editor unless explicitly
|
||||
marked schema-verified. Where behavior diverged from Epic's docs, this file
|
||||
records what the server actually did.
|
||||
|
||||
## ProgrammaticToolset — batching without breaking the serial rule
|
||||
|
||||
`editor_toolset.toolsets.programmatic.ProgrammaticToolset` is the sanctioned
|
||||
way to do N operations in one MCP round-trip. It is ONE tool call on the
|
||||
game thread, so the serial-call rule stays intact; you're just letting a
|
||||
script make the sub-calls server-side.
|
||||
|
||||
Contract (verified):
|
||||
|
||||
1. Call `get_execution_environment` ONCE per session before the first
|
||||
script. It returns `instructions` (read them — they are authoritative),
|
||||
`supported_modules`, and `language`.
|
||||
2. `execute_tool_script` takes `{"script": "<python>"}`. The script must
|
||||
define `run() -> Dict[str, Any]`.
|
||||
3. Inside the script, `execute_tool(tool_name, json_input)` calls any
|
||||
registered tool. `tool_name` is FULLY QUALIFIED INCLUDING the tool
|
||||
segment (`"editor_toolset.toolsets.primitive.PrimitiveTools.add_cube"`)
|
||||
— unlike top-level `call_tool`, there is no separate toolset/tool split.
|
||||
`json_input` is a JSON **string** (use `json.dumps`).
|
||||
4. `execute_tool` returns a dict-like object; unwrap results with
|
||||
`["returnValue"]`. It raises `RuntimeError` on failure — no manual error
|
||||
checking.
|
||||
5. Allowed imports (5.8): `json`, `math`, `datetime`, `copy`, `re`, `time`.
|
||||
Nothing else — no `unreal`, no `os`, no file I/O.
|
||||
6. The whole script's return value comes back as a JSON string in
|
||||
`returnValue`.
|
||||
|
||||
Worked example (verified — 12-column colonnade, 36 components, ONE
|
||||
round-trip that would otherwise be 37 serial calls):
|
||||
|
||||
```python
|
||||
import json, math
|
||||
|
||||
def add_cylinder(actor_ref, name, radius, height, x, y, z):
|
||||
return execute_tool(
|
||||
"editor_toolset.toolsets.primitive.PrimitiveTools.add_cylinder",
|
||||
json.dumps({"actor": actor_ref, "name": name, "radius": radius,
|
||||
"height": height,
|
||||
"local_transform": {"location": {"x": x, "y": y, "z": z}}}))
|
||||
|
||||
def run():
|
||||
spawn = execute_tool(
|
||||
"editor_toolset.toolsets.scene.SceneTools.add_to_scene_from_class",
|
||||
json.dumps({"actor_type": {"refPath": "/Script/Engine.Actor"},
|
||||
"name": "Colonnade",
|
||||
"xform": {"location": {"x": 75800, "y": 84900, "z": 44300}}}))
|
||||
host = spawn["returnValue"]
|
||||
n, ring_r = 12, 900.0
|
||||
for i in range(n):
|
||||
a = 2.0 * math.pi * i / n
|
||||
add_cylinder(host, "Shaft_%02d" % i, 40, 360,
|
||||
ring_r * math.cos(a), ring_r * math.sin(a), 210)
|
||||
return {"colonnade": host["refPath"], "columns": n}
|
||||
```
|
||||
|
||||
When to reach for it: any loop over 5+ homogeneous operations (placement
|
||||
rings, grid scatter, bulk renames, bulk property sweeps). When NOT to:
|
||||
operations where you need to see intermediate results to decide the next
|
||||
step — the script can't ask you questions mid-run.
|
||||
|
||||
Failure surface: `print()` goes to the UE log, not the MCP return — return
|
||||
diagnostics in the result dict instead. A script exception returns the
|
||||
traceback as the tool error text.
|
||||
|
||||
## Blueprint authoring — the DSL loop
|
||||
|
||||
`editor_toolset.toolsets.blueprint.BlueprintTools` (53 tools) authors real
|
||||
Blueprints. The graph surface is an s-expression DSL, and the workflow that
|
||||
survives contact with the live server is:
|
||||
|
||||
1. **`create`** — `{"folder_path": "/Game/Blueprints", "asset_name":
|
||||
"BP_Spinner", "asset_type": {"refPath": "/Script/Engine.Actor"}}` →
|
||||
returns the Blueprint's refPath (`/Game/Blueprints/BP_Spinner.BP_Spinner`).
|
||||
2. **`list_graphs`** — returns graph refPaths in colon form:
|
||||
`...BP_Spinner.BP_Spinner:EventGraph`,
|
||||
`...BP_Spinner.BP_Spinner:UserConstructionScript`.
|
||||
3. **`get_graph_dsl_docs`** — pulls ~9k chars of grammar documentation off
|
||||
the live server. Read it before writing DSL; it covers `event`/`fn`,
|
||||
`bind`, `if`/`for`/`while`/`switch`, multi-exec continuation blocks
|
||||
(`(:then ...)`, `(:CastFailed ...)`), auto-generated underscore
|
||||
variables for data output pins, and quoted pin names.
|
||||
4. **Resolve every node ID with `find_node_types` BEFORE writing DSL** —
|
||||
`{"graph": {"refPath": "<graph>"}, "type_id_filter": "MakeRotator",
|
||||
"context_pins": []}` → exact IDs. Node IDs are pipe-delimited category
|
||||
paths and must match the live registry exactly. Verified gotchas:
|
||||
- Engine events use K2 display names: `EventTick` (with `DeltaSeconds`
|
||||
param), `EventBeginPlay` — `(event Tick ...)` fails with
|
||||
"AddEvent|Tick does not exist".
|
||||
- `Math|Rotator|MakeRotator`, not bare `MakeRotator`.
|
||||
- `Utilities|Operators|Multiply` (wildcard operator), not
|
||||
`Multiply_FloatFloat`.
|
||||
- `Transformation|AddActorLocalRotation`, not
|
||||
`Utilities|Transformation|AddActorLocalRotation` — category prefixes
|
||||
in doc examples don't always match the live registry. The registry
|
||||
wins.
|
||||
- There is no `(self)` node; the target is implicit — omit `:self`
|
||||
entirely for calls on the owning actor.
|
||||
5. **`write_graph_dsl`** — `{"graph": {"refPath": "<EventGraph>"}, "code":
|
||||
"<dsl>"}`. Returns `null` on success. On failure the error is an
|
||||
AssertionError naming the exact failing node and its enclosing form —
|
||||
fix ONE node at a time and rerun; the error moves to the next problem.
|
||||
6. **`compile_blueprint`** — `{"blueprint": {"refPath": ...},
|
||||
"warnings_as_errors": false}`. Returns `null` on success.
|
||||
7. **Spawn an instance** — `SceneTools.add_to_scene_from_asset` with
|
||||
`{"asset_path": "/Game/Blueprints/BP_Spinner.BP_Spinner", ...}`.
|
||||
NOTE: this tool takes `asset_path` as a plain STRING, not an `asset`
|
||||
refPath object — the error schema is the tiebreaker (see below). The
|
||||
spawned actor's class is `BP_Spinner_C` (the `_C` generated-class
|
||||
suffix, visible in the returned refPath).
|
||||
|
||||
Verified end-to-end: created `BP_Spinner`, wrote a Tick handler that yaws
|
||||
the actor 90°/s (`(event EventTick (DeltaSeconds) (Transformation|AddActorLocalRotation
|
||||
:DeltaRotation (Math|Rotator|MakeRotator :Roll 0.0 :Pitch 0.0 :Yaw
|
||||
(Utilities|Operators|Multiply DeltaSeconds 90.0))))`), compiled clean,
|
||||
spawned it, and attached a visible mesh via `PrimitiveTools.add_cube` on
|
||||
the instance.
|
||||
|
||||
Variables, functions, dispatchers: `add_variable` (`type_name` strings),
|
||||
`add_object_variable`/`add_struct_variable`, `add_function_graph` +
|
||||
`add_function_param`, `add_event_dispatcher`, `set_variable_replication` —
|
||||
same refPath discipline. `read_graph_dsl` round-trips existing graphs back
|
||||
to DSL for inspection/editing.
|
||||
|
||||
## Schema-in-error is a first-class discovery mechanism
|
||||
|
||||
When a call is missing/mistyping a required param, the server returns the
|
||||
COMPLETE input schema of the tool in the error text. This is faster than
|
||||
re-running `describe_toolset` and is authoritative for the exact function
|
||||
you called. Two verified cases where it corrected the surface:
|
||||
|
||||
- `add_to_scene_from_asset` — advertised conceptually as taking an asset
|
||||
reference; live schema requires `asset_path` (string).
|
||||
- `StartPIE` — `{}` fails, and the error hands you the full
|
||||
`PIESessionOptions` schema.
|
||||
|
||||
Rule: on a param error, READ the schema in the error before anything else.
|
||||
|
||||
## PIE sessions (schema-verified)
|
||||
|
||||
`EditorAppToolset.StartPIE` requires an `options` object
|
||||
(`FPIESessionOptions`):
|
||||
|
||||
- `bSimulate` (required): `true` = Simulate-In-Editor — world ticks,
|
||||
physics/AI run, no player pawn possessed. `false` = standard PIE with
|
||||
possession.
|
||||
- `playMode` (required): `PlayMode_InViewPort`, `PlayMode_InEditorFloating`,
|
||||
`PlayMode_Simulate`, etc. Out-of-process modes (NewProcess, MobilePreview,
|
||||
VR, QuickLaunch) are silently downgraded to in-viewport — the tool needs
|
||||
in-process PIE for delegate-based completion tracking.
|
||||
- `warmupSeconds` (required): extra settle time after the engine fires
|
||||
PostPIEStarted (BeginPlay has run) before the call returns. `0` = return
|
||||
as soon as PIE is up.
|
||||
- `startTransform` (optional): spawn the pawn/reference at a specific
|
||||
transform instead of PlayerStart.
|
||||
|
||||
`IsPIERunning` returns a bare boolean. Runtime-state inspection during PIE
|
||||
(actor transforms ticking, LogsToolset reads) plus `StopPIE` complete the
|
||||
loop: start simulate → read state / logs → stop → judge.
|
||||
|
||||
The test loop this enables: compile Blueprint → StartPIE (simulate) →
|
||||
sample an actor transform twice a few seconds apart → confirm your Tick
|
||||
logic actually runs → StopPIE. Remember pitfall 15: PIE mutates world
|
||||
state; take editor-world measurements before or after, not across, a PIE
|
||||
session.
|
||||
|
||||
## Sequencer — orientation for a 140-tool surface
|
||||
|
||||
`animation_toolset.toolsets.sequencer.SequencerTools` is the largest
|
||||
toolset (140 tools) and follows an open-sequence-implicit-target model:
|
||||
`create_level_sequence` / `open_sequence` / `get_focused_sequence`, then
|
||||
most calls operate on the focused sequence.
|
||||
|
||||
Capability map (names verified via describe; group by prefix):
|
||||
|
||||
- **Structure**: `add_actors` (possessables), `add_spawnable_from_class` /
|
||||
`add_spawnable_from_instance`, `create_camera` (returns a camera-cut
|
||||
ready binding), bindings CRUD (`get_bindings`, `find_binding_by_name`,
|
||||
`remove_binding`, `rebind_component`, `fix_actor_references`).
|
||||
- **Tracks/sections**: `add_track_to_binding` / `add_track_to_sequence`,
|
||||
`add_section`, `set_section_range`/`set_section_blend_type`/ease in-out,
|
||||
`set_camera_cut_binding`.
|
||||
- **Timing**: `set_playback_range`, `set_display_rate`,
|
||||
`set_tick_resolution`, `set_work_range`, marked frames.
|
||||
- **Transport**: `play`, `pause`, `play_to`, `set_playhead_frame`,
|
||||
`force_evaluate`, `set_playback_speed`.
|
||||
- **Keyframing** lives in the sibling
|
||||
`animation_toolset.toolsets.keyframing.SequencerKeyframingTools` (22
|
||||
tools): `get_channel_names` → `add_key_float`/`add_key_bool`/... →
|
||||
`get_keys`, `set_default_value`, `bake_channel_keys`, curve-editor
|
||||
control.
|
||||
- **Baking/IO**: `bake_transform`, `import_export` sibling toolset (FBX
|
||||
etc.), `copy_tracks`/`paste_tracks`.
|
||||
- **Runtime conditions / custom bindings / ControlRig**: dedicated sibling
|
||||
toolsets (`SequencerConditionTools`, `SequencerCustomBindingTools`,
|
||||
`SequencerControlRigTools`, `ControlRigTools`).
|
||||
|
||||
Minimal cinematic recipe skeleton: `create_level_sequence` →
|
||||
`create_camera` → `add_actors` for subjects → keyframe camera transform
|
||||
channels at frame A and B → `set_playback_range` → `play` → capture/judge.
|
||||
|
||||
## Editor self-debugging with LogsToolset
|
||||
|
||||
`EditorToolset.LogsToolset`: `GetLogCategories`, `Get/SetVerbosity`,
|
||||
`GetLogEntries`. After any failed operation or suspicious silence, pull
|
||||
recent log entries filtered to the relevant category (`LogBlueprint`,
|
||||
`LogNiagara`, `LogModelContextProtocol`, ...) instead of guessing. This is
|
||||
also how you see `print()` output from ProgrammaticToolset scripts and
|
||||
Python toolset internals.
|
||||
|
||||
## Automation testing
|
||||
|
||||
`AutomationTestToolset.AutomationTestToolset`: `DiscoverTests` /
|
||||
`ListTests` → `RunTests` or `RunTestsByFilter` → `GetTestStatus` (poll —
|
||||
test runs are async on the editor) → `GetTestResults` → `StopTests` if
|
||||
needed. This is the CI-shaped loop for "make a change, prove nothing
|
||||
broke" inside a live editor session.
|
||||
|
||||
## Asset intelligence
|
||||
|
||||
- `SemanticSearchToolset`: `Search` (hybrid vector + BM25 over project
|
||||
assets) and `FindSimilar` — use for "find me a rusty metal material"
|
||||
style requests before falling back to `AssetTools.find_assets` name
|
||||
matching.
|
||||
- `StaticMeshTools`: `import_file` (bring in external meshes),
|
||||
`set_nanite_enabled`, `generate_lods`/`set_lod_thresholds`,
|
||||
`generate_convex_collisions`, `get_triangle_count`/`get_bounds` — the
|
||||
optimization pass after any import.
|
||||
- `ConfigSettingsToolset`: `ListContainers`/`ListCategories`/`ListSections`
|
||||
→ `GetSectionSchema` → `SetSectionProperties` (saves to config). The
|
||||
remote path to Project Settings and Editor Preferences — rendering
|
||||
defaults, exposure defaults, auto-start flags — without touching ini
|
||||
files by hand.
|
||||
- `ToolsetRegistry.AgentSkillToolset`: `ListSkills`/`GetSkills`/
|
||||
`CreateSkill`/`UpdateSkill` — project-embedded agent skills that ship
|
||||
with the .uproject. If a project has them, list them FIRST; they encode
|
||||
project-specific conventions that outrank this file's generic guidance.
|
||||
|
||||
## Choosing a strategy (decision table)
|
||||
|
||||
| Situation | Reach for |
|
||||
|---|---|
|
||||
| 5+ homogeneous ops (scatter, bulk edit) | ProgrammaticToolset script |
|
||||
| Gameplay behavior, event logic | BlueprintTools DSL loop |
|
||||
| Camera moves / animation over time | SequencerTools + KeyframingTools |
|
||||
| "Does it actually behave at runtime?" | StartPIE (simulate) + transform/log sampling |
|
||||
| "Find an asset like X" | SemanticSearch, then AssetTools |
|
||||
| Imported mesh is heavy | StaticMeshTools nanite/LOD/collision pass |
|
||||
| Change editor/project settings | ConfigSettingsToolset |
|
||||
| Anything failed silently | LogsToolset GetLogEntries |
|
||||
| Project has its own agent skills | AgentSkillToolset first |
|
||||
@@ -0,0 +1,350 @@
|
||||
# Unreal MCP — Pitfalls & Lessons
|
||||
|
||||
Read before your first session; return whenever something misbehaves. Ordered
|
||||
by when they bite: setup → calling discipline → editor state → content →
|
||||
delivery.
|
||||
|
||||
## Setup & Connection
|
||||
|
||||
### 1. Start order: editor first, Hermes second
|
||||
|
||||
Hermes probes MCP servers at session start. If the editor (and its server)
|
||||
isn't up yet, no `mcp_unreal_engine_*` tools exist in the session. Fix:
|
||||
launch the editor, confirm the server bound (Output Log shows
|
||||
`LogModelContextProtocol` with the address), then open a NEW Hermes session.
|
||||
Tools don't hot-appear mid-session.
|
||||
|
||||
### 2. Server enabled but no tools advertised
|
||||
|
||||
The Unreal MCP plugin ships the SERVER, not the tools. If `list_toolsets`
|
||||
returns nothing/near-nothing, the toolset provider plugin (AllToolsets) or
|
||||
Toolset Registry isn't enabled in this project. Fix in Edit > Plugins,
|
||||
restart the editor, restart the Hermes session.
|
||||
|
||||
### 3. macOS: full Xcode is required, not just Command Line Tools
|
||||
|
||||
On a Mac, the editor needs Xcode to compile shaders for Metal. Without it,
|
||||
first launch dies with a modal "Xcode Not Found" dialog and the editor
|
||||
exits as soon as it's dismissed (verified UE 5.8 behavior — the log shows
|
||||
`RequestExit` right after the dialog). Fix: install full Xcode from the App
|
||||
Store, open it once to accept the license / install components, and if it
|
||||
lives anywhere other than `/Applications/Xcode.app`, point the toolchain at
|
||||
it: `sudo xcode-select -s /path/to/Xcode.app`. Verify with
|
||||
`xcode-select -p` (should print an Xcode path, not the bare CLT path).
|
||||
Expect the first successful editor launch after that to spend a long time
|
||||
compiling shaders.
|
||||
|
||||
### 4. Port 8000 conflicts
|
||||
|
||||
Common collisions: local dev servers, Jupyter, other MCP hosts. Symptom: the
|
||||
server fails to bind (Output Log) or Hermes' probe times out. Fix: change
|
||||
Server Port Number in Editor Preferences > Model Context Protocol AND the
|
||||
`url` in `~/.hermes/config.yaml` (`mcp_servers.unreal-engine`), then restart
|
||||
both sides. Verify: `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8000/mcp`
|
||||
(non-000 means something is listening; whether it's Unreal is a different
|
||||
question — check the Output Log).
|
||||
|
||||
### 4b. "Connection refused" mid-session
|
||||
|
||||
The editor was closed, crashed, or the server was stopped
|
||||
(`ModelContextProtocol.StopServer`). Don't retry the tool in a loop — tell
|
||||
the user, have them relaunch/restart the server, then reconnect (new session
|
||||
if tools were lost).
|
||||
|
||||
### 5. GenerateClientConfig is not for Hermes
|
||||
|
||||
`ModelContextProtocol.GenerateClientConfig` writes config files for Claude
|
||||
Code/Cursor/VSCode/Gemini/Codex into the project root. Hermes' connection
|
||||
lives in `~/.hermes/config.yaml` via `hermes mcp install unreal-engine`.
|
||||
Running GenerateClientConfig neither helps nor harms Hermes — just don't
|
||||
mistake it for the Hermes setup step.
|
||||
|
||||
## Calling Discipline
|
||||
|
||||
### 6. One call at a time — never batch MCP calls
|
||||
|
||||
The server executes tool calls serially on the game thread and Epic
|
||||
explicitly warns against overlapping calls. Hermes executes same-turn tool
|
||||
calls concurrently — so batching two `mcp_unreal_engine_*` calls in one turn
|
||||
IS issuing overlapping calls. Strictly sequential: call, await, then next.
|
||||
This deliberately overrides the general "batch independent calls" guidance.
|
||||
|
||||
### 7. The editor freezes during every call — keep calls small
|
||||
|
||||
Game-thread execution means the editor UI hitches for the duration of each
|
||||
tool call. A 30-second operation is 30 seconds of frozen editor. Split big
|
||||
asks (e.g. "spawn 200 trees") into chunks so the user's editor stays
|
||||
responsive and any failure loses only one chunk.
|
||||
|
||||
### 8. Modal dialogs deadlock the loop
|
||||
|
||||
Anything that pops a modal (some deletes, import options, save prompts,
|
||||
experimental-plugin warnings) blocks the game thread — and your call —
|
||||
until a human clicks. If a call hangs far beyond its normal duration, tell
|
||||
the user to look at the editor for a dialog. Prefer tool paths/parameters
|
||||
that avoid interactive prompts; save proactively so "unsaved changes"
|
||||
prompts don't appear at bad times.
|
||||
|
||||
### 9. Timeouts: Hermes gives up before Unreal does
|
||||
|
||||
Hermes' default per-call timeout is 120 s. Asset imports, first-shader
|
||||
compiles, big saves, and renders can exceed it — the call "fails" while the
|
||||
editor happily finishes the work. Symptoms: timeout error, then the next
|
||||
scene query shows the operation actually completed. Fixes: raise
|
||||
`mcp_servers.unreal-engine.timeout` in config for heavy sessions; after any
|
||||
timeout, RE-QUERY state before retrying, or you'll do the work twice
|
||||
(duplicate actors are the classic case).
|
||||
|
||||
### 10. Stale schemas after editor-side changes
|
||||
|
||||
Toolsets are cached: after enabling a plugin, authoring a toolset, or Live
|
||||
Coding, run `ModelContextProtocol.RefreshTools` in the editor console, then
|
||||
re-run `list_toolsets`/`describe_toolset`. New C++ `UFUNCTION`s need a full
|
||||
editor restart regardless. If a call fails with "unknown tool" that
|
||||
`describe_toolset` just showed, refresh + reconnect.
|
||||
|
||||
### 10b. Ref-object vs plain-string params are inconsistent — the error schema is the tiebreaker
|
||||
|
||||
Most tools take object references as `{"refPath": ...}` objects, but some
|
||||
take plain string paths — live-verified: `add_to_scene_from_class` takes
|
||||
`actor_type` as a refPath OBJECT, while `add_to_scene_from_asset` takes
|
||||
`asset_path` as a STRING. Don't pattern-match across tools. When a call
|
||||
fails on params, the error text contains the complete input schema for
|
||||
that exact function — read it and fix; it's faster and more authoritative
|
||||
than re-describing the toolset.
|
||||
|
||||
### 10c. Blueprint DSL node IDs must come from find_node_types
|
||||
|
||||
DSL docs examples and intuition both produce wrong node IDs
|
||||
(live-verified: `(event Tick)` fails — it's `EventTick`; `MakeRotator`
|
||||
fails — it's `Math|Rotator|MakeRotator`; doc-style
|
||||
`Utilities|Transformation|AddActorLocalRotation` fails — the registry says
|
||||
`Transformation|AddActorLocalRotation`; there is no `(self)` node — omit
|
||||
`:self` for the owning actor). Resolve EVERY node ID with
|
||||
`find_node_types` against the target graph before writing DSL. Errors are
|
||||
progressive (one failing node at a time, named exactly) — fix and rerun.
|
||||
|
||||
### 11. Experimental means drift
|
||||
|
||||
Tool names, parameters, and result shapes may change across engine versions.
|
||||
The live schema from `describe_toolset` is the only contract. If this
|
||||
skill's examples and the live schema disagree, the schema wins — and patch
|
||||
this skill afterward.
|
||||
|
||||
## Editor & Scene State
|
||||
|
||||
### 12. Never assume a fresh level — and NEVER double-spawn environment actors
|
||||
|
||||
Query the scene before the first edit. Template levels (Open World, etc.)
|
||||
ALREADY contain a DirectionalLight, SkyAtmosphere, SkyLight,
|
||||
ExponentialHeightFog, and often VolumetricCloud. Spawning your own creates
|
||||
duplicates that compound (double fog = whiteout, double sky = wrong
|
||||
exposure) and are invisible in a screenshot until things look
|
||||
inexplicably wrong. Live-verified failure: spawning a "golden hour kit"
|
||||
into the default Open World template produced two of everything and a
|
||||
254/255-luminance whiteout. Rule: `find_actors` for each environment class
|
||||
FIRST; configure what exists; spawn only what's missing.
|
||||
|
||||
### 12b. Read the existing sun before imposing physical light values
|
||||
|
||||
Scene-craft tables give physical values (golden hour ≈ 10k lux), but a
|
||||
template's world is calibrated as a SYSTEM — the default Open World sun is
|
||||
`intensity: 10` (lux-ish units under default exposure), not 100,000. Setting
|
||||
12,000 lux into that world blows the frame to pure white regardless of
|
||||
small exposure tweaks. Live-verified rule: `get_properties` the existing
|
||||
sun's intensity first. If the scene is calibrated low (single-digit sun),
|
||||
work RELATIVE to it (e.g. golden hour ≈ 0.5–1× the template's noon value,
|
||||
warm temperature, low pitch) and let auto-exposure adapt, or rebuild the
|
||||
whole exposure chain deliberately (manual EV100 + physical values
|
||||
everywhere). Mixing the two conventions is the #1 whiteout cause.
|
||||
`ObjectTools.reset_properties` is the escape hatch — it restores per-project
|
||||
defaults when you've painted into a corner.
|
||||
|
||||
### 12c. Verify exposure objectively, not just by eye
|
||||
|
||||
A capture can look "bright" in vision judgment while being unrecoverable.
|
||||
Cheap objective check on any capture (editor-host filesystem):
|
||||
`ffprobe -f lavfi -i "movie=<png>,signalstats" -show_entries
|
||||
frame_tags=lavfi.signalstats.YAVG -of json` — YAVG > 250 means blown
|
||||
white, < 5 means black. Use it whenever a lighting change should have
|
||||
moved the histogram; it distinguishes "fog whiteout" from "exposure
|
||||
whiteout" faster than iterating blind.
|
||||
|
||||
### 13. In-memory edits are lost on crash — save per milestone
|
||||
|
||||
Everything you do lives in unsaved packages until a save happens. The editor
|
||||
is an application that can crash, especially mid-experimental-feature. Save
|
||||
the level + dirty packages after every milestone (`AssetTools.save_assets`,
|
||||
`SceneTools.save_actor`). Caveat: an UNTITLED level (`/Temp/Untitled_*`,
|
||||
the state after File > New or launching without a map argument) may route a
|
||||
save through the Save-As dialog — a modal that deadlocks the MCP loop
|
||||
(pitfall 8). Prefer starting from a saved level (pass the map path on the
|
||||
editor command line, or `SceneTools.load_level` a real `/Game/...` map)
|
||||
before doing hours of work.
|
||||
|
||||
### 14. Label ≠ Name ≠ path — use the full path as the stable identifier
|
||||
|
||||
Outliner shows actor LABELS (settable, duplicable, human-friendly). Internal
|
||||
NAMES are unique per level but auto-generated (`StaticMeshActor_3`). The only
|
||||
identifier that survives renames and disambiguates duplicates is the full
|
||||
object path: `/Game/Maps/Level.Level:PersistentLevel.BP_Character_C_0`.
|
||||
Tools may accept label, name, or path — read the schema. When you create an
|
||||
actor, immediately set a meaningful label, and record whatever handle the
|
||||
tool returns for later operations.
|
||||
|
||||
### 14b. Asset path forms mean different things
|
||||
|
||||
| Form | Example | Loads |
|
||||
|---|---|---|
|
||||
| Package | `/Game/Foo/Bar` | The package (asset-registry queries) |
|
||||
| Package.Asset | `/Game/Foo/Bar.Bar` | The primary asset (most load/assign args) |
|
||||
| Package.Asset_C | `/Game/Foo/Bar.Bar_C` | A Blueprint's **generated class** |
|
||||
|
||||
"Class not found: /Game/Path/BP_Foo" almost always means the missing `_C`
|
||||
suffix — spawning by Blueprint class needs the generated-class form.
|
||||
|
||||
### 14c. Property writes can silently no-op — round-trip verify
|
||||
|
||||
UPROPERTY names are PascalCase at the reflection layer; snake_case lookups
|
||||
through some write paths silently change nothing and return no error. After
|
||||
any property write that matters, READ THE VALUE BACK and compare (allowing
|
||||
formatting normalization like `1` vs `1.000000`). If it didn't take, retry
|
||||
with the exact PascalCase name shown by the property dump/schema.
|
||||
|
||||
### 15. Play In Editor changes the world (literally)
|
||||
|
||||
If the user hits Play, queries/edits may target the transient PIE world, and
|
||||
edits to it evaporate when play stops. If results look inexplicably
|
||||
transient or actor lists suddenly differ, ask whether PIE is running; do
|
||||
edit work outside PIE.
|
||||
|
||||
### 16. Undo exists, but don't lean on it
|
||||
|
||||
Editor transactions power Ctrl+Z; tool-driven changes may or may not create
|
||||
clean transaction boundaries depending on the tool's implementation. Treat
|
||||
undo as the user's manual escape hatch, not your rollback mechanism — your
|
||||
rollback is: query state, compute the inverse edit, apply it.
|
||||
|
||||
## Content & Assets
|
||||
|
||||
### 17. Long package names, not file paths
|
||||
|
||||
Assets are addressed as `/Game/Folder/Asset.Asset` (project content),
|
||||
`/Engine/...` (engine content), `/Script/Module.Class` (native classes).
|
||||
Windows-style or absolute filesystem paths are wrong everywhere except
|
||||
import/export file arguments and screenshot output paths.
|
||||
|
||||
### 18. Filesystem results land on the EDITOR host
|
||||
|
||||
Screenshots, renders, and exports write to the machine running Unreal (e.g.
|
||||
`<Project>/Saved/Screenshots/...`). If Hermes runs elsewhere (SSH backend,
|
||||
container), `read_file` on that path reads the wrong filesystem. Same-machine
|
||||
setups (the default here) can read captures directly.
|
||||
|
||||
### 19. Referenced ≠ loaded
|
||||
|
||||
Engine basics (`/Engine/BasicShapes/...`) are always available, but project
|
||||
assets may need loading before use, and a typo'd asset path often fails
|
||||
soft (empty mesh, default material) rather than loud. After assigning
|
||||
meshes/materials, re-query the actor to confirm the reference stuck.
|
||||
|
||||
### 20. Material edits: instances, not parents
|
||||
|
||||
Editing a parent Material recompiles shaders (slow, global blast radius).
|
||||
Create a Material Instance (Dynamic or Constant per the tool surface), set
|
||||
scalar/vector/texture parameters on it, assign to the mesh. Parameter names
|
||||
must match the parent's exposed parameters exactly — query/describe before
|
||||
setting; a misnamed parameter usually no-ops silently.
|
||||
|
||||
### 20b. Shader/asset compilation is async — don't judge or proceed early
|
||||
|
||||
Material creation/edits kick off shader compilation that can run seconds to
|
||||
minutes; Niagara compiles, DDC builds, and package saves are async too. A
|
||||
screenshot taken mid-compile shows the old (or default-checkerboard) state.
|
||||
After material work, wait for compilation before judging visuals (poll a
|
||||
compile/errors predicate if the tool surface has one; otherwise screenshot
|
||||
after a delay and re-check if it looks wrong). Same discipline after saves:
|
||||
don't chain a disk-read straight after a write.
|
||||
|
||||
### 20c. Emissive needs intensity > 1 to bloom
|
||||
|
||||
Emissive at ≤1.0 looks self-lit but never blooms. 3–10 gives visible glow —
|
||||
and the Post Process Volume must have Bloom enabled (default on).
|
||||
|
||||
### 20d. Crash patterns to avoid outright
|
||||
|
||||
Engine-level, any server: (a) deleting or transforming an ASSET while level
|
||||
actors still reference it → `RegisteredElementType` assertion, editor down,
|
||||
unsaved work gone — walk references first, swap actors to a replacement,
|
||||
then delete; (b) spawn→delete→spawn the same actor in rapid succession can
|
||||
corrupt the actor registry — don't tight-loop create/destroy cycles;
|
||||
(c) Niagara/MetaSound assertion during PIE reverts to last on-disk save —
|
||||
save BEFORE entering PIE when those subsystems are involved.
|
||||
|
||||
## Delivery
|
||||
|
||||
### 21. Screenshot judgment is part of the job
|
||||
|
||||
Don't declare a lighting/composition milestone done from numbers alone —
|
||||
capture the viewport, `vision_analyze` it, and art-direct (silhouette,
|
||||
exposure, horizon placement, scale against human height). The user is
|
||||
non-technical; you are the one with eyes on both the brief and the frame.
|
||||
|
||||
### 21b. Editor sprite icons appear in captures — hide them at the source
|
||||
|
||||
Viewport captures include per-component editor sprites (light bulbs,
|
||||
speaker icons, and the plain grey/blue billboard every empty Actor gets).
|
||||
They are editor overlay, NOT scene content — and for video/hero work they
|
||||
photobomb every frame. Do NOT try to remove them in post; a sprite
|
||||
overlapping scene geometry defeats 2D cleanup (live-verified: three
|
||||
inpainting strategies all either smeared letter edges or left outline
|
||||
pixels).
|
||||
|
||||
Fix at the source, before capturing: sprites are real components on the
|
||||
actor. `ActorTools.get_components` → find `Billboard`/`Sprite`/`Arrow`
|
||||
components → `ObjectTools.set_properties` with `{"bVisible": false}` on
|
||||
each (round-trip verify). `remove_component` fails on construction-time
|
||||
default subobjects ("Could not find subobject handle") — hide, don't
|
||||
remove. Sweep the whole scene in one ProgrammaticToolset script: iterate
|
||||
`find_actors`, hide every matching component (verified: 148 actors
|
||||
scanned, 13 sprites hidden, one round-trip). An editor capture is still
|
||||
not proof a Niagara effect is emitting — verify effects via actor state or
|
||||
a PIE capture.
|
||||
|
||||
### 21c. The viewport axis gizmo survives bShowUI=false — plan a post-crop
|
||||
|
||||
`CaptureViewport` with `bShowUI: false` hides menus and toolbars but the
|
||||
bottom-left XYZ axis gizmo is still burned into every frame
|
||||
(live-verified on 5.8; at 2027x1534 it occupies roughly the region below
|
||||
y≈1350, x<300). For video/hero deliverables, compose with spare margin and
|
||||
crop it out in post (e.g. a 16:9 punch-in via ffmpeg
|
||||
`crop=2027:1140:0:180,scale=1920:1080`) — deterministic across every frame
|
||||
of a sequence, no per-frame edits. Unlike the gizmo, actor sprite icons
|
||||
CAN be removed at the source — hide their components before the shoot
|
||||
(see 21b); do that instead of post-cleanup.
|
||||
|
||||
### 21d. Frame sequences: one client session, serial captures, resumable loop
|
||||
|
||||
For multi-frame virtual-camera moves (orbits, cranes) drive
|
||||
`CaptureViewport` in a loop from ONE MCP session, strictly serial (the
|
||||
game thread renders each), and write frames idempotently
|
||||
(skip-if-exists) so a killed run resumes where it stopped. Budget
|
||||
realistically: ~15-20 s/frame at editor resolution on a laptop — a 240-frame
|
||||
10 s @ 24 fps shot is roughly an hour. Smoothstep-ease camera parameters;
|
||||
constant angular velocity reads mechanical. Remove the VolumetricCloud
|
||||
actor if its low-res gather smears blocky artifacts across sky frames.
|
||||
|
||||
### 22. Report package paths + file paths
|
||||
|
||||
The user needs: what actors/assets now exist (labels + `/Game/...` paths),
|
||||
where the level was saved, and absolute filesystem paths of any
|
||||
captures/renders (delivered as `MEDIA:` where appropriate).
|
||||
|
||||
## Sources
|
||||
|
||||
Grounded in Epic's UE 5.8 Unreal MCP documentation, Epic's own agent-facing
|
||||
skill pack for this server (unreal-engine-skills-for-claude-code), and
|
||||
engine-level field reports from the UE-via-MCP community (ue5-mcp field
|
||||
manual). Engine behaviors (crash patterns, Lumen mobility, async compiles,
|
||||
reflection casing) are server-agnostic; tool names and schemas remain
|
||||
whatever the live `describe_toolset` says.
|
||||
@@ -0,0 +1,237 @@
|
||||
# Unreal MCP — Worked Recipes
|
||||
|
||||
Complete build sequences from a plain-English brief to a delivered capture.
|
||||
Written against the discovery contract, because the live tool surface is
|
||||
project-dependent: each step names the **capability to locate** (via
|
||||
`list_toolsets` / `describe_toolset`) and the **exact values** to feed it —
|
||||
not hardcoded tool names, which drift while the plugin is experimental.
|
||||
|
||||
## Recipe grammar
|
||||
|
||||
Every step = four parts:
|
||||
|
||||
INTENT what this step achieves
|
||||
DISCOVER which toolset/tool capability to use (locate via describe_toolset)
|
||||
VALUES the exact arguments/numbers (from scene-craft.md)
|
||||
VERIFY the query or screenshot that proves it worked
|
||||
|
||||
Dispatch shape: `call_tool` with `toolset_name`, `tool_name`, and an
|
||||
`arguments` object matching the described schema — result returns on the
|
||||
same turn.
|
||||
|
||||
LIGHTING RULE for every recipe: set **Mobility = Movable** on every light
|
||||
you spawn (Lumen GI ignores Static/Stationary lights — the #1 "why is GI
|
||||
dead" cause).
|
||||
|
||||
Session preamble for every recipe (do once):
|
||||
|
||||
1. `list_toolsets` → note the qualified names (e.g.
|
||||
`editor_toolset.toolsets.scene.SceneTools`,
|
||||
`EditorToolset.EditorAppToolset`).
|
||||
2. `describe_toolset` on each group you'll touch → cache schemas.
|
||||
3. Query current level (`SceneTools.get_current_level`) and inventory the
|
||||
environment: `find_actors` for DirectionalLight, SkyAtmosphere,
|
||||
SkyLight, ExponentialHeightFog, PostProcessVolume, VolumetricCloud.
|
||||
**Configure existing environment actors; spawn only what's missing** —
|
||||
template levels ship with most of them, and duplicates compound into
|
||||
whiteouts.
|
||||
4. Read the existing sun's `intensity` — it tells you the scene's exposure
|
||||
calibration (template worlds are often calibrated around `intensity: 10`,
|
||||
not physical lux; see pitfalls 12b before applying scene-craft absolute
|
||||
values).
|
||||
5. Locate your verification path: `EditorAppToolset.CaptureViewport` with a
|
||||
`captureTransform` is the virtual camera — no viewport piloting needed.
|
||||
|
||||
Save the level + dirty packages after every phase marked 💾. One tool call
|
||||
at a time throughout — no batching, ever.
|
||||
|
||||
---
|
||||
|
||||
## Recipe A — Daylight exterior clearing (blocking-first exterior)
|
||||
|
||||
Brief: "a sunny clearing with some rocks and a path"
|
||||
|
||||
**Phase 1 — environment shell**
|
||||
|
||||
- INTENT sky + sun + atmosphere exist and track each other.
|
||||
DISCOVER actor-spawn capability (spawn by class).
|
||||
VALUES spawn `SkyAtmosphere`, `SkyLight` (real-time capture),
|
||||
`DirectionalLight` at rotation (0, −55, 40) [roll, pitch, yaw],
|
||||
intensity 90,000 lux, temperature 5,800 K, "atmosphere sun light" on;
|
||||
`ExponentialHeightFog` density 0.008.
|
||||
VERIFY actor list shows all four; screenshot reads as daytime sky, not
|
||||
black (if black: exposure — see Phase 3).
|
||||
|
||||
**Phase 2 — ground & blocking**
|
||||
|
||||
- INTENT walkable ground plane.
|
||||
DISCOVER spawn-from-asset capability.
|
||||
VALUES `/Engine/BasicShapes/Plane.Plane` at (0,0,0), scale (100,100,1)
|
||||
→ 100×100 m ground. Label `Ground`.
|
||||
- INTENT rock cluster + path silhouette from primitives (placeholder for
|
||||
real assets if the project has none).
|
||||
VALUES 5–9 `/Engine/BasicShapes/Cube.Cube` at scattered locations within
|
||||
±2,000 cm of origin, non-uniform scales between (1.5,1.5,1) and (4,3,2),
|
||||
yaws randomized 0–360°, sunk 10–30 cm into the ground so nothing floats.
|
||||
A path: 6–10 flattened cubes scale ≈(1.2,0.8,0.05) snaking through.
|
||||
VERIFY screenshot at eye height (camera z≈165) along the path axis: rocks
|
||||
read as varied, nothing floats, scale sane against the 180 cm yardstick
|
||||
(place one 180 cm-tall cylinder temporarily as a human stand-in, delete
|
||||
after checking). 💾
|
||||
|
||||
**Phase 3 — exposure & mood**
|
||||
|
||||
- INTENT deterministic exposure.
|
||||
DISCOVER PostProcessVolume spawn + property-set capability.
|
||||
VALUES PPV unbound=true, metering Manual, EV100 = 14.5.
|
||||
VERIFY screenshot: bright but not blown; shadows readable.
|
||||
|
||||
**Phase 4 — deliver**
|
||||
|
||||
- INTENT hero still.
|
||||
VALUES `HighResShot 3840x2160` from a framed viewpoint (see Recipe C
|
||||
Phase 2 for framing rules).
|
||||
VERIFY file exists in Saved/Screenshots; `vision_analyze` against brief;
|
||||
iterate lighting yaw/fog once if flat. 💾 Report actor labels + paths.
|
||||
|
||||
---
|
||||
|
||||
## Recipe B — Moody practical-lit interior
|
||||
|
||||
Brief: "a dim cozy room at night, warm lamp, blue moonlight through window"
|
||||
|
||||
**Phase 1 — room shell from primitives**
|
||||
|
||||
- VALUES floor: Cube at (0,0,−10) scale (6,6,0.2) → 6×6 m room. Four walls:
|
||||
cubes scale (6,0.2,3) / (0.2,6,3) positioned at ±300 on the respective
|
||||
axis, z=140 (walls 280 cm tall, sitting on the floor plane; keep tops at
|
||||
z≈290). Ceiling: cube scale (6,6,0.2) at z≈290 — spawn it LAST so you can
|
||||
screenshot the interior while open-topped. One window: leave a gap in a
|
||||
wall by using two shorter wall segments with a 120×120 cm opening at
|
||||
sill height 90 cm.
|
||||
VERIFY top-down + interior screenshots; door/window heights sane.
|
||||
|
||||
**Phase 2 — lighting (the point of this recipe)**
|
||||
|
||||
- INTENT kill the sun; interior reads as night.
|
||||
VALUES if the template level has a DirectionalLight: intensity → 0.05 lux
|
||||
temperature 4,300 K, pitch −20°, yawed to rake through the window (this
|
||||
is the "moon"). SkyLight intensity scale down to ≈0.05–0.1.
|
||||
- INTENT warm practical.
|
||||
VALUES PointLight at lamp position (e.g. corner table, z≈120):
|
||||
800 lumens (or ≈64 candela), temperature 2,700 K, attenuation radius
|
||||
600 cm, source radius 10 cm (softer shadows).
|
||||
- INTENT cool window rim.
|
||||
VALUES SpotLight outside the window aimed through it: 2,000 lumens,
|
||||
6,500–8,000 K if faking without moon; skip if the directional moon
|
||||
already rakes through visibly. Inner/outer cone 25°/50°.
|
||||
- INTENT exposure for dim interior.
|
||||
VALUES PPV unbound, Manual, EV100 = 4.5; fog: ExponentialHeightFog
|
||||
density 0.015 + volumetric fog on; practical's volumetric scattering
|
||||
intensity 2–4 so the lamp glows.
|
||||
VERIFY screenshot from a corner at z≈160: warm pool around lamp, cool
|
||||
slash from window, deep-but-readable shadows. The warm/cool split IS the
|
||||
deliverable — iterate intensities (never move both at once) until it
|
||||
reads. 💾
|
||||
|
||||
**Phase 3 — dress & deliver**
|
||||
|
||||
- VALUES if Starter Content exists, swap primitives: `/Game/StarterContent/
|
||||
Props/SM_TableRound`, `SM_Chair`, `SM_Lamp_Ceiling`, materials
|
||||
`M_Wood_Pine` on floor, `M_Basic_Wall` on walls. Otherwise assign
|
||||
MaterialInstances with warm-neutral base colors to primitives.
|
||||
VERIFY final `HighResShot 3840x2160`, vision-check, 💾, report.
|
||||
|
||||
---
|
||||
|
||||
## Recipe C — Golden-hour cinematic still (camera craft)
|
||||
|
||||
Brief: "make it golden hour and give me a cinematic shot of <subject>"
|
||||
|
||||
**Phase 1 — relight for golden hour**
|
||||
|
||||
- VALUES DirectionalLight: intensity 12,000 lux, temperature 3,200 K,
|
||||
pitch −8°, yaw set so the sun is 30–60° OFF the camera axis behind the
|
||||
subject (rim + long shadows — never light flat from the camera).
|
||||
Fog density 0.02 + volumetric fog, sun volumetric scattering 2–6.
|
||||
PPV EV100 = 11.
|
||||
VERIFY screenshot: long shadows, warm rim on subject edges.
|
||||
|
||||
**Phase 2 — the camera**
|
||||
|
||||
- INTENT a framed shot WITHOUT touching the user's viewport.
|
||||
DISCOVER `EditorAppToolset.CaptureViewport` with `captureTransform` — a
|
||||
virtual camera; no CineCamera or viewport piloting needed for stills.
|
||||
VALUES position: subject-distance by lens-equivalent framing — for a
|
||||
prop/monument subject ~500–800 cm back, height 120–160 cm; rotation
|
||||
aimed so the subject sits on a thirds intersection, horizon in upper or
|
||||
lower third. Slight upward pitch (+2° to +5°) from below eye height
|
||||
reads heroic and guarantees sky/horizon in frame.
|
||||
For an actual CineCameraActor (user wants a camera in the level, DoF,
|
||||
or a Sequencer shot): spawn `/Script/CinematicCamera.CineCameraActor`,
|
||||
set focal/aperture/focus via ObjectTools on its CineCameraComponent,
|
||||
then capture with `captureTransform` matching its transform.
|
||||
VERIFY capture at viewport res first; iterate framing cheaply, then take
|
||||
the final.
|
||||
|
||||
**Phase 3 — deliver**
|
||||
|
||||
- VALUES `HighResShot 3840x2160` (or user's target res) through the
|
||||
piloted camera. For a sequence/turntable instead of a still: this needs a
|
||||
Level Sequence with a Camera Cut track + Movie Render Queue — treat as
|
||||
its own task; warn about first-render shader-compile stall.
|
||||
VERIFY read file, `vision_analyze`: rim light present? focus falloff on
|
||||
the right plane? horizon off-center? Iterate at most twice, then deliver
|
||||
MEDIA: path + what was changed. 💾
|
||||
|
||||
---
|
||||
|
||||
## Recipe D — Import an asset and populate the scene
|
||||
|
||||
Brief: "here's model.fbx / a Fab asset — put a ring of them around the fountain"
|
||||
|
||||
**Phase 1 — import**
|
||||
|
||||
- INTENT asset lands in `/Game/Imported` with no dialog stall.
|
||||
DISCOVER import capability (if none advertised: custom toolset wrapping
|
||||
`unreal.AssetImportTask` with `automated=True` — see tool-surface.md;
|
||||
the `automated` flag is what prevents a modal import dialog from
|
||||
freezing the whole MCP loop).
|
||||
VALUES destination `/Game/Imported`, save=true.
|
||||
VERIFY asset-exists query on the resulting long package name; spawn one
|
||||
instance at origin, screenshot, check scale against 180 cm yardstick —
|
||||
DCC exports are routinely 100× off (meters vs centimeters). Fix by
|
||||
actor scale or reimport with unit conversion.
|
||||
|
||||
**Phase 2 — populate**
|
||||
|
||||
- INTENT ring of N instances around a center C.
|
||||
VALUES for i in 0..N−1: angle θ=360·i/N, position = C + (r·cosθ,
|
||||
r·sinθ, 0) with r = fountain radius + clearance (e.g. 350 cm), yaw =
|
||||
θ+90° so each faces the center (or +270° to face outward — check one
|
||||
instance first and LOOK). Spawn one, verify facing, then loop the rest
|
||||
one call at a time.
|
||||
VERIFY count query matches N; screenshot from above (camera z≈1,500
|
||||
looking down) for spacing; eye-level screenshot for scale. 💾
|
||||
|
||||
**Phase 3 — deliver**
|
||||
|
||||
- Report: asset path, N instances with label prefix, level saved,
|
||||
overview + eye-level captures as MEDIA paths.
|
||||
|
||||
---
|
||||
|
||||
## When a recipe's capability is missing
|
||||
|
||||
If discovery shows no shipped tool for a step (no import tool, no
|
||||
console-exec for HighResShot, etc.):
|
||||
|
||||
1. Say so plainly; don't fake the step.
|
||||
2. Offer the custom-toolset path (tool-surface.md) — a 20-line Python
|
||||
toolset usually covers the gap; it needs `RefreshTools` + session
|
||||
restart to appear.
|
||||
3. Or hand the user the one-liner to run in the editor's Python/console
|
||||
themselves, with exact text.
|
||||
|
||||
Never claim a phase done without its VERIFY evidence. The user can't check
|
||||
the editor for you — the screenshots are the ground truth they see.
|
||||
@@ -0,0 +1,280 @@
|
||||
# Unreal MCP — Scene-Craft Cheat Sheet
|
||||
|
||||
The numbers and conventions that make a scene read as *good* instead of
|
||||
merely present. Sources: physical/photographic standards (stable), UE
|
||||
conventions (stable), and practical ranges from production use (marked ≈).
|
||||
UE-version-specific defaults drift; when a live schema or editor value
|
||||
disagrees with this sheet, trust the editor and patch the sheet.
|
||||
|
||||
## Units & Conventions (bedrock — memorize)
|
||||
|
||||
| Thing | Convention |
|
||||
|---|---|
|
||||
| Distance | 1 Unreal Unit = **1 cm** |
|
||||
| Axes | **Z-up**, X-forward, Y-right (left-handed) |
|
||||
| Rotation | Rotator in **degrees**: Roll (around X), Pitch (around Y), Yaw (around Z) |
|
||||
| Color | Linear RGBA, each channel 0–1 (`FLinearColor`) |
|
||||
| Light color | Prefer `use_temperature` + Kelvin over tinting RGB |
|
||||
| Scale | Multiplier per axis (1,1,1 = authored size) |
|
||||
|
||||
Directional-light aiming: the ROTATION points the light. Pitch −90° = sun
|
||||
straight overhead (noon); pitch −5° to −15° = sun grazing the horizon
|
||||
(golden hour); yaw picks the compass direction the light travels toward.
|
||||
|
||||
### Human-scale reference (sanity-check every layout against these)
|
||||
|
||||
| Reference | Size (cm) |
|
||||
|---|---|
|
||||
| Eye height (standing) | 160–175 |
|
||||
| Door | 200–210 tall × 80–90 wide |
|
||||
| Ceiling, residential | 240–300 |
|
||||
| One building storey | 300–400 |
|
||||
| Counter/desk height | 75–110 |
|
||||
| Chair seat | 45 |
|
||||
| Stair riser / tread | ≈18 / ≈28 |
|
||||
| UE default mannequin | ≈180 tall |
|
||||
| Car | ≈450 long × 180 wide × 145 tall |
|
||||
|
||||
If a "house" door comes out 400 cm tall, the scene reads as toy/giant.
|
||||
Always place one human-scale object early as a yardstick.
|
||||
|
||||
## Content Paths
|
||||
|
||||
| Root | Meaning |
|
||||
|---|---|
|
||||
| `/Game/...` | Project content (Content/ folder) |
|
||||
| `/Engine/...` | Engine-shipped content, present in every project |
|
||||
| `/Script/Module.Class` | Native classes (e.g. `/Script/Engine.PointLight`) |
|
||||
|
||||
Long package name form: `/Game/Props/SM_Chair.SM_Chair` (package.object).
|
||||
|
||||
Always-available engine primitives (great for blocking before real assets):
|
||||
|
||||
/Engine/BasicShapes/Cube.Cube (100×100×100 cm at scale 1)
|
||||
/Engine/BasicShapes/Sphere.Sphere (100 cm diameter)
|
||||
/Engine/BasicShapes/Cylinder.Cylinder (100 cm ⌀ × 100 cm)
|
||||
/Engine/BasicShapes/Cone.Cone
|
||||
/Engine/BasicShapes/Plane.Plane (100×100 cm, single-sided)
|
||||
|
||||
Their default material is plain grey; assign a MaterialInstance for anything
|
||||
presentational. If the project has Starter Content, useful packs live under
|
||||
`/Game/StarterContent/` (Props, Materials like `M_Basic_Wall`, `M_Wood_Pine`,
|
||||
`M_Metal_Steel`, Particles). Query before assuming Starter Content exists.
|
||||
|
||||
Common actor classes for spawning: `StaticMeshActor`, `PointLight`,
|
||||
`SpotLight`, `RectLight`, `DirectionalLight`, `SkyLight`,
|
||||
`ExponentialHeightFog`, `SkyAtmosphere`, `VolumetricCloud`,
|
||||
`PostProcessVolume`, `CameraActor`, `CineCameraActor`, `PlayerStart`.
|
||||
|
||||
## Lighting — physically based values
|
||||
|
||||
UE5 lights default to physical units (directional in lux, point/spot in
|
||||
candela or lumens, exposure in EV100). Use real-world values; they
|
||||
compose correctly with exposure instead of fighting it.
|
||||
|
||||
### Sun (DirectionalLight, lux)
|
||||
|
||||
**Calibration check first (live-verified):** template levels often ship a
|
||||
sun at `intensity: 10` with auto-exposure tuned around it — physical lux
|
||||
values below will blow such a scene to white. Read the existing sun's
|
||||
intensity; if it's single/double digits, scale moods RELATIVE to it (noon =
|
||||
template value, golden hour ≈ 0.5–0.7×, overcast ≈ 0.3×, night ≈ 0.01×)
|
||||
and rely on temperature + pitch for the mood. The absolute table applies
|
||||
when you own the whole exposure chain (manual EV100 + physical values
|
||||
everywhere):
|
||||
|
||||
| Condition | Intensity (lux) | Pitch | Temperature |
|
||||
|---|---|---|---|
|
||||
| Noon, clear | 75,000–120,000 | −60° to −90° | 5,500–6,000 K |
|
||||
| Afternoon | 40,000–75,000 | −30° to −50° | 5,000–5,500 K |
|
||||
| Golden hour | 5,000–20,000 | −5° to −15° | 2,800–3,500 K |
|
||||
| Overcast | 5,000–20,000 (soft) | −45° ± | 6,500–7,500 K |
|
||||
| Blue hour / dusk | 10–100 | −2° to +5° | 8,000–12,000 K |
|
||||
| Full-moon night | 0.05–0.3 | −30° to −60° | 4,000–4,500 K (cool-blue read comes from exposure + grade) |
|
||||
|
||||
Overcast: also drop directional shadow contrast (soften via larger source
|
||||
angle) and let the sky light dominate.
|
||||
|
||||
### Sky light
|
||||
|
||||
One SkyLight per level, Real-Time Capture (SLS Captured Scene) when using
|
||||
SkyAtmosphere — it then tracks the sun automatically. Don't stack multiple
|
||||
sky lights; don't leave a stale static capture after big lighting changes
|
||||
(recapture if not real-time).
|
||||
|
||||
### Local lights (point/spot/rect)
|
||||
|
||||
Rules of thumb in lumens (candela ≈ lumens/(4π) for a point light):
|
||||
|
||||
| Source | Lumens |
|
||||
|---|---|
|
||||
| Candle flame | 10–15 (≈1,850 K) |
|
||||
| 40 W incandescent equiv. | 450 (2,700 K) |
|
||||
| 60 W equiv. | 800 (2,700–3,000 K) |
|
||||
| 100 W equiv. | 1,600 (3,000 K) |
|
||||
| Bright ceiling fixture | 2,000–4,000 (3,000–4,000 K) |
|
||||
| Fluorescent tube / office | 2,500–5,000 (4,000–5,000 K) |
|
||||
| Streetlight (sodium) | 5,000–15,000 (≈2,000 K, orange) |
|
||||
| Car headlight | 1,000–1,500 each (4,300–6,000 K) |
|
||||
| Campfire | 100–300, flicker (1,700–2,000 K) |
|
||||
|
||||
Spot cone: inner 20–35°, outer 40–60° for a natural falloff. Attenuation
|
||||
radius: keep tight (a few hundred cm for practicals) — giant radii cost
|
||||
performance and flatten the scene. Cast-shadow off for pure fill lights.
|
||||
|
||||
### Color temperature vocabulary (Kelvin)
|
||||
|
||||
1,700–1,900 match/candle · 2,700 warm bulb · 3,200 tungsten studio ·
|
||||
3,500 golden hour · 4,300 moonlight-read · 5,600 daylight/flash ·
|
||||
6,500 overcast · 7,500–10,000 shade/blue hour.
|
||||
Warm subject + cool ambient (or inverse) is the cheapest way to make a
|
||||
shot read "lit" instead of "flat".
|
||||
|
||||
### Exposure (PostProcessVolume — the #1 "why is it black/white" knob)
|
||||
|
||||
Auto-exposure fights deterministic lighting reads. For agent-driven work,
|
||||
prefer **manual exposure** in a PPV:
|
||||
|
||||
1. Spawn/locate a PostProcessVolume, set **Infinite Extent (Unbound) = true**.
|
||||
2. Metering Mode = Manual, then set Exposure Compensation ≈ 0 and EV100 to
|
||||
the scene value:
|
||||
|
||||
| Scene | EV100 |
|
||||
|---|---|
|
||||
| Bright sun exterior | 14–16 |
|
||||
| Overcast exterior | 11–13 |
|
||||
| Golden hour | 10–12 |
|
||||
| Bright interior (day, windows) | 7–9 |
|
||||
| Dim practical-lit interior | 4–6 |
|
||||
| Street at night | 2–4 |
|
||||
| Moonlit exterior | −2 to 0 |
|
||||
|
||||
If you keep auto-exposure instead (Metering: Auto Histogram), clamp it:
|
||||
Min/Max EV100 within ±2 of the target so it can't swim. Symptom table:
|
||||
scene renders black with lights present → EV100 too high for the light
|
||||
levels; blown white → EV100 too low.
|
||||
|
||||
### Global illumination & reflections
|
||||
|
||||
UE5 defaults: **Lumen** GI + Lumen reflections, no lightmass bake needed —
|
||||
lighting is live; just keep "Allow Static Lighting" defaults alone.
|
||||
**Critical: Lumen GI only considers lights with Movable mobility.** Spawned
|
||||
lights can default to Stationary/Static and then contribute nothing to GI —
|
||||
set Mobility = Movable explicitly on every light you place, and check
|
||||
mobility first when "GI isn't working". Metal/mirror surfaces read
|
||||
correctly only with something to reflect: give the scene a sky and
|
||||
surroundings before judging materials.
|
||||
|
||||
### Fog & atmosphere
|
||||
|
||||
- **SkyAtmosphere** for a physically-plausible sky (sun disk, horizon
|
||||
gradient); pairs with directional light "Atmosphere Sun Light = true".
|
||||
- **ExponentialHeightFog**: density default 0.02. ≈ Practical ranges:
|
||||
0.005–0.015 subtle depth cue · 0.02–0.05 moody/morning · 0.05–0.2 heavy.
|
||||
Enable **Volumetric Fog** on it for light shafts through it; then dial
|
||||
per-light "Volumetric Scattering Intensity" (1–10) on the key lights.
|
||||
- **VolumetricCloud** for real sky clouds (exterior only; costs GPU).
|
||||
- Night sky: drop fog density, add faint cool fill (skylight at low
|
||||
intensity) so shadows aren't pure black.
|
||||
|
||||
## Mood recipes (compact)
|
||||
|
||||
| Mood | Sun/Key | Sky | Fog | EV100 | Grade notes |
|
||||
|---|---|---|---|---|---|
|
||||
| Crisp noon | 100k lux, pitch −70°, 5,800 K | Real-time capture | 0.005 | 15 | Neutral |
|
||||
| Golden hour | 10k lux, pitch −8°, 3,200 K | Real-time capture | 0.02 + volumetric | 11 | Warm key, long shadows: rotate yaw for rim/side light |
|
||||
| Overcast | 10k lux soft, 7,000 K | Dominant | 0.01 | 12 | Low contrast, saturation carries color |
|
||||
| Night, moonlit | 0.25 lux, 4,300 K + practicals ≈800 lm 2,700 K | Very low | 0.015 | −1 to 1 | Cool ambient vs warm practicals |
|
||||
| Horror interior | No sun; 1–2 practicals, hard shadows | Minimal | 0.03–0.06 volumetric | 4–5 | Single motivated key, deep blacks |
|
||||
| Sci-fi corridor | Rect lights 2,000 lm 6,500–8,000 K + colored accents | None | 0.02 volumetric | 6 | Complementary accent pair (cyan/orange) |
|
||||
|
||||
## Camera & framing (CineCameraActor)
|
||||
|
||||
Use CineCameraActor (not plain Camera) for anything presentational — it has
|
||||
real filmback/lens/DoF controls.
|
||||
|
||||
| Intent | Focal length | Aperture |
|
||||
|---|---|---|
|
||||
| Establishing / interior wide | 18–28 mm | f/5.6–8 |
|
||||
| Neutral "human eye" | 35–50 mm | f/4 |
|
||||
| Portrait / subject isolation | 85–135 mm | f/1.4–2.8 |
|
||||
| Compression (stacked background) | 100–200 mm | f/2.8–5.6 |
|
||||
|
||||
- Filmback: default 16:9 digital film (≈23.76 × 13.365 mm) is fine; leave it.
|
||||
- Focus: Manual focus distance = distance camera→subject in cm; shallow DoF
|
||||
needs long lens + wide aperture + subject far from background.
|
||||
- Placement: eye-level ≈ 155–170 cm for neutral shots; below ≈ 100 cm =
|
||||
heroic/imposing; high angle = diminishing. Keep the horizon off
|
||||
dead-center; put subjects on thirds. Slight camera pitch (−2° to −8°)
|
||||
usually beats perfectly level for interiors.
|
||||
- Aspect/eye candy: enable camera's "Constrain Aspect Ratio" for clean
|
||||
letterboxed stills.
|
||||
|
||||
## Capture & render
|
||||
|
||||
- **Viewport screenshot**: console `HighResShot 1` (viewport res),
|
||||
`HighResShot 2` (2×), or `HighResShot 3840x2160`. Output:
|
||||
`<Project>/Saved/Screenshots/<Platform>/`. Filenames auto-increment
|
||||
(`HighresScreenshot00000.png`).
|
||||
- To frame from a camera: pilot/possess the CineCamera (or set viewport to
|
||||
its view) before HighResShot; verify by screenshotting first at 1×.
|
||||
- **Movie Render Queue** (MRQ) is the quality path for finals/sequences:
|
||||
needs a Level Sequence with the camera bound (Camera Cut track); renders
|
||||
PNG/EXR sequences or stills at arbitrary resolution with anti-aliasing
|
||||
temporal sample counts. First render after opening a project stalls on
|
||||
shader compilation — warn the user, don't declare it hung.
|
||||
- Judge results by looking: read the file back and `vision_analyze` every
|
||||
capture against the brief.
|
||||
|
||||
## Editor Python quick reference
|
||||
|
||||
Custom toolsets and any shipped Python-execution tool speak the `unreal`
|
||||
module. Canonical entry points (verify names against the live editor —
|
||||
Epic migrates libraries to subsystems over time):
|
||||
|
||||
```python
|
||||
import unreal
|
||||
|
||||
# Actors (EditorActorSubsystem supersedes EditorLevelLibrary for these)
|
||||
eas = unreal.get_editor_subsystem(unreal.EditorActorSubsystem)
|
||||
actors = eas.get_all_level_actors()
|
||||
actor = eas.spawn_actor_from_class(unreal.PointLight, unreal.Vector(0, 0, 200))
|
||||
mesh_a = eas.spawn_actor_from_object(
|
||||
unreal.EditorAssetLibrary.load_asset("/Engine/BasicShapes/Cube.Cube"),
|
||||
unreal.Vector(0, 0, 50))
|
||||
actor.set_actor_label("Key Light")
|
||||
actor.set_actor_location(unreal.Vector(100, 0, 250), False, True)
|
||||
actor.set_actor_rotation(unreal.Rotator(0, -30, 45), True) # roll, pitch, yaw
|
||||
eas.destroy_actor(actor)
|
||||
|
||||
# Assets
|
||||
unreal.EditorAssetLibrary.does_asset_exist("/Game/Props/SM_Chair")
|
||||
unreal.EditorAssetLibrary.list_assets("/Game/Props", recursive=True)
|
||||
unreal.EditorAssetLibrary.save_directory("/Game", only_if_is_dirty=True)
|
||||
|
||||
# Level save
|
||||
les = unreal.get_editor_subsystem(unreal.LevelEditorSubsystem)
|
||||
les.save_current_level()
|
||||
|
||||
# Undo-friendly mutation
|
||||
with unreal.ScopedEditorTransaction("Agent: dress set") as trans:
|
||||
... # property edits inside are one undo step
|
||||
|
||||
# Import (FBX/textures)
|
||||
task = unreal.AssetImportTask()
|
||||
task.filename = "/abs/path/model.fbx"
|
||||
task.destination_path = "/Game/Imported"
|
||||
task.automated = True # suppresses the import dialog — critical for MCP
|
||||
task.save = True
|
||||
unreal.AssetToolsHelpers.get_asset_tools().import_asset_tasks([task])
|
||||
|
||||
# Editor property access works on anything reflected
|
||||
light_comp = actor.get_component_by_class(unreal.PointLightComponent)
|
||||
light_comp.set_editor_property("intensity", 800.0)
|
||||
light_comp.set_editor_property("use_temperature", True)
|
||||
light_comp.set_editor_property("temperature", 2700.0)
|
||||
```
|
||||
|
||||
`set_editor_property`/`get_editor_property` with snake_case names is the
|
||||
universal fallback when a dedicated setter doesn't exist — property names
|
||||
match what the Details panel shows (spaces removed).
|
||||
@@ -0,0 +1,364 @@
|
||||
# Unreal MCP — Tool Surface Reference
|
||||
|
||||
How Epic's editor-embedded MCP server organizes, advertises, and executes
|
||||
tools, and how to extend the surface when the shipped tools run out.
|
||||
Everything here is against UE 5.8's experimental plugin (id
|
||||
`ModelContextProtocol`); expect drift between engine versions — the live
|
||||
`describe_toolset` schema always outranks this file.
|
||||
|
||||
## Architecture in one paragraph
|
||||
|
||||
The **Unreal MCP** plugin hosts the HTTP server inside the editor process
|
||||
(default `http://127.0.0.1:8000/mcp`, loopback-only, no auth, HTTP + SSE
|
||||
only — no stdio/WebSocket). It implements the protocol but ships no tools of
|
||||
its own. Tools come from **Toolsets** — classes deriving from
|
||||
`UToolsetDefinition` (C++) or `unreal.ToolsetDefinition` (Python) — collected
|
||||
at startup by the **Toolset Registry** subsystem (sibling plugin,
|
||||
auto-enabled; the registry itself ships no toolsets either). The shipped
|
||||
tools live in per-domain plugins under `Engine/Plugins/Experimental/
|
||||
Toolsets/` — the workhorse is **EditorToolset** (core editor toolsets,
|
||||
Python + C++) — and **AllToolsets** is a one-checkbox aggregator plugin
|
||||
that depends on ~21 of them (verified from `AllToolsets.uplugin`, 5.8):
|
||||
AIModule, AnimationAssistant, AutomationTest, ConfigSettings, Conversation,
|
||||
DataRegistry, DataflowAgent, Editor, GameFeatures, GameplayTags, GAS,
|
||||
MCPClient, Niagara, PCG, Physics, Plugin, SemanticSearch, SlateInspector,
|
||||
StateTree, UMG, WorldConditions. Project plugins and Game Feature Plugins
|
||||
can contribute more. Unreal MCP wraps every registered tool call as an MCP
|
||||
Tool. Execution is **serialized onto the game thread** — one tool call at a
|
||||
time, editor UI blocked while each runs.
|
||||
|
||||
## Tool-search mode (the default contract)
|
||||
|
||||
With `Enable Tool Search` on (default), `tools/list` advertises exactly three
|
||||
meta-tools:
|
||||
|
||||
| Meta-tool | Args | Returns |
|
||||
|---|---|---|
|
||||
| `list_toolsets` | — | Registered toolset names + descriptions |
|
||||
| `describe_toolset` | toolset name | JSON Schemas for every tool in that toolset |
|
||||
| `call_tool` | toolset/tool name + arguments object | The tool's result, same turn |
|
||||
|
||||
Discipline:
|
||||
|
||||
- `list_toolsets` once per session; re-run only after `RefreshTools`, plugin
|
||||
changes, or reconnect.
|
||||
- `describe_toolset` before first use of any toolset. Parameter names, types,
|
||||
and required fields come from the schema — never from memory or this file.
|
||||
- Results: primitive results arrive wrapped as `{"result": ...}` (CVar
|
||||
`ModelContextProtocol.WrapPODToolResultsInObject`, default true).
|
||||
Structured results serialize with field-level schema.
|
||||
- Errors come back as tool-call errors with the engine-side message — read
|
||||
them; they usually name the offending parameter or missing asset.
|
||||
|
||||
Eager mode (`Enable Tool Search` off) advertises every tool individually.
|
||||
Under Hermes that means each tool becomes `mcp_unreal_engine_<tool_name>` at
|
||||
session start, and `hermes mcp configure unreal-engine` can prune the list.
|
||||
Schema payload grows with every registered toolset, and tool authors are told
|
||||
NOT to rely on eager advertising — stay in tool-search mode unless a very
|
||||
small fixed surface is wanted.
|
||||
|
||||
## call_tool dispatch semantics (live-verified, 5.8)
|
||||
|
||||
Verified against a running 5.8 server; these details are where naive
|
||||
clients die:
|
||||
|
||||
- `list_toolsets` returns **fully-qualified** toolset names — Python:
|
||||
`editor_toolset.toolsets.scene.SceneTools`; C++:
|
||||
`EditorToolset.EditorAppToolset`. Epic's prose says "SceneTools"; the
|
||||
registry speaks qualified names. Use them verbatim in `describe_toolset`
|
||||
and `call_tool`'s `toolset_name`.
|
||||
- `tool_name` must be the **short** name (`get_current_level`,
|
||||
`CaptureViewport`). Passing the fully-qualified tool name fails with
|
||||
"Unknown tool" even though `describe_toolset` displays qualified names.
|
||||
- `call_tool` args: `{"toolset_name": ..., "tool_name": ..., "arguments":
|
||||
{...}}`; result returns on the same turn (the HTTP response blocks until
|
||||
the game thread finishes the call).
|
||||
- **`TOptional` parameters must be passed explicitly as `null`** — omitting
|
||||
them errors with `input param "X" needs a default value`. E.g.
|
||||
`CaptureViewport` minimal call is `{"captureTransform": null,
|
||||
"annotations": null, "bShowUI": false}`.
|
||||
- **Schema `required` is literal.** `find_actors` marks `name`, `tag`,
|
||||
`collision_channels` required even though they're semantically optional —
|
||||
pass `""` / `[]` to mean "any".
|
||||
- **Property names are camelCase with UE's `b` prefix intact** at this
|
||||
reflection layer: `bUseTemperature`, `bAtmosphereSunLight`, `fogDensity`,
|
||||
`bRealTimeCapture`, `mobility`. Writing `useTemperature` does NOT error
|
||||
the whole call — the response names each property that could not be set
|
||||
(schema-in-error style; READ error text, it lists the exact failures and
|
||||
often the full input schema).
|
||||
- **Object references travel as `{"refPath": "<soft object path>"}`**
|
||||
everywhere (actors, classes, components). Class refs use
|
||||
`/Script/Module.Class` (e.g. `/Script/Engine.PointLight`); actor refs are
|
||||
the full path (`/Temp/Untitled_1.Untitled_1:PersistentLevel.DirectionalLight_UAID_...`).
|
||||
Spawn/find tools RETURN refPaths — capture and reuse them.
|
||||
- **`ObjectTools.set_properties` takes `values` as a JSON *string***, not
|
||||
an object: `{"instance": {"refPath": ...}, "values":
|
||||
"{\"intensity\": 10.0}"}`. `get_properties` likewise returns a JSON
|
||||
string inside `returnValue`. Double-encode/decode accordingly.
|
||||
- Primitive results arrive wrapped as `{"returnValue": ...}` inside the
|
||||
text content block.
|
||||
|
||||
### HTTP wire behavior (for raw clients / debugging)
|
||||
|
||||
- `initialize` → plain JSON response + `Mcp-Session-Id` header you must
|
||||
echo on every subsequent request; `notifications/initialized` → 202
|
||||
empty; `tools/call` → **`text/event-stream`**: the result arrives as an
|
||||
`event: message` + `data: <jsonrpc>` frame only when the game thread
|
||||
finishes. A client that treats the response as plain JSON reads an empty
|
||||
body. Send `Accept: application/json, text/event-stream` always.
|
||||
|
||||
## Shipped toolsets
|
||||
|
||||
The registry is project-dependent; `describe_toolset` on the live server is
|
||||
the only source of truth for schemas. The core surface below is verified
|
||||
against EditorToolset's source in the 5.8 install (Python:
|
||||
`.../EditorToolset/Content/Python/editor_toolset/toolsets/`; C++:
|
||||
`EditorAppToolset.h`).
|
||||
|
||||
**EditorToolset plugin (the core), Python toolsets** (live-verified on 5.8;
|
||||
qualified prefix `editor_toolset.toolsets.<module>.<Class>`):
|
||||
|
||||
| Toolset | Verified tools (subset) |
|
||||
|---|---|
|
||||
| `scene.SceneTools` | `load_level`, `get_current_level`, `find_actors` (by name/type/tag/bounds), `add_to_scene_from_class`, `add_to_scene_from_asset`, `remove_from_scene`, `save_actor`, `create_level_instance`, folders |
|
||||
| `actor.ActorTools` | `get_label`/`set_label`, tags, `get_actor_transform`/`set_actor_transform` (`xform` fields optional = "don't change"), parenting, components |
|
||||
| `primitive.PrimitiveTools` | `add_cube` (dimensions), `add_sphere` (radius), `add_cylinder`/`add_cone` (radius+height) — adds StaticMeshComponents with `local_transform` to a host actor: spawn `/Script/Engine.Actor`, then compose. The fastest blocking path, zero asset dependencies |
|
||||
| `object.ObjectTools` | `list_properties` (returns full JSON schema of every property), `get_properties`/`set_properties` (JSON-string `values`), `reset_properties` (restore defaults — also your rollback), `get_class`, `search_subclasses` |
|
||||
| `material_instance.MaterialInstanceTools` | `create`, `list_parameters`, `get/set_scalar_parameter`, `get/set_vector_parameter` |
|
||||
| `asset.AssetTools` | `find_assets`, `load_asset`, `exists`, `save_assets`, `is_dirty`, `get_dependencies`/`get_referencers` (check before delete!), `delete`, `move`, `duplicate`, folders, `read_file`/`write_file` (project-scoped) |
|
||||
| `blueprint.BlueprintTools` (+ dsl/layout/node) | Blueprint authoring |
|
||||
| `material.MaterialTools`, `static_mesh.StaticMeshTools`, `texture.TextureTools`, `data_table.DataTableTools`, … | per-asset-type operations |
|
||||
| `programmatic.ProgrammaticToolset` | **the batching escape hatch** — see below |
|
||||
|
||||
**`EditorToolset.EditorAppToolset` (C++, same plugin) — the agent's eyes
|
||||
(full live list):** `CaptureViewport`, `CaptureEditorImage`,
|
||||
`CaptureAssetImage`, `GetCameraTransform`/`SetCameraTransform`,
|
||||
`GetSelectedActors`/`SelectActors`/`FocusOnActors`/`GetVisibleActors`,
|
||||
`WorldPosToScreenCoords`/`ScreenCoordsToWorld`,
|
||||
`GetSelectedAssets`/`SelectAssets`,
|
||||
`GetContentBrowserPath`/`SetContentBrowserPath`, `OpenEditorForAsset`,
|
||||
`GetOpenAssets`, `SearchCVars`, `StartPIE`/`StopPIE`/`IsPIERunning`.
|
||||
|
||||
`CaptureViewport` specifics (live-verified): args `{"captureTransform":
|
||||
<transform-or-null>, "annotations": <config-or-null>, "bShowUI": false}`.
|
||||
Returns base64 PNG (decode + save it yourself) plus camera
|
||||
location/rotation/FOV. `captureTransform` captures from any pose WITHOUT
|
||||
moving the user's viewport — use it as a virtual camera. Annotation config
|
||||
`{"gridSpacingCm": 500, "gridExtentCm": 3000, "gridHeight": <ground Z>,
|
||||
"labelActors": true}` overlays a projected ground grid and actor callouts;
|
||||
**grid coordinate labels are in METERS** (world cm ÷ 100). Use annotated
|
||||
captures for placement work, clean ones for beauty checks.
|
||||
|
||||
Also confirmed live: `ToolsetRegistry.AgentSkillToolset`,
|
||||
`EditorToolset.LogsToolset` (read Output Log + set verbosity — useful for
|
||||
self-debugging), `SemanticSearchToolset` (hybrid vector+BM25 asset search),
|
||||
five `NiagaraToolsets.NiagaraToolset_*` groups, `PCGToolset` (+Spatial),
|
||||
`UMGToolSet`, three `GASToolsets.*`, `AutomationTestToolset`,
|
||||
`ConfigSettingsToolset` (read/write Project Settings & Editor Preferences
|
||||
sections by schema — the remote path to exposure defaults, rendering
|
||||
settings, etc.), `SlateInspectorToolset`, `PluginToolset`,
|
||||
`animation_toolset.toolsets.sequencer.SequencerTools` + keyframing/
|
||||
controlrig/outliner siblings, `aimodule_toolset` BehaviorTreeTools,
|
||||
`state_tree_toolset` StateTreeTools, and more — 67 toolsets on a blank
|
||||
project with AllToolsets enabled.
|
||||
|
||||
Known gap: no mesh-modelling tools — spawn/place/instance existing meshes,
|
||||
yes; author new geometry, no. The supported route to parametric geometry is
|
||||
a custom Python toolset wrapping **Geometry Script** (`UDynamicMesh`:
|
||||
append box/cylinder/sphere, booleans, then `Create New Static Mesh Asset
|
||||
from Mesh` to bake an `SM_` asset). For organic/sculpted meshes, model in
|
||||
Blender and import.
|
||||
|
||||
First-session move: `list_toolsets`, then `describe_toolset` each group you
|
||||
plan to use, and keep those schemas in working memory for the session.
|
||||
|
||||
## ProgrammaticToolset — sanctioned batching
|
||||
|
||||
The serial-call rule makes N-step edits slow over the wire. The shipped
|
||||
answer is `ProgrammaticToolset` (verified in `programmatic.py`):
|
||||
|
||||
1. `get_execution_environment` — **mandatory first call** (the tool's own
|
||||
docstring requires it); returns the allowed modules, script constraints,
|
||||
and usage instructions.
|
||||
2. `execute_tool_script(script)` — runs a **sandboxed** Python script that
|
||||
defines `run() -> dict`. Inside, you call other registered tools
|
||||
programmatically and glue them with logic — one MCP round-trip for a
|
||||
whole loop (e.g. spawn 20 actors with computed transforms).
|
||||
|
||||
Sandbox facts (from source): allowed imports are `json`, `math`,
|
||||
`datetime`, `copy`, `re`, `time` only; `open()` is restricted to
|
||||
project-contained paths; scripts run inside an editor **transaction scope**
|
||||
(undo-friendly); it is tool orchestration, NOT general Python — arbitrary
|
||||
`unreal.*` calls are not the contract. Data returns via `run()`'s dict.
|
||||
|
||||
Use it whenever a recipe loop exceeds ~5 homogeneous calls; keep one-off
|
||||
edits as plain `call_tool`.
|
||||
|
||||
## Project Agent Skills (AgentSkillToolset)
|
||||
|
||||
Projects and plugins can register **Agent Skills** — named instruction
|
||||
bundles for project-specific conventions and workflows (naming schemes,
|
||||
folder layout, canonical multi-step sequences). They are NOT listed by
|
||||
`list_toolsets`; reach them through `call_tool`:
|
||||
|
||||
1. `AgentSkillToolset.ListSkills` → names + descriptions of registered
|
||||
skills.
|
||||
2. If one matches the task, `AgentSkillToolset.GetSkills` on it → full
|
||||
instructions, then FOLLOW THEM — a project skill exists precisely
|
||||
because the project's way differs from the obvious way, and it takes
|
||||
precedence over this skill's generic defaults.
|
||||
|
||||
Check at the start of unfamiliar work in any project, not just once ever.
|
||||
|
||||
## Seeing your work: screenshots and captures
|
||||
|
||||
An agent that can't see the viewport is flying blind. In order of preference:
|
||||
|
||||
1. **`EditorAppToolset.CaptureViewport`** (confirmed shipped) — returns the
|
||||
image through MCP as base64 PNG with camera metadata; supports capturing
|
||||
from an arbitrary transform without disturbing the user's viewport, and
|
||||
an optional annotation overlay (world-space meter grid + actor callouts)
|
||||
for spatial-placement work. This is the default verification tool.
|
||||
2. **Console `HighResShot` via any console/exec tool** when you need
|
||||
resolutions beyond the viewport: `HighResShot 3840x2160` writes to
|
||||
`<Project>/Saved/Screenshots/<Platform>/` on the EDITOR host's
|
||||
filesystem; read the file back (same machine) with `vision_analyze`.
|
||||
3. **Custom toolset escape hatch** for anything else (e.g. camera-actor
|
||||
framed captures with MRQ-quality settings).
|
||||
|
||||
Always `vision_analyze` the capture and art-direct against the brief before
|
||||
declaring a milestone done.
|
||||
|
||||
## Plugin configuration reference
|
||||
|
||||
Editor Preferences > General > Model Context Protocol:
|
||||
|
||||
| Property | Default | Notes |
|
||||
|---|---|---|
|
||||
| Auto Start Server | `false` | Turn on for frictionless sessions |
|
||||
| Server Port Number | `8000` | Change on conflict; mirror in Hermes config url |
|
||||
| Server URL Path | `/mcp` | Same |
|
||||
| Enable Tool Search | `true` | Keep on (see above) |
|
||||
|
||||
Console commands (editor console, backtick):
|
||||
|
||||
| Command | Effect |
|
||||
|---|---|
|
||||
| `ModelContextProtocol.StartServer [port]` | Start server (optional port override) |
|
||||
| `ModelContextProtocol.StopServer` | Stop server, close all sessions |
|
||||
| `ModelContextProtocol.RefreshTools` | Re-poll toolset providers — run after authoring/hot-reload/Game-Feature activation |
|
||||
| `ModelContextProtocol.GenerateClientConfig <Client\|All>` | Write client config files (ClaudeCode/Cursor/VSCode/Gemini/Codex) — NOT used for Hermes |
|
||||
|
||||
Command-line flags for launching the editor pre-configured:
|
||||
`-ModelContextProtocolStartServer` (force start regardless of preference),
|
||||
`-ModelContextProtocolPort=N`.
|
||||
|
||||
Console variables:
|
||||
|
||||
| CVar | Default | Notes |
|
||||
|---|---|---|
|
||||
| `ModelContextProtocol.WrapPODToolResultsInObject` | `true` | Primitive results wrapped as `{"result": ...}` |
|
||||
| `ModelContextProtocol.AudioResultOggFormat` | `false` | OGG instead of WAV for audio results |
|
||||
| `ModelContextProtocol.ProgressIntervalSeconds` | `1.0` | Min interval between progress notifications |
|
||||
| `ModelContextProtocol.PaginationPageSize` | `0` | 0 = no pagination of list results |
|
||||
| `ModelContextProtocol.EnableAnalytics` | `true` | Epic telemetry gate |
|
||||
|
||||
## Debugging the connection
|
||||
|
||||
- **Output Log** at editor startup logs bind address/port/path — first stop
|
||||
when the server seems absent. Port-in-use and missing-dependency failures
|
||||
surface here.
|
||||
- **Log verbosity:** `Log LogModelContextProtocol Verbose` in the editor
|
||||
console.
|
||||
- **MCP Inspector** (`npx @modelcontextprotocol/inspector`, point at
|
||||
`http://127.0.0.1:8000/mcp`, transport "Streamable HTTP") lists every
|
||||
advertised tool with schemas and offers form-based invocation — isolates
|
||||
"server broken" from "agent calling it wrong".
|
||||
- **After Live Coding / authoring:** connected clients can hold stale
|
||||
schemas. `ModelContextProtocol.RefreshTools`, then reconnect (new Hermes
|
||||
session) if schemas still look stale.
|
||||
|
||||
## Extending the surface: custom toolsets
|
||||
|
||||
When shipped tools don't cover an operation, the supported path is authoring
|
||||
a project toolset — NOT trying to smuggle arbitrary code through unrelated
|
||||
tools. Python toolsets are first-class and hot-loadable, so prefer them.
|
||||
|
||||
### Python toolset (recommended)
|
||||
|
||||
Any enabled plugin's `Content/Python/` directory (or the project's) can hold
|
||||
toolset modules; the registry discovers them at startup. Shape (mirrors
|
||||
Epic's shipped `ActorTools`):
|
||||
|
||||
```python
|
||||
import unreal
|
||||
import toolset_registry
|
||||
|
||||
@unreal.uclass()
|
||||
class MySceneTools(unreal.ToolsetDefinition):
|
||||
"""One-line toolset description — surfaces to the agent in list_toolsets."""
|
||||
|
||||
@toolset_registry.tool_call
|
||||
@staticmethod
|
||||
def take_viewport_screenshot(filename: str, width: int, height: int) -> str:
|
||||
"""Capture the active viewport to Saved/Screenshots.
|
||||
|
||||
Args:
|
||||
filename: Base filename without extension.
|
||||
width: Output width in pixels.
|
||||
height: Output height in pixels.
|
||||
|
||||
Returns:
|
||||
Absolute path the screenshot will be written to.
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
Conventions that matter (they generate the schema the agent sees):
|
||||
|
||||
- `@unreal.uclass()` on the class; inherit `unreal.ToolsetDefinition`.
|
||||
- Class docstring = toolset description; write it for an agent audience.
|
||||
- Each advertised function: `@toolset_registry.tool_call` + `@staticmethod`.
|
||||
Functions without the decorator stay private.
|
||||
- Type hints (`str`, `bool`, `list[str]`, `unreal.Actor`, dataclasses) drive
|
||||
the JSON Schema; Google-style docstrings (`Args:`/`Returns:`) become the
|
||||
parameter descriptions. Write them with API-surface care.
|
||||
- Small, single-responsibility tools with structured return types beat
|
||||
mega-tools returning prose. Data leaves the tool via its RETURN VALUE —
|
||||
`print()`/stdout go to the UE log, not back over MCP.
|
||||
|
||||
After authoring: `ModelContextProtocol.RefreshTools` in the editor console,
|
||||
then re-`list_toolsets` from Hermes. Users on Claude Code can scaffold with
|
||||
the `create-toolset` skill from Epic's `unreal-mcp` plugin pack; the
|
||||
conventions above still apply.
|
||||
|
||||
### C++ toolset
|
||||
|
||||
Derive from `UToolsetDefinition`, mark the class `UCLASS(BlueprintType,
|
||||
Hidden)`, expose static `UFUNCTION(meta = (AICallable))` methods; doc
|
||||
comments reflect into schemas. Use only when Python can't reach the API,
|
||||
when reflected `USTRUCT` signatures are needed, or when the Python boundary
|
||||
cost matters. Exclude a function with `meta = (AIIgnore)`. Live Coding
|
||||
propagates edited function bodies, but NEW `UFUNCTION`s require a full
|
||||
editor restart. There is also a direct-registration path
|
||||
(`IModelContextProtocolTool` + `IModelContextProtocolModule::AddTool()`) for
|
||||
runtime-shaped tools; caller owns deregistration.
|
||||
|
||||
## Runtime and cooked builds
|
||||
|
||||
The server is editor-hosted by default but not editor-only: runtime modules
|
||||
can host it in cooked builds via `IModelContextProtocolModule::StartServer()`.
|
||||
The Toolset Registry adapter (and the three tool-search meta-tools) are
|
||||
editor-only, though — cooked-build tools must be registered explicitly
|
||||
through `AddTool()` and are advertised eagerly. MCP Resources and Prompts are
|
||||
not advertised by any shipping toolset.
|
||||
|
||||
## Known limitations (5.8, experimental)
|
||||
|
||||
- HTTP + SSE transports only; loopback-only listener; non-loopback `Origin`
|
||||
headers rejected; no auth layer. Not safe beyond the local machine.
|
||||
- Serial game-thread execution: overlapping calls unsupported; editor UI
|
||||
blocks during each call.
|
||||
- Feature-incomplete by Epic's own labeling; APIs and data formats subject
|
||||
to change without notice.
|
||||
- Live Coding does not propagate new `UFUNCTION` declarations.
|
||||
Reference in New Issue
Block a user