Skip to content

LuckyEngine merge: robustness, placements, meshoptimizer LODs, offline Help - #36

Open
sheazywi wants to merge 30 commits into
devfrom
feature/lucky-merge
Open

sheazywi wants to merge 30 commits into
devfrom
feature/lucky-merge

Conversation

@sheazywi

Copy link
Copy Markdown
Member

What

Brings the transferable methods from the LuckyEngine study into Lux (docs/LUCKY_MERGE_PLAN.md, phases 1-8), fixes two bugs found afterwards, and adds docs/NEXT_UP_BACKLOG.md, the full open list for what comes next.

Area Change
Render command queue Grows by 4 MiB pages instead of writing past a fixed 10 MiB arena; payloads aligned for their own callable; usage and peak in the Profiler panel
Imported meshes A mesh placed by several nodes keeps every placement (one submesh per placement, geometry and LODs shared); mesh bounds cover all placements; explicit submesh lists include the placements of what they name
Renderer2D Per-chunk texture/font tables: any number of textures and fonts per scene. Tables are bound on the render thread in command order, removing a main-thread write to material inputs the render thread reads
Font atlas cache Keyed by font contents and settings, validated on read, written atomically
WeakRef Control block + Lock() returning a Ref; no raw dereference; a new object at a freed address is never mistaken for the old one. The global live-address registry is gone
LODs meshoptimizer 1.1 (vendored, MIT), attribute-aware with locked borders, instead of position-grid averaging
Help Offline manual (Editor/Resources/Docs, 17 pages) in a Help panel (F1, Help → Manual), ImGuiEx::RenderMarkdown, a ? button on component sections. New projects get a buildable .csproj (before, a new project could not build scripts)
Runtime packs Mesh entries parsed with bounds checks and validated before the MeshSource is created; a failed async runtime load is final instead of pending forever

Why

The research found concrete failures: an unchecked command arena, instanced glTF/FBX meshes showing one copy, a 2D renderer that writes past its 32 texture slots, stale/corrupt font caches read blindly, a WeakRef that can accept a different object at a reused address, LODs that ignored their targets and smeared shading, untrusted pack data parsed with trusting counts, and no in-editor help.

Verification

  • Release Editor and Lux-Runtime build; Debug Editor build.
  • Golden images (Benchmark + RenderFeatures, all feature variants): with LODs off the branch is pixel-identical to dev. With LODs on, error vs full detail is 8.9% → 2.9% of pixels on Benchmark (mean 1.12 → 0.30), 0.21% → 0.03% on RenderFeatures; GPU 1.87 → 1.81 ms (single run). The new images were accepted as the reference (lm8). No validation messages.
  • Command queue: standalone stress (35 MB in one frame, 64-byte-aligned captures, 5 MiB command, order kept).
  • WeakRef: ASan stress, 300k last-release vs Lock() races on 3 threads, no use-after-free; 111 forced address reuses rejected.
  • Node placements: Assimp with Lux's import flags turns tests/meshes/SharedMeshNodes.gltf (1 mesh, 3 nodes) into 3 placements.
  • Runtime mesh reader: headless harness against Core.lib — valid entry loads, 12 corruption cases rejected, 2,000 random corruptions never crash.
  • Manual: 0 broken links; all 7 C# snippets compile against ScriptCore; the generated .csproj builds with dotnet.
  • Not yet driven in the editor: the Help panel UI, the tutorial on a new project, >31 textures in one scene, a corrupt font cache, the 2.1/2.2 fixes at runtime. These are section 1 of docs/NEXT_UP_BACKLOG.md.

Regeneration

Required: new files (EditorDocs, ComponentDocs.h, ImGuiMarkdown, HelpPanel, Core/vendor/meshoptimizer) and Core/premake5.lua. Run scripts\Win-GenProjects.bat.

Docs

.claude/docs/Architecture-LuxEngine.md (placements, async failure state, Help/manual, new-project .csproj, runtime pack validation, WeakRef), Conventions.md (WeakRef rules, Markdown/DocsButton helpers), Threading.md (command queue capacity). Plans: docs/LUCKY_MERGE_PLAN.md, docs/NEXT_UP_BACKLOG.md; research in docs/lucky-research/.

Consider-tier, left as is

  • Mirrored (negative-scale) placements may render and collide inside-out (backlog 2.4).
  • meshoptimizer LODs add import time (Sponza +170 ms); a geometry cache is backlog 3.4.

…E_PLAN.md)

- Render command queue grows by 4 MiB pages instead of overrunning a fixed
  10 MiB arena; payloads aligned per callable; usage in the Profiler panel.
- Imported meshes keep every node placement: a mesh placed by several nodes
  gets one submesh per placement sharing its geometry and LODs; mesh bounds
  cover all placements (AABB::Transformed). Fixture tests/meshes.
- Renderer2D: per-chunk texture/font slot tables, so a scene can use any
  number of textures and fonts; tables bound on the render thread.
- Font atlas cache keyed by font contents and settings, validated on read,
  written atomically.
- WeakRef gets a control block and Lock(); no raw dereference. The global
  live-address registry is removed.
- LODs built with meshoptimizer 1.1 (vendored, MIT), attribute-aware with
  locked borders, instead of position-grid averaging. Goldens with LODs off
  are unchanged; LOD error vs full detail on Benchmark 8.9% -> 2.9% of pixels.
  New golden reference: lm8.
- Offline manual in Editor/Resources/Docs with a Help panel (F1), Markdown
  rendering (ImGuiEx::RenderMarkdown) and a help button on component
  sections. New projects get a buildable C# project file.
- Runtime asset-pack meshes are parsed with bounds checks and validated
  before the MeshSource is created.
- golden_run.py --options and a noMeshLODs variant.

New files: regenerate projects (scripts\Win-GenProjects.bat).
…e placements

- RuntimeAssetSystem reports queued loads that fail back to the manager
  (RetrieveReadyAssets), which drops them from the pending set and keeps
  them failed: placeholder kept, no re-queue every frame, retried only by
  ReloadData / ReloadDataAsync. The failure log is now tagged.
- Mesh / StaticMesh::SetSubmeshes: an explicit submesh selection also
  includes the placements the importer appends for each selected submesh,
  so selections saved before placements existed show every instance.
- docs/NEXT_UP_BACKLOG.md: the open list after the Lucky merge (2.1 and
  2.2 marked fixed).
Copilot AI balanced review requested due to automatic review settings October 26, 2026 02:55
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5c1f8da7-54aa-42c5-924e-098a26615618

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

…y, sample intermediates

- Mirrored (negative-determinant) imported placements index a reversed-winding copy of their
  range, so they render and collide right side out; explicit submesh lists match placements by
  vertex range.
- C# Entity.Destroy() defers through SubmitToDestroyEntity instead of destroying inside the
  script update loop; manual updated.
- The sample project's C# intermediates move from Assets/Scripts to Cache/ScriptBuild, like new
  projects, and their asset registry entries are removed.
- Remove the unregistered RenderStatsPanel and the unused MeshSource::GetRootNode().
- Backlog: section 1 results (Linux compile-only, start-up timings, headless import check) and
  new item 2.8 (entity-level negative scale).
The existing workflow now has a version job (LUX_VERSION_SERIES from Version.h), zips the
Windows and Linux release builds with the sample project, and on pushes to dev publishes one
v2026.1-<hash> pre-release with generated notes, keeping the 10 newest. The build writes the
gitignored BuildVersion.h; --version prints the full version. Also ignores graphify-out/.
… the shader

2.8: an entity with a negative-determinant transform draws with mirrored pipeline variants
(front face flipped) through pre-depth, meshlet, G-buffer and transparent passes, its collider
keeps the signed scale with swapped winding, and DecomposeTransform puts the reflection in
scale.x instead of asserting. Adds the Mirroring sample scene and RF_TwoSided material.

2.9: pipelines declare only the vertex attributes the vertex shader reads (parsed from the
SPIR-V entry point), which removes the Debug validation warning on depth-only pipelines.
…cters, prefab structure

3.4  Generated LODs are cached in <project>/Cache/Geometry (.lgeo, keyed by geometry contents,
     LOD settings and meshoptimizer version; validated, written atomically). Sponza 290 -> 138 ms.
3.5  StreamReader bounds every length and count by the bytes left; the pack index rejects bad
     ranges and counts; no VERIFY on a damaged pack; textures are size-checked before upload.
3.1  C# OnFixedUpdate(fixedDt) runs before each fixed physics step (one Jolt step at a time).
3.2  The character controller is a real JPH::CharacterVirtual, updated in Jolt's order (it used
     to integrate gravity into the transform with no collision). Moving a kinematic body that had
     gone to sleep no longer deadlocks the physics step.
3.10 Prefab instances follow the prefab's hierarchy (added, removed, re-parented children and
     sibling order) while user deletions and moves are recorded and kept; variants follow their
     base; old instances are converted and relinked. Dropped-in instances now link to their prefab
     and instantiated children keep their local transforms.
     C# OnCollisionBegin/End now fire for 3D bodies and character controllers (they only ever
     fired for Box2D), one per entity pair.
…rs, prefab restore

3.4  The asset pack carries the LODs the editor generated (geometry cache bytes after the mesh's
     index buffer, MeshFlags::HasBakedLODs); the player decodes them instead of generating
     (Sponza 16.7 ms vs 439 ms, identical output; damaged bytes regenerate).
3.5  StreamReader::ReadRaw no longer asserts on a short read; the runtime project reader bounds its
     tag count and refuses truncated files. Prefabs are packed, script asset fields are listed in a
     scene's assets, and C# Entity.Instantiate(prefab[, position, rotation]) spawns them: physics
     bodies and character controllers are created once the instance is whole (also fixing prefabs
     dropped in during Simulate), then scripts start with OnCreate.
3.2  C# OnTriggerBegin/End for 3D triggers; triggers see kinematic bodies, so character controllers
     set them off. Trigger events are kept out of physics audio.
3.10 Variants follow the base's sibling order. Converting a scene saved before prefab edits were
     recorded logs what it took as deleted, and the Prefab section offers Restore.
…gizmos, profiling readiness

Sun: a directional light shines along its entity's forward, but shading read that vector as
"toward the light" while the cascades read it as the travel direction. Every sample scene's sun
sat below the horizon (no direct sunlight anywhere) and above-horizon suns got inverted
shadows. Scene now passes the renderer the direction toward the light, and CalculateCascades
puts the eye on the sun side. Every golden image changes (scenes are sun-lit now).

3.8  Directional shadow pipelines clamp depth and the cascade caster frustums drop their near
     plane (CPU mask and GPU culling): off-screen casters far sunward still shadow the view.
     No measured cost. New LowSun scene (60 m pillars behind the camera, 10-degree sun).
3.7  Cluster light culling transforms lights once per 64-light chunk per workgroup (0.105 ->
     0.089 ms on 656 lights); list overflow is read back into renderer stats, the Renderer
     Debugger and golden perf.json, with a one-time warning. New ManyLights scene.
3.9, 3.18, 3.3: measured, not worth doing (numbers in the backlog).
3.11 C# Gizmos (DrawLine/DrawRay/DrawWireBox/DrawWireSphere), kept per update phase so lines
     hold still; drawn through the render packet in Play, not in Dist.
3.6  Golden perf: warm-up after the readback, frames with restarted streaming dropped,
     incomplete runs flagged, median/p95/p99/stddev, pass occurrence, build/GPU metadata;
     thumbnails paused during captures; golden_run.py compares and flags non-comparable pairs.
- tools/ScriptApiDocs: a developer-only .NET 9 tool that reads ScriptCore.dll and
  its XML docs and writes Editor/Resources/Docs/api (a page per public type, an
  index by category). --check writes nothing and fails on stale or extra pages,
  public members without a <summary>, or docs for members that are no longer
  public. CI runs the check after the Windows Release build.
- ScriptCore emits ScriptCore.xml; every public type and member now has a doc
  comment, written against ScriptGlue and the engine code behind each call.
- Help panel: "API Reference" section, listing only its index unless an API page
  is open. Linked from the manual's start page and the C# Scripts page.
- Backlog 3.14 audited: nothing queues main-thread work that can outlive its
  owner yet, so no owner-tagged events for now.
…uthoring, tutorials (3.17)

New pages, each written from the editor source:
- Materials and Material Editor: surface model, opaque/cutout/transparent,
  texture maps and ORM channels, every editor section, save/revert/undo.
- Game View: what it renders, Play behaviour, accessibility overlay.
- Physics Layers: Project Settings layers, Layer ID, the either-side rule.
- Authoring Audio in FMOD Studio: project and bank settings, bank building,
  and the parameters the engine drives (Occlusion, ReverbSend, footsteps,
  impacts, music State/Intensity/Layer_).
- Prefabs, Step by Step and Your First 2D Game tutorials.
- Exporting a Game gains its settings table and preflight list.

Linked from the start page and the related component pages; every link in the
manual resolves. CharacterControllerComponent.Move's doc now matches the
physics page (call it every frame with speed * ts).
EditorToolHost runs named commands with JSON parameters for scripted or
external tools. Any thread submits into a bounded queue (256) and gets a
future with a typed outcome (completed, rejected, cancelled, failed), a message
and a JSON result. EditorLayer runs the queue on the main thread right after
the undo poll, only while no user edit is pending or active, against the scene
the user sees. Requests are checked when they run: a request naming an older
scene generation is rejected, and edits are rejected while the scene runs.
Each completed edit is committed at once as its own undo step.

Built-in commands: tools.list, scene.info, entity.list/get/create/delete/
rename/reparent, component.add/set/remove. Component edits are generic: the
entity is serialized, its YAML changed and patched back through the same path
undo uses, then serialized again so an unknown component or field is rejected
and rolled back. Delete and reparent keep the prefab bookkeeping the Hierarchy
does. Entity IDs travel as strings, since a uint64 does not fit a JSON number.

Self-tests run from the Renderer Debugger. A headless harness also passed them,
200 concurrent submits from four threads (200 entities, 200 undo steps) and
cancellation of queued requests at shutdown.
Lux-Watchdog.exe (Watchdog/, Windows only, no engine dependencies, built
next to Editor.exe) watches the editor from outside. The editor starts it from
EntryPoint (LUX_CRASH_WATCHDOG) and shares a small mapping and two events
with it.

- Unhandled exceptions, failed asserts, abort/terminate, pure virtual calls and
  invalid CRT parameters: the editor's filter signals the watchdog, which
  writes the minidump from outside; then the logs are flushed (helper thread,
  2 s limit) and the process ends.
- Hangs: Application::Run advances a heartbeat each iteration; with no
  progress for 30 s the watchdog writes a quiet hang dump.
- Any exit without MarkCleanExit: a report folder under the per-user
  CrashReports (dump, report.txt with the exception code and module+offset or
  the exit code, the session's logs) and one dialog offering to open it.
  Newest 10 kept.

Skipped under a debugger and when LUX_NO_WATCHDOG is set (the golden runner
sets it, since it kills editors that time out); LUX_WATCHDOG_QUIET drops the
dialog. Tested with a harness in every mode: clean exit (no report), exit
code, access violation, assert, uncaught C++ exception, abort, hang, and a hang
that recovers. The crash dump symbolizes to the faulting source line.
Fail-fast and stack overflow exits get a report without a dump.
…nload (3.12)

Measured first, with a headless loop that switches the active project
(asset manager, audio, script assemblies) and runs a scene with a script and
a rigid body each time, tracking memory, handles and threads.

- Every Play session leaked about 143 MiB: vaWorldDestroy does not free what
  a world's updates allocated (raw-SDK repro: create, 10 updates, documented
  drain and teardown, every call VA_SUCCESS). RaytracedAudioScene now drains
  and empties its world on Stop and returns it to a process-level pool; the
  next Start reuses it (pending shutdown cleared, bounds and materials
  re-applied). A world still holding emitters after teardown is destroyed
  instead. Application releases the pool at exit.
- Unloaded script contexts were only freed by some later GC (a few MiB and
  mapped files per project switch or reload). Unloading is now followed by
  collect, wait for finalizers, collect.

After: 12 project switches with scenes, no growth in memory, handles or
threads; no crash in 36 switches. The .NET host was already per process and
scripts per project; physics keeps no per-project state, so no restructuring
was needed beyond this.
clang with libstdc++ 15 rejects std::vector<std::pair<std::string, YamlTree>>
inside YamlTree itself: the pair needs the complete type. Keys and values now
live in two parallel vectors (a vector of an incomplete type is allowed since
C++17). Behaviour unchanged: the tool host self-tests and the threaded
submit test pass.
…ct compile into the surface shaders, graph parameters (4.4 phases 7, 9-11)
…, docs (4.4 phases 8, 12 partly); backlog 4.15 lean export
…graph parameters in the entity inspector; comment resizes persist
…ed parameter layout for the run time (4.4 phase 12)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants