diff options
Diffstat (limited to 'notes/rml-compositing-phase1.md')
| -rw-r--r-- | notes/rml-compositing-phase1.md | 257 |
1 files changed, 257 insertions, 0 deletions
diff --git a/notes/rml-compositing-phase1.md b/notes/rml-compositing-phase1.md new file mode 100644 index 0000000..45c8e8c --- /dev/null +++ b/notes/rml-compositing-phase1.md @@ -0,0 +1,257 @@ +# RML compositing — Phase 1 architecture (the design doc) + +> **Status: ACTIVE design, gated only by Phase-2 implementation.** Phase 0 (the +> spike) **PASSED GO** on the real CF-AX3 GPU class — see +> `reports/rml-compositing-spike.md`. This doc settles the architecture the +> direction note (`notes/rml-compositing.md`) deferred to Phase 1. Phase 2 then +> implements it wave-by-wave, behind a flag, per `ORCHESTRATOR.md`. + +## 0. Inputs that are already settled (do not relitigate) +- **GO**: live zero-copy import, RCSS 3D transform on live pixels, input accurate + through the transform, per-subsurface surface trees, idle dirty-gate, and the + FBO→dmabuf→`wlr_scene_buffer` present path all proven on Haswell+crocus. +- **Perf reality (Phase 0 Stage-0 measurement)**: ~30fps under a 4-window load, + **fill-bound** — the whole-output composite is ~10–15ms GPU, CPU work ~2ms, + present dominated by the fence wait. Damage limiting is the recovery lever. +- **Surface-tree answer (spike §3)**: **per-subsurface elements** by default, with + a **per-window render-to-texture (RTT) hook** for the one case that needs it + (a single effect that must treat a whole window tree as one flat surface — + genie warp, cross-seam blur). +- **Contract decision (user)**: **RCSS is the single source of truth for ALL + layout and animation.** C++/extensions DRIVE the document (what exists, which + data, which classes) through a **typed** substrate API, but never own placement + geometry imperatively. Tiling, stage-manager, floating, effects = RCSS. +- **Naming (GLOSSARY)**: *RML compositing* (the approach), *surface element* (an + RML element backed by a live client surface's shared texture). + +## 1. The contract principle (Option 2), reconciled with the constitution +AGENTS.md forbids **string-keyed cross-extension dependency discovery** ("a +missing dependency must be a compile/link error"). RCSS-as-layout does **not** +violate this: that rule governs how *units find each other*, not the styling +vocabulary. So: + +- **Units couple through typed C++ symbols** (service handles, hook descriptors, + the substrate API) — unchanged. A missing `ext-xdg-shell` is still a link error. +- **Within a surface, layout/animation is RCSS.** C++ pushes *data* (typed + `bind_*`/`bind_list_*`) and *intent* (set a class, dirty a binding); the + document decides geometry and tweens. This is already how `UiSurface` works + (`notes`/the `ui.hpp` contract) — Phase 1 does not invent a paradigm, it adds a + **live** surface primitive and **input-back** to the existing one. + +Litmus: an extension may never read another extension's state by string name, and +may never compute a window's on-screen rectangle and command "draw it there." It +provides the window list + per-window data; RCSS lays them out and animates. + +## 2. What already exists and is REUSED verbatim +The `kernel` `UiSubstrate`/`UiSurface` contract (`packages/kernel/include/unbox/ +kernel/ui.hpp`) already provides everything Option 2 needs **except live windows**: +- `create_surface(UiSurfaceSpec)` → a `UiSurface` = one RML document = one + composited node; per-pixel alpha; `SceneLayer`. +- Typed data bindings: `bind_int/double/bool/string`, **`bind_list` + + `bind_list_string/int/...` + `bind_list_event`** (the list pattern), `dirty()`. +- Interaction: `bind_event`, **`bind_drag`** (captured drag stream in surface-local + px), `on_touch_mode_changed`. +- **`transition_timing(element_id, property)`** — read RCSS-authored + duration/delay/easing from C++ so animators reuse hot-reloadable RCSS values. +- **`Preview`** = a FROZEN toplevel snapshot imported as a texture, shown via + `<img src=source_uri()>` in any ui surface. `create_preview(wlr_scene_tree*)`. +- Dev **hot-reload** of RML/RCSS, error-isolated. + +**Phase 1 = make `Preview` LIVE + route input back into it.** That is the whole +new mechanism; the layout/animation/contract machinery is already shipped. + +## 3. New kernel primitive: the live surface element +A live analogue of `Preview`. Proposed contract (in `ui.hpp`, kernel-owned): + + class SurfaceElement { // GLOSSARY: "surface element" + public: + // The <img src> URI resolving to this surface's LIVE shared texture inside + // any ui surface of this substrate (e.g. "unbox-surface://7"). Stable for life. + virtual auto source_uri() const -> std::string = 0; + virtual auto width() const -> int = 0; // current surface px (tracks commits) + virtual auto height() const -> int = 0; + // NO refresh(): unlike Preview, this updates itself every client commit + // (seq-gated re-import) and drives the client's frame callbacks. + virtual ~SurfaceElement() = default; + }; + + // On UiSubstrate: + virtual auto create_surface_element(wlr_surface* client) // a BORROW + -> std::unique_ptr<SurfaceElement> = 0; + +Semantics (all proven in the spike, generalized from `spike_gl.hpp`): +- **Zero-copy, seq-gated**: re-imports the client's current buffer only when + `wlr_surface_state.seq` advances (pool-reuse-proof); double-buffered + `wlr_buffer_lock`/unlock; idle client ⇒ zero work. +- **Drives the client loop**: the substrate sends `wl_surface` frame callbacks + each composited frame (a live element, unlike `Preview`, is responsible for the + client's progress — the spike's "stuck-frame" fix). +- **Surface tree** (see §6): one `create_surface_element(toplevel root)` manages + the toplevel + its subsurfaces + popups as **child elements**, each its own live + texture at its tree offset. +- **Lifetime**: owned by the contributing extension via `unique_ptr`; destroying + it drops the import + frame-callback duty. The `wlr_surface*` is a borrow valid + until the owner drops the element (extensions already track map/unmap). + +Wallpaper/layer-shell surfaces use the **same** `create_surface_element` (spike +criterion 5). + +## 4. The compositor document + window-layout model (RCSS-driven) +This is the heart of Option 2. Windows are not per-window scene nodes; they are +**surface elements inside a ui surface document**, laid out by RCSS. + +- A window-management extension (today `ext-xdg-shell`; later a tiling/stage + extension) owns **one ui surface** at `SceneLayer` for app content (call it the + *window field*). It does NOT compute geometry. +- It feeds windows through the **existing list binding**: `bind_list("wins", …)` + with per-row fields — crucially a `live_uri` string field returning each + window's `SurfaceElement::source_uri()`, plus whatever the RCSS layout keys off + (focused bool, app_id string, a layout-slot int/percent, z-order, etc.). +- The RML authors the layout: + + <div data-model="wm"> + <div class="field tiling"> <!-- class chosen by C++ intent --> + <div data-for="w : wins" class="win" + data-class-focused="w.focused" + style="--slot: {{ w.slot }};"> + <img src="{{ w.live_uri }}"/> + </div> + </div> + </div> + + Tiling = RCSS flex/grid keyed on `--slot`; stage-manager = the same list under a + `.stage` class with `transform`/perspective per card; floating = absolutely + positioned from bound `x/y`. **Switching layout = swapping a class / changing + bound data**, animated by RCSS `transition` — the user's "everything is laid out + and animated in RCSS." +- **Animation timing** comes from RCSS; C++ that must coordinate (e.g. a gesture) + reads it via `transition_timing()` (already shipped) and drives progress with + bound values — never hand-rolled geometry. + +This makes tiling/stage/effects *policies expressed as RML+RCSS + a window list*, +exactly the constitution's "kernel names no feature." + +## 5. Unified input (pick → wl_seat), folded into the substrate +The substrate already routes `data-event*`/`bind_drag` for ui surfaces. Phase 1 +adds **client input-back** for surface elements: +- On pointer/touch, the substrate feeds the screen point to `Context::Process*` + (transform-aware pick). If the hovered element is a surface element, it maps the + point to surface-local via **`Element::Project()`** (the spike's fix — projects + through the element's real 3D transform, no-op when untransformed) and forwards + via `wlr_seat_pointer/touch_notify_*`. +- Keyboard focus follows the focused window (the wm extension calls a focus path; + `wlr_seat_keyboard_notify_*`). +- **Cursor stays a wlr hardware plane**, never drawn in RMLUi (recompose-on-move + would be fatal). +- Implicit grab / click-to-focus stays wm-extension policy; the substrate only + provides the pick→local→seat translation as a typed primitive. + +Contract sketch (kernel): a surface element created from a `wlr_surface` is +**automatically** input-routed by the substrate (it knows the element↔surface +map); the wm extension does not wire seat calls itself. This subsumes +`ext-xdg-shell`'s current pointer/touch routing. + +## 6. Surface trees: per-subsurface + RTT hook +- Default: `create_surface_element(root)` builds **one child element per + subsurface/popup**, positioned at its tree offset; DOM order = composite order; + popups are not parent-clipped (own absolutely-positioned elements). This is the + spike's criterion-4 result. +- A small **"place child relative to parent's resolved box"** layout helper is + needed so a moving/transformed parent drags its children (spike §3 edge note) — + pure layout glue. +- **RTT hook** (do not build until an effect needs it): a per-element opt-in that + flattens a window's whole tree to one texture (RmlUi `SaveLayerAsTexture`) so a + tree-spanning effect transforms/filters one surface. Element-level policy, not a + global mode. + +## 7. Present + performance posture +- **Present path**: reuse Phase-0 Plan A — RMLUi composites into an FBO on a + `wlr_swapchain` dmabuf, handed to a `wlr_scene_buffer`; EGL fence, no `glFinish`. + `wlr_scene` is reduced to **presenter of one full-output buffer + the cursor + plane + (later) scanout bypass**. +- **Dirty-gate (ours)**: schedule + `Render()` only on a real signal — a client + commit (wlroots), an active RCSS animation (`GetNextUpdateDelay()` finite), or an + input-driven state change. Idle ≈ no GPU. Proven in the spike. +- **Damage-limited compositing — Option B (build here, correctly).** Now that we + own the real compositor (not a throwaway), do it the production way: + 1. Per-element dirt → screen-space damage region: project each changed surface's + `wl_surface` damage rect through its element transform (forward of the spike's + `project_to_screen`) → AABB → union (cap; fall back to full-frame when it + explodes); static-transform elements use AABB, animating ones are full-damage + for the animation. + 2. Render damage into the swapchain with **buffer-age accumulation** + (`wlr_damage_ring` keyed on the presented buffer; repaint the union over the + buffer's age) and a **scissor** on RmlUi's draw; redraw all elements + intersecting the region in z-order (blending-correct). + 3. Feed the region to **`wlr_scene_buffer_set_buffer_with_damage`** so output + + KMS partial-update benefit (battery/thermal on a 15W fanless panel). + - A **damage-debug tint** toggle (same trick as the spike's click crosshair) to + watch the reshaded region shrink and catch buffer-age staleness. +- **Fullscreen-video scanout bypass (deferred, separate from damage).** When one + opaque, untransformed, fullscreen surface has nothing composited on top, pull it + out of the RMLUi composite and hand it to `wlr_scene`/scanout directly (RMLUi + draws nothing that frame). Trigger = the fullscreen STATE. Damage limiting can't + help a full-rate video; this can. Size it by measurement; not a blocker. + +## 8. Cross-unit contract changes (what Phase 2 touches) +| Unit | Change | Contract impact | +|---|---|---| +| **kernel** (`ui.hpp`, present/frames, input) | add `SurfaceElement` + `create_surface_element`; auto input-back for surface elements; damage-limited present; dirty-gate as the scheduler | NEW public surface in `ui.hpp`; present internals private | +| **ext-window-field** (NEW core unit; user decision §10.1) | owns the window-field ui surface + the window list (each row a `SurfaceElement` `live_uri` + layout data) + layout policy (tiling/stage/floating as RCSS); subscribes to `ext-xdg-shell`'s map/unmap/focus; drives focus + click-to-focus policy | NEW contract: the layout/tiling service (typed); consumes `ext-xdg-shell::Service` + kernel `SurfaceElement` | +| **ext-xdg-shell** | stop owning a `wlr_scene_tree` per toplevel for COMPOSITING; expose each toplevel's **root `wlr_surface`** so `ext-window-field` can make a `SurfaceElement`; `hide()/show()` become list membership / a hidden class, not scene-node toggles; pointer/touch routing moves to the substrate | `Toplevel::scene_tree()` **retired**; add `Toplevel::wl_surface()` (typed borrow); `geometry()` becomes the RCSS-resolved element box (read-back) — a real change-request | +| **ext-layer-shell** | layer surfaces become surface elements at the right `SceneLayer`; wallpaper via the identical path | analogous to xdg-shell | +| **ext-stage-dock** | minimize/restore re-expressed as **RCSS over the live window list** (a `.minimized` class / a dock list of live URIs); drop the frozen-`Preview` snapshot path (previews can stay live now) | consumes the new list/`SurfaceElement`; coordinates with `ext-window-field`; `Preview` may remain for thumbnails of hidden windows | +| **host-bin** (orchestrator-owned) | composition-root wiring (+ the NEW `ext-window-field` unit) + the **Phase-2 feature flag** to switch compositing path | — | + +The biggest contract churn is **`ext-xdg-shell`** losing window compositing +(`scene_tree()`/`hide()`/`show()`/`geometry()` redesigned around exposing the root +`wlr_surface`) **and the new `ext-window-field` unit** owning layout. Settle both +contracts before Wave 2 fans out. + +## 9. Unit/ownership map + Phase 2 wave plan +Topological, disjoint-where-possible (per `ORCHESTRATOR.md` §2). Behind a flag so +the session stays usable each wave. + +1. **Wave 1 — kernel substrate.** `SurfaceElement` + `create_surface_element` + (live import, seq-gate, frame-callback duty, surface-tree children) + auto + input-back. Damage-limited present + dirty-gate scheduler. (Pure-core damage + math is doctested; glue tested on the headless backend.) *opus agent.* +2. **Wave 2 — ext-xdg-shell + ext-layer-shell** (disjoint): retire per-window + scene-tree compositing; `ext-xdg-shell` exposes `Toplevel::wl_surface()` + the + new focus/geometry contract; `ext-layer-shell` exposes its surfaces likewise. + Depends on Wave 1's contract. (These two are disjoint and can summon together.) +3. **Wave 3 — ext-window-field** (NEW): owns the window-field ui surface, the + `bind_list` of live windows, and RCSS layout (tiling/stage/floating); subscribes + to ext-xdg-shell map/unmap/focus; drives focus policy. Depends on Wave 2. +4. **Wave 4 — ext-stage-dock**: minimize/restore as RCSS over the live window + list; coordinate with ext-window-field; thumbnails via live elements or + `Preview`. Depends on Wave 3. +5. **Wave 5 — perf hardening**: damage-debug tooling, scanout bypass, real-seat + numbers; refine tiling (now an RCSS layout over the window field). + +## 10. Open sub-decisions (USER — boundary calls before/within Phase 2) +1. **RESOLVED (user): a NEW `ext-window-field` / `ext-tiling` core extension owns + the window-field ui surface, the window list, and layout policy** (tiling / + stage / floating as RCSS). `ext-xdg-shell` keeps the xdg protocol and only + supplies toplevel handles + their root `wlr_surface`s; it no longer owns window + compositing. This is the new unit added in Wave 2. +2. **One window-field document vs one per output/workspace?** Recommendation: one + per output to start; workspaces = bound class/data on it. +3. **`Preview` retirement vs coexistence.** Keep `Preview` for thumbnails of + *hidden* windows (no live buffer), use live `SurfaceElement` everywhere else? + Recommendation: coexist. +4. **Flag strategy for Phase 2** (config `unbox.toml` key vs build flag) to run + old `wlr_scene` compositing and new RML compositing side-by-side during the + migration. Recommendation: `unbox.toml` runtime key. + +## 11. Risks & fallback +- **Perf after damage limiting + scanout bypass still below budget on the real + panel** → fall back to `wlr_scene` compositing with transient + snapshot-through-RMLUi effects (the stage-dock `Preview` path already proves that + half). This is the row-71 reopen trigger. +- **RCSS-only layout proves unworkable for a real tiling/effects extension** + (e.g. needs imperative geometry RCSS can't express) → revisit the contract + decision with a typed-placement-service escape hatch for that one case (kept out + unless earned, per the rules discipline). +- **`ext-xdg-shell` contract churn** is the integration risk; settle its new + window-list/geometry contract with its owner-agent before Wave 2 fans out. |