1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
|
# RML compositing — Phase 1 architecture (the design doc)
> **Status: ACTIVE design, gated only by Phase-2 implementation.** Phase 0 (the
> spike) **PASSED GO** on the real CF-AX3 GPU class — see
> `reports/rml-compositing-spike.md`. This doc settles the architecture the
> direction note (`notes/rml-compositing.md`) deferred to Phase 1. Phase 2 then
> implements it wave-by-wave, behind a flag, per `ORCHESTRATOR.md`.
## 0. Inputs that are already settled (do not relitigate)
- **GO**: live zero-copy import, RCSS 3D transform on live pixels, input accurate
through the transform, per-subsurface surface trees, idle dirty-gate, and the
FBO→dmabuf→`wlr_scene_buffer` present path all proven on Haswell+crocus.
- **Perf reality (Phase 0 Stage-0 measurement)**: ~30fps under a 4-window load,
**fill-bound** — the whole-output composite is ~10–15ms GPU, CPU work ~2ms,
present dominated by the fence wait. Damage limiting is the recovery lever.
- **Surface-tree answer (spike §3)**: **per-subsurface elements** by default, with
a **per-window render-to-texture (RTT) hook** for the one case that needs it
(a single effect that must treat a whole window tree as one flat surface —
genie warp, cross-seam blur).
- **Contract decision (user)**: **RCSS is the single source of truth for ALL
layout and animation.** C++/extensions DRIVE the document (what exists, which
data, which classes) through a **typed** substrate API, but never own placement
geometry imperatively. Tiling, stage-manager, floating, effects = RCSS.
- **Naming (GLOSSARY)**: *RML compositing* (the approach), *surface element* (an
RML element backed by a live client surface's shared texture).
## 1. The contract principle (Option 2), reconciled with the constitution
AGENTS.md forbids **string-keyed cross-extension dependency discovery** ("a
missing dependency must be a compile/link error"). RCSS-as-layout does **not**
violate this: that rule governs how *units find each other*, not the styling
vocabulary. So:
- **Units couple through typed C++ symbols** (service handles, hook descriptors,
the substrate API) — unchanged. A missing `ext-xdg-shell` is still a link error.
- **Within a surface, layout/animation is RCSS.** C++ pushes *data* (typed
`bind_*`/`bind_list_*`) and *intent* (set a class, dirty a binding); the
document decides geometry and tweens. This is already how `UiSurface` works
(`notes`/the `ui.hpp` contract) — Phase 1 does not invent a paradigm, it adds a
**live** surface primitive and **input-back** to the existing one.
Litmus: an extension may never read another extension's state by string name, and
may never compute a window's on-screen rectangle and command "draw it there." It
provides the window list + per-window data; RCSS lays them out and animates.
## 2. What already exists and is REUSED verbatim
The `kernel` `UiSubstrate`/`UiSurface` contract (`packages/kernel/include/unbox/
kernel/ui.hpp`) already provides everything Option 2 needs **except live windows**:
- `create_surface(UiSurfaceSpec)` → a `UiSurface` = one RML document = one
composited node; per-pixel alpha; `SceneLayer`.
- Typed data bindings: `bind_int/double/bool/string`, **`bind_list` +
`bind_list_string/int/...` + `bind_list_event`** (the list pattern), `dirty()`.
- Interaction: `bind_event`, **`bind_drag`** (captured drag stream in surface-local
px), `on_touch_mode_changed`.
- **`transition_timing(element_id, property)`** — read RCSS-authored
duration/delay/easing from C++ so animators reuse hot-reloadable RCSS values.
- **`Preview`** = a FROZEN toplevel snapshot imported as a texture, shown via
`<img src=source_uri()>` in any ui surface. `create_preview(wlr_scene_tree*)`.
- Dev **hot-reload** of RML/RCSS, error-isolated.
**Phase 1 = make `Preview` LIVE + route input back into it.** That is the whole
new mechanism; the layout/animation/contract machinery is already shipped.
## 3. New kernel primitive: the live surface element
A live analogue of `Preview`. Proposed contract (in `ui.hpp`, kernel-owned):
class SurfaceElement { // GLOSSARY: "surface element"
public:
// The <img src> URI resolving to this surface's LIVE shared texture inside
// any ui surface of this substrate (e.g. "unbox-surface://7"). Stable for life.
virtual auto source_uri() const -> std::string = 0;
virtual auto width() const -> int = 0; // current surface px (tracks commits)
virtual auto height() const -> int = 0;
// NO refresh(): unlike Preview, this updates itself every client commit
// (seq-gated re-import) and drives the client's frame callbacks.
virtual ~SurfaceElement() = default;
};
// On UiSubstrate:
virtual auto create_surface_element(wlr_surface* client) // a BORROW
-> std::unique_ptr<SurfaceElement> = 0;
Semantics (all proven in the spike, generalized from `spike_gl.hpp`):
- **Zero-copy, seq-gated**: re-imports the client's current buffer only when
`wlr_surface_state.seq` advances (pool-reuse-proof); double-buffered
`wlr_buffer_lock`/unlock; idle client ⇒ zero work.
- **Drives the client loop**: the substrate sends `wl_surface` frame callbacks
each composited frame (a live element, unlike `Preview`, is responsible for the
client's progress — the spike's "stuck-frame" fix).
- **Surface tree** (see §6): one `create_surface_element(toplevel root)` manages
the toplevel + its subsurfaces + popups as **child elements**, each its own live
texture at its tree offset.
- **Lifetime**: owned by the contributing extension via `unique_ptr`; destroying
it drops the import + frame-callback duty. The `wlr_surface*` is a borrow valid
until the owner drops the element (extensions already track map/unmap).
Wallpaper/layer-shell surfaces use the **same** `create_surface_element` (spike
criterion 5).
## 4. The compositor document + window-layout model (RCSS-driven)
This is the heart of Option 2. Windows are not per-window scene nodes; they are
**surface elements inside a ui surface document**, laid out by RCSS.
- A window-management extension (today `ext-xdg-shell`; later a tiling/stage
extension) owns **one ui surface** at `SceneLayer` for app content (call it the
*window field*). It does NOT compute geometry.
- It feeds windows through the **existing list binding**: `bind_list("wins", …)`
with per-row fields — crucially a `live_uri` string field returning each
window's `SurfaceElement::source_uri()`, plus whatever the RCSS layout keys off
(focused bool, app_id string, a layout-slot int/percent, z-order, etc.).
- The RML authors the layout:
<div data-model="wm">
<div class="field tiling"> <!-- class chosen by C++ intent -->
<div data-for="w : wins" class="win"
data-class-focused="w.focused"
style="--slot: {{ w.slot }};">
<img src="{{ w.live_uri }}"/>
</div>
</div>
</div>
Tiling = RCSS flex/grid keyed on `--slot`; stage-manager = the same list under a
`.stage` class with `transform`/perspective per card; floating = absolutely
positioned from bound `x/y`. **Switching layout = swapping a class / changing
bound data**, animated by RCSS `transition` — the user's "everything is laid out
and animated in RCSS."
- **Animation timing** comes from RCSS; C++ that must coordinate (e.g. a gesture)
reads it via `transition_timing()` (already shipped) and drives progress with
bound values — never hand-rolled geometry.
This makes tiling/stage/effects *policies expressed as RML+RCSS + a window list*,
exactly the constitution's "kernel names no feature."
## 5. Unified input (pick → wl_seat), folded into the substrate
The substrate already routes `data-event*`/`bind_drag` for ui surfaces. Phase 1
adds **client input-back** for surface elements:
- On pointer/touch, the substrate feeds the screen point to `Context::Process*`
(transform-aware pick). If the hovered element is a surface element, it maps the
point to surface-local via **`Element::Project()`** (the spike's fix — projects
through the element's real 3D transform, no-op when untransformed) and forwards
via `wlr_seat_pointer/touch_notify_*`.
- Keyboard focus follows the focused window (the wm extension calls a focus path;
`wlr_seat_keyboard_notify_*`).
- **Cursor stays a wlr hardware plane**, never drawn in RMLUi (recompose-on-move
would be fatal).
- Implicit grab / click-to-focus stays wm-extension policy; the substrate only
provides the pick→local→seat translation as a typed primitive.
Contract sketch (kernel): a surface element created from a `wlr_surface` is
**automatically** input-routed by the substrate (it knows the element↔surface
map); the wm extension does not wire seat calls itself. This subsumes
`ext-xdg-shell`'s current pointer/touch routing.
## 6. Surface trees: per-subsurface + RTT hook
- Default: `create_surface_element(root)` builds **one child element per
subsurface/popup**, positioned at its tree offset; DOM order = composite order;
popups are not parent-clipped (own absolutely-positioned elements). This is the
spike's criterion-4 result.
- A small **"place child relative to parent's resolved box"** layout helper is
needed so a moving/transformed parent drags its children (spike §3 edge note) —
pure layout glue.
- **RTT hook** (do not build until an effect needs it): a per-element opt-in that
flattens a window's whole tree to one texture (RmlUi `SaveLayerAsTexture`) so a
tree-spanning effect transforms/filters one surface. Element-level policy, not a
global mode.
## 7. Present + performance posture
- **Present path**: reuse Phase-0 Plan A — RMLUi composites into an FBO on a
`wlr_swapchain` dmabuf, handed to a `wlr_scene_buffer`; EGL fence, no `glFinish`.
`wlr_scene` is reduced to **presenter of one full-output buffer + the cursor
plane + (later) scanout bypass**.
- **Dirty-gate (ours)**: schedule + `Render()` only on a real signal — a client
commit (wlroots), an active RCSS animation (`GetNextUpdateDelay()` finite), or an
input-driven state change. Idle ≈ no GPU. Proven in the spike.
- **Damage-limited compositing — Option B (build here, correctly).** Now that we
own the real compositor (not a throwaway), do it the production way:
1. Per-element dirt → screen-space damage region: project each changed surface's
`wl_surface` damage rect through its element transform (forward of the spike's
`project_to_screen`) → AABB → union (cap; fall back to full-frame when it
explodes); static-transform elements use AABB, animating ones are full-damage
for the animation.
2. Render damage into the swapchain with **buffer-age accumulation**
(`wlr_damage_ring` keyed on the presented buffer; repaint the union over the
buffer's age) and a **scissor** on RmlUi's draw; redraw all elements
intersecting the region in z-order (blending-correct).
3. Feed the region to **`wlr_scene_buffer_set_buffer_with_damage`** so output +
KMS partial-update benefit (battery/thermal on a 15W fanless panel).
- A **damage-debug tint** toggle (same trick as the spike's click crosshair) to
watch the reshaded region shrink and catch buffer-age staleness.
- **Fullscreen-video scanout bypass (deferred, separate from damage).** When one
opaque, untransformed, fullscreen surface has nothing composited on top, pull it
out of the RMLUi composite and hand it to `wlr_scene`/scanout directly (RMLUi
draws nothing that frame). Trigger = the fullscreen STATE. Damage limiting can't
help a full-rate video; this can. Size it by measurement; not a blocker.
## 8. Cross-unit contract changes (what Phase 2 touches)
| Unit | Change | Contract impact |
|---|---|---|
| **kernel** (`ui.hpp`, present/frames, input) | add `SurfaceElement` + `create_surface_element`; auto input-back for surface elements; damage-limited present; dirty-gate as the scheduler | NEW public surface in `ui.hpp`; present internals private |
| **ext-window-field** (NEW core unit; user decision §10.1) | owns the window-field ui surface + the window list (each row a `SurfaceElement` `live_uri` + layout data) + layout policy (tiling/stage/floating as RCSS); subscribes to `ext-xdg-shell`'s map/unmap/focus; drives focus + click-to-focus policy | NEW contract: the layout/tiling service (typed); consumes `ext-xdg-shell::Service` + kernel `SurfaceElement` |
| **ext-xdg-shell** | stop owning a `wlr_scene_tree` per toplevel for COMPOSITING; expose each toplevel's **root `wlr_surface`** so `ext-window-field` can make a `SurfaceElement`; `hide()/show()` become list membership / a hidden class, not scene-node toggles; pointer/touch routing moves to the substrate | `Toplevel::scene_tree()` **retired**; add `Toplevel::wl_surface()` (typed borrow); `geometry()` becomes the RCSS-resolved element box (read-back) — a real change-request |
| **ext-layer-shell** | layer surfaces become surface elements at the right `SceneLayer`; wallpaper via the identical path | analogous to xdg-shell |
| **ext-stage-dock** | minimize/restore re-expressed as **RCSS over the live window list** (a `.minimized` class / a dock list of live URIs); drop the frozen-`Preview` snapshot path (previews can stay live now) | consumes the new list/`SurfaceElement`; coordinates with `ext-window-field`; `Preview` may remain for thumbnails of hidden windows |
| **host-bin** (orchestrator-owned) | composition-root wiring (+ the NEW `ext-window-field` unit) + the **Phase-2 feature flag** to switch compositing path | — |
The biggest contract churn is **`ext-xdg-shell`** losing window compositing
(`scene_tree()`/`hide()`/`show()`/`geometry()` redesigned around exposing the root
`wlr_surface`) **and the new `ext-window-field` unit** owning layout. Settle both
contracts before Wave 2 fans out.
## 9. Unit/ownership map + Phase 2 wave plan
Topological, disjoint-where-possible (per `ORCHESTRATOR.md` §2). Behind a flag so
the session stays usable each wave.
1. **Wave 1 — kernel substrate.** `SurfaceElement` + `create_surface_element`
(live import, seq-gate, frame-callback duty, surface-tree children) + auto
input-back. Damage-limited present + dirty-gate scheduler. (Pure-core damage
math is doctested; glue tested on the headless backend.) *opus agent.*
2. **Wave 2 — ext-xdg-shell + ext-layer-shell** (disjoint): retire per-window
scene-tree compositing; `ext-xdg-shell` exposes `Toplevel::wl_surface()` + the
new focus/geometry contract; `ext-layer-shell` exposes its surfaces likewise.
Depends on Wave 1's contract. (These two are disjoint and can summon together.)
3. **Wave 3 — ext-window-field** (NEW): owns the window-field ui surface, the
`bind_list` of live windows, and RCSS layout (tiling/stage/floating); subscribes
to ext-xdg-shell map/unmap/focus; drives focus policy. Depends on Wave 2.
4. **Wave 4 — ext-stage-dock**: minimize/restore as RCSS over the live window
list; coordinate with ext-window-field; thumbnails via live elements or
`Preview`. Depends on Wave 3.
5. **Wave 5 — perf hardening**: damage-debug tooling, scanout bypass, real-seat
numbers; refine tiling (now an RCSS layout over the window field).
## 10. Open sub-decisions (USER — boundary calls before/within Phase 2)
1. **RESOLVED (user): a NEW `ext-window-field` / `ext-tiling` core extension owns
the window-field ui surface, the window list, and layout policy** (tiling /
stage / floating as RCSS). `ext-xdg-shell` keeps the xdg protocol and only
supplies toplevel handles + their root `wlr_surface`s; it no longer owns window
compositing. This is the new unit added in Wave 2.
2. **One window-field document vs one per output/workspace?** Recommendation: one
per output to start; workspaces = bound class/data on it.
3. **`Preview` retirement vs coexistence.** Keep `Preview` for thumbnails of
*hidden* windows (no live buffer), use live `SurfaceElement` everywhere else?
Recommendation: coexist.
4. **Flag strategy for Phase 2** (config `unbox.toml` key vs build flag) to run
old `wlr_scene` compositing and new RML compositing side-by-side during the
migration. Recommendation: `unbox.toml` runtime key.
## 11. Risks & fallback
- **Perf after damage limiting + scanout bypass still below budget on the real
panel** → fall back to `wlr_scene` compositing with transient
snapshot-through-RMLUi effects (the stage-dock `Preview` path already proves that
half). This is the row-71 reopen trigger.
- **RCSS-only layout proves unworkable for a real tiling/effects extension**
(e.g. needs imperative geometry RCSS can't express) → revisit the contract
decision with a typed-placement-service escape hatch for that one case (kept out
unless earned, per the rules discipline).
- **`ext-xdg-shell` contract churn** is the integration risk; settle its new
window-list/geometry contract with its owner-agent before Wave 2 fans out.
|
