diff options
| author | Adam Malczewski <[email protected]> | 2026-06-15 07:08:23 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-15 07:08:23 +0900 |
| commit | c38f328c8b38546a19fc4cd955e01e70b42d624d (patch) | |
| tree | 6c34654598b7aabd6f1dfbac0c2b36bb0e206100 | |
| parent | 74d75fdd8b8654446338ae03bf2a133512408ec6 (diff) | |
| download | unbox-c38f328c8b38546a19fc4cd955e01e70b42d624d.tar.gz unbox-c38f328c8b38546a19fc4cd955e01e70b42d624d.zip | |
kernel(rml-compositing W1b): surface trees + input-back + keyboard-focus
Extends the live SurfaceElement to the whole surface TREE and routes input
back to clients.
Surface trees:
- create_surface_element(root) now manages subsurfaces + xdg popups as
per-subsurface child elements (unbox-surface://N.K), each its own live
seq-gated texture at its tree offset; the substrate re-walks the live tree
each dirty tick (wlr_surface_for_each_surface + popup walk), reconciling by
wl_surface identity and dropping a node the instant it leaves the tree.
- Frame-callback duty now walks the WHOLE tree per composited frame.
- Parent-relative child placement (place_child_box, pure core): child <img>
positioned relative to the root img's resolved box, so a moving/resized
parent drags its children; popups unclipped. Caller tracks only the root.
Input-back (automatic; the wm wires no seat calls):
- pure core src/input_core.hpp (port of the spike's spike_input_core):
project_to_screen/unproject_to_local + place_child_box, doctested
(criterion-3 round-trip < 0.01px; affine exact).
- a pointer/touch pick landing on a surface-element node maps the point through
the node img's real RCSS transform via Element::Project, then box->surface-
local, and forwards to that client via wl_seat at surface-local px. Normal
RML picks still fire bind_event/bind_drag unchanged. Cursor stays a wlr plane.
- SurfaceElement::focus_keyboard() (new public method): seat keyboard-focus
MECHANISM only (focus POLICY is the window-field wave); cleared on destroy.
Tests: pure-core doctests + a headless test driving a REAL in-process client
(toplevel root + subsurface + xdg popup): tree composes as >=3 child <img>,
whole-tree frame-done, pointer enter/motion/button + touch at expected surface-
local coords (incl. a rotateY(35) transformed-element case proving Project),
keyboard enter+key. kernel suite 72c/375a green; build-asan 0 records (the
seat/per-node-import/listener lifetimes clean). Spike untouched.
| -rw-r--r-- | packages/kernel/include/unbox/kernel/server.hpp | 17 | ||||
| -rw-r--r-- | packages/kernel/include/unbox/kernel/ui.hpp | 81 | ||||
| -rw-r--r-- | packages/kernel/include/unbox/kernel/wlr.hpp | 7 | ||||
| -rw-r--r-- | packages/kernel/meson.build | 32 | ||||
| -rw-r--r-- | packages/kernel/src/input_core.hpp | 250 | ||||
| -rw-r--r-- | packages/kernel/src/server.cpp | 83 | ||||
| -rw-r--r-- | packages/kernel/src/server_impl.hpp | 6 | ||||
| -rw-r--r-- | packages/kernel/src/ui_substrate.cpp | 754 | ||||
| -rw-r--r-- | packages/kernel/src/ui_substrate.hpp | 3 | ||||
| -rw-r--r-- | packages/kernel/tests/test_kernel.cpp | 732 | ||||
| -rw-r--r-- | tasks.md | 28 |
11 files changed, 1843 insertions, 150 deletions
diff --git a/packages/kernel/include/unbox/kernel/server.hpp b/packages/kernel/include/unbox/kernel/server.hpp index fa34e02..0de5a0f 100644 --- a/packages/kernel/include/unbox/kernel/server.hpp +++ b/packages/kernel/include/unbox/kernel/server.hpp @@ -172,6 +172,23 @@ public: // is kept). Test instrumentation; single-thread only. auto ui_reload_surface() -> bool; + // ---- surface-element input-back test seams (kernel suite only) ---- + // Headless has no input devices, so these drive the substrate's input + // routing directly (the same route_* the cursor/touch handlers call) at a + // known layout point, so a test can assert a surface element forwards the + // pick to its client at surface-LOCAL coords via the seat. Single-thread only. + void ui_route_pointer_motion_for_test(double lx, double ly, unsigned int time_msec); + void ui_route_pointer_button_for_test(double lx, double ly, bool pressed, + unsigned int time_msec); + void ui_route_touch_down_for_test(int id, double lx, double ly, unsigned int time_msec); + void ui_route_touch_up_for_test(int id, unsigned int time_msec); + // Add a minimal virtual keyboard to the seat (headless has none) so the + // keyboard-focus primitive can deliver a wl_keyboard enter, then inject a key + // the seat forwards to the focused surface (the post-filter equivalent of the + // input.cpp key path). Lets the suite prove SurfaceElement::focus_keyboard. + void ui_add_test_keyboard(); + void ui_send_key_for_test(unsigned int keycode, bool pressed); + // Pin the substrate's touch-mode for tests (none = automatic). Mirrors // UiSubstrate::TouchModeOverride; lets the suite drive the state machine and // its on_touch_mode_changed notification. Test instrumentation; diff --git a/packages/kernel/include/unbox/kernel/ui.hpp b/packages/kernel/include/unbox/kernel/ui.hpp index 13ea2ae..a5e29fc 100644 --- a/packages/kernel/include/unbox/kernel/ui.hpp +++ b/packages/kernel/include/unbox/kernel/ui.hpp @@ -322,17 +322,35 @@ protected: // extension via unique_ptr; destruction drops the import, ends the // frame-callback duty, and unregisters the URI. Event-loop thread only. // +// REPRESENTS THE WHOLE SURFACE TREE. create_surface_element(root) transparently +// manages the root wl_surface AND its subsurfaces AND (for an xdg surface) its +// xdg popups as per-subsurface CHILD elements, each its own live texture at its +// tree offset; the substrate places the child <img> elements itself (relative to +// the root img's resolved box, so a moving/resized parent drags them; popups are +// NOT parent-clipped). A consumer authors only ONE <img src=source_uri()> for +// the root — it does NOT (and cannot) address the children; the element is the +// whole tree. source_uri()/width()/height() describe the ROOT surface. +// +// INPUT-BACK IS AUTOMATIC. When a pointer/touch event's transform-aware pick on +// a hosting ui surface lands on this element (root OR a subsurface/popup child), +// the substrate forwards it to that client surface at surface-LOCAL coordinates +// via the kernel's wl_seat — through the element's real RCSS transform (no extra +// wiring; the cursor stays a wlr hardware plane). Keyboard focus is opt-in via +// focus_keyboard() (the seat MECHANISM only — which window gets focus is a wm +// policy, a later wave). +// // HOW IT DIFFERS FROM Preview (which it otherwise mirrors): -// - LIVE, not frozen: it re-imports the client's current buffer every commit +// - LIVE, not frozen: it re-imports each node's current buffer every commit // (seq-gated — a static client costs ZERO work, an updating one costs one -// re-import per committed frame), so there is NO refresh(): it updates -// itself. +// re-import per committed frame per changed node), so there is NO refresh(): +// it updates itself. // - It DRIVES the client's frame callbacks: while the element exists the -// substrate sends the backing wl_surface its frame-done each composited -// frame, so the client keeps producing buffers (without this a client draws -// once and waits forever — the spike's stuck-frame fix). -// - A client commit DIRTIES the hosting ui surface(s): the next frame -// re-renders the updated texture (a static client schedules nothing). +// substrate sends the WHOLE TREE its frame-done each composited frame, so the +// client keeps producing buffers (without this a client draws once and waits +// forever — the spike's stuck-frame fix). +// - A client commit DIRTIES the hosting ui surface(s): the next frame re-walks +// the tree + re-renders the updated textures (a static client schedules +// nothing). // // LIFETIME (part of the contract — see .unbox/rules/listener-lifetime.md). The // wl_surface passed to create_surface_element is a BORROW: the substrate samples @@ -340,22 +358,34 @@ protected: // element the moment the surface unmaps or is destroyed (extensions already // track map/unmap). Sampling a surface element after its wl_surface has been // destroyed is UNDEFINED BEHAVIOUR — the substrate cannot detect a freed -// wl_surface. (Wave 1 is SINGLE-SURFACE: one element per wl_surface, no -// subsurface/popup child trees yet — that is Wave 1b.) +// wl_surface. Subsurfaces/popups are managed internally (the substrate drops a +// child node the instant its surface leaves the tree), so the caller tracks only +// the ROOT surface's lifetime. class SurfaceElement { public: virtual ~SurfaceElement() = default; SurfaceElement(const SurfaceElement&) = delete; auto operator=(const SurfaceElement&) -> SurfaceElement& = delete; - // The <img src> value resolving to this surface's LIVE texture inside any ui - // surface of this substrate (e.g. "unbox-surface://7"). Stable for life. + // The <img src> value resolving to this surface's LIVE (ROOT) texture inside + // any ui surface of this substrate (e.g. "unbox-surface://7"). Stable for + // life. The substrate derives child node URIs from this and places their + // <img> elements itself — do NOT author them. [[nodiscard]] virtual auto source_uri() const -> std::string = 0; - // The client surface's current pixel size (tracks commits). 0 until the + // The ROOT client surface's current pixel size (tracks commits). 0 until the // first buffer has been imported. [[nodiscard]] virtual auto width() const -> int = 0; [[nodiscard]] virtual auto height() const -> int = 0; + // Give this element's ROOT client surface keyboard focus on the kernel's + // seat: subsequent keys route to it (via the seat the kernel already drives). + // This is the seat MECHANISM only — focus POLICY (which window, click-to- + // focus) is a window-manager extension's job (a later wave). Idempotent. + // Calling it on an element whose root surface is unmapped is harmless. The + // substrate clears this focus automatically when the element is destroyed, so + // the seat never stays focused on a dead surface. + virtual void focus_keyboard() = 0; + // NO refresh() (unlike Preview): a surface element updates itself every // client commit (seq-gated re-import) and drives the client's frame // callbacks while it exists. @@ -411,16 +441,23 @@ public: // // CONTRAST WITH create_preview: a Preview is a FROZEN one-shot snapshot of a // scene subtree that you refresh() manually; a SurfaceElement is LIVE and - // SELF-UPDATING — it re-imports `client`'s current buffer on every commit - // (seq-gated, so a static client is free) and the substrate DRIVES - // `client`'s frame callbacks while the element exists, so the client keeps - // drawing. There is no refresh(). + // SELF-UPDATING — it re-imports `client`'s (and its subsurfaces'/popups') + // current buffers on every commit (seq-gated, so a static client is free) and + // the substrate DRIVES the whole tree's frame callbacks while the element + // exists, so the client keeps drawing. There is no refresh(). + // + // `client` is the ROOT of a surface TREE: the returned element transparently + // manages the root's subsurfaces and xdg popups as per-subsurface child + // elements (each its own live texture at its tree offset, placed by the + // substrate), and routes pointer/touch back to whichever node the pick lands + // on (see SurfaceElement above). The consumer authors only the root's + // <img src=source_uri()>. // - // LIFETIME: `client` is a BORROW. The caller guarantees it outlives the - // returned element and MUST drop the element on the surface's unmap/destroy - // (see SurfaceElement above + .unbox/rules/listener-lifetime.md). Wave 1 is - // single-surface (one element per wl_surface; subsurface/popup child trees - // are Wave 1b). + // LIFETIME: `client` (the ROOT) is a BORROW. The caller guarantees it + // outlives the returned element and MUST drop the element on the root + // surface's unmap/destroy (see SurfaceElement above + + // .unbox/rules/listener-lifetime.md). Subsurfaces/popups are managed + // internally — the caller tracks only the root. [[nodiscard]] virtual auto create_surface_element(wlr_surface* client) -> std::unique_ptr<SurfaceElement> = 0; diff --git a/packages/kernel/include/unbox/kernel/wlr.hpp b/packages/kernel/include/unbox/kernel/wlr.hpp index c890774..ca99c17 100644 --- a/packages/kernel/include/unbox/kernel/wlr.hpp +++ b/packages/kernel/include/unbox/kernel/wlr.hpp @@ -50,6 +50,13 @@ extern "C" { #include <wlr/render/wlr_renderer.h> // Producer-side interface for the spike's custom data-ptr wlr_buffer (Plan B). #include <wlr/interfaces/wlr_buffer.h> +// Producer-side interface for the kernel-suite virtual keyboard test seam +// (wlr_keyboard_init/finish + wlr_keyboard_impl): headless has no input devices, +// so the surface-element keyboard-focus test creates a minimal wlr_keyboard to +// give the seat a keyboard. Plain declarations + a 2-field struct; no header- +// inline function with a function-local static, so the `#define static` blanking +// above is inert across it (re-audited, same as wlr_buffer's interface header). +#include <wlr/interfaces/wlr_keyboard.h> #include <wlr/types/wlr_compositor.h> #include <wlr/types/wlr_cursor.h> #include <wlr/types/wlr_data_device.h> diff --git a/packages/kernel/meson.build b/packages/kernel/meson.build index bcb9364..90f9ce7 100644 --- a/packages/kernel/meson.build +++ b/packages/kernel/meson.build @@ -84,15 +84,39 @@ kernel_dep = declare_dependency( # wayland-client (TEST-ONLY): the surface-element integration test spins an # in-process Wayland CLIENT thread that connects to the headless server, creates -# a wl_surface + wl_shm buffer, and commits — the only in-process way to get a -# REAL wlr_surface with an advancing commit seq to exercise the live import. -# It is NOT a kernel-library dependency (the kernel is a compositor, never a -# client); scoped to the test executable only. +# a wl_surface + wl_shm buffer (+ a subsurface + an xdg toplevel/popup for the +# Wave-1b surface-TREE test), and commits — the only in-process way to get a +# REAL wlr_surface tree with an advancing commit seq to exercise the live import +# + input-back. It is NOT a kernel-library dependency (the kernel is a +# compositor, never a client); scoped to the test executable only. wayland_client_dep = dependency('wayland-client') +# CLIENT-side xdg-shell bindings for the Wave-1b surface-tree test (a real xdg +# popup is a tree child element). Generated from the canonical xdg-shell.xml the +# same way ext-xdg-shell's client test does (header + private-code-as-header, +# #included once by the single test TU). The SERVER side runs in a TEST +# extension via wlr_xdg_shell (kernel-private wlr.hpp), so the kernel itself +# still names no shell — xdg-shell stays a feature, provided by a test ext. +wayland_protocols_dir = dependency('wayland-protocols').get_variable('pkgdatadir') +xdg_shell_xml = wayland_protocols_dir / 'stable' / 'xdg-shell' / 'xdg-shell.xml' +xdg_shell_client_header = custom_target( + 'kernel-xdg-shell-client-header', + input: xdg_shell_xml, + output: 'xdg-shell-client-protocol.h', + command: [wayland_scanner, 'client-header', '@INPUT@', '@OUTPUT@'], +) +xdg_shell_client_code = custom_target( + 'kernel-xdg-shell-client-code', + input: xdg_shell_xml, + output: 'xdg-shell-client-protocol-code.h', + command: [wayland_scanner, 'private-code', '@INPUT@', '@OUTPUT@'], +) + kernel_test = executable( 'kernel-tests', 'tests/test_kernel.cpp', + xdg_shell_client_header, + xdg_shell_client_code, dependencies: [kernel_dep, doctest_dep, wayland_client_dep], ) test('kernel', kernel_test, suite: 'kernel') diff --git a/packages/kernel/src/input_core.hpp b/packages/kernel/src/input_core.hpp new file mode 100644 index 0000000..550d318 --- /dev/null +++ b/packages/kernel/src/input_core.hpp @@ -0,0 +1,250 @@ +#pragma once + +#include <array> +#include <cmath> +#include <optional> + +// Pure decision core for surface-element input-back — NO wlroots / GL / RMLUi +// types, so it is doctest-able with nothing running (AGENTS.md: effects at the +// edges, pure cores tested hard). This is the PRODUCTION port of the proven +// throwaway spike core (src/spike/spike_input_core.hpp, doctested at 0.000000px): +// the screen->surface-local inversion the live input-back path rides on. +// +// The live path (ui_substrate.cpp route_*) does the actual transform-aware pick +// + projection through RmlUi's own Element::Project()/GetAbsoluteOffset() (the +// same matrices RmlUi composes for `transform`), exactly as the spike's +// route_point does. THIS core proves the underlying MATH objectively: given a +// surface-local point, project it THROUGH an RCSS transform to the flat-output +// "screen" point a finger touches, then confirm the inverse recovers the +// original surface-local point to sub-pixel tolerance through a +// perspective+rotateY. If forward∘inverse is identity, the geometry the live +// wl_seat translation depends on is sound (criterion 3). +// +// Column-vector math with COLUMN-MAJOR 4x4 matrices, matching RmlUi's Matrix4f +// `transform` convention. Single-thread; no state. + +namespace unbox::kernel { + +// A column-major 4x4 matrix: m[col*4 + row]. v' = M * v. +struct Mat4 { + std::array<double, 16> m{}; + + static auto identity() -> Mat4 { + Mat4 r; + r.m = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1}; + return r; + } + + auto at(int row, int col) const -> double { return m[static_cast<std::size_t>(col) * 4 + row]; } + auto at(int row, int col) -> double& { return m[static_cast<std::size_t>(col) * 4 + row]; } +}; + +// Column-major multiply: returns A*B. +inline auto mul(const Mat4& a, const Mat4& b) -> Mat4 { + Mat4 r; + for (int col = 0; col < 4; ++col) { + for (int row = 0; row < 4; ++row) { + double s = 0.0; + for (int k = 0; k < 4; ++k) { + s += a.at(row, k) * b.at(k, col); + } + r.at(row, col) = s; + } + } + return r; +} + +// A homogeneous 4-vector. +struct Vec4 { + double x{}, y{}, z{}, w{}; +}; + +inline auto apply(const Mat4& mtx, const Vec4& v) -> Vec4 { + return Vec4{ + mtx.at(0, 0) * v.x + mtx.at(0, 1) * v.y + mtx.at(0, 2) * v.z + mtx.at(0, 3) * v.w, + mtx.at(1, 0) * v.x + mtx.at(1, 1) * v.y + mtx.at(1, 2) * v.z + mtx.at(1, 3) * v.w, + mtx.at(2, 0) * v.x + mtx.at(2, 1) * v.y + mtx.at(2, 2) * v.z + mtx.at(2, 3) * v.w, + mtx.at(3, 0) * v.x + mtx.at(3, 1) * v.y + mtx.at(3, 2) * v.z + mtx.at(3, 3) * v.w, + }; +} + +// ---- RCSS-equivalent transform builders (column-major) ---------------------- + +// CSS `perspective(d)`: m[3][2] = -1/d (column-major: at(3,2)). A point at +// model-z is foreshortened by w = 1 - z/d after the divide. +inline auto perspective(double d) -> Mat4 { + Mat4 r = Mat4::identity(); + r.at(3, 2) = -1.0 / d; + return r; +} + +// CSS `rotateY(theta)` (radians). Right-handed about +Y. +inline auto rotate_y(double theta) -> Mat4 { + Mat4 r = Mat4::identity(); + const double c = std::cos(theta); + const double s = std::sin(theta); + r.at(0, 0) = c; + r.at(0, 2) = s; + r.at(2, 0) = -s; + r.at(2, 2) = c; + return r; +} + +// CSS `translate(tx,ty)` in the XY plane. +inline auto translate(double tx, double ty) -> Mat4 { + Mat4 r = Mat4::identity(); + r.at(0, 3) = tx; + r.at(1, 3) = ty; + return r; +} + +// ---- The transform RCSS actually applies around transform-origin ------------- +// +// RCSS resolves `transform` about `transform-origin` (default 50% 50%): it +// translates the origin to (0,0), applies the listed functions, then translates +// back. This builds that full operator for a surface element of size w*h with +// the given origin, so the math matches what RmlUi computes for the element. +inline auto rcss_transform_about_origin(const Mat4& t, double origin_x, double origin_y) -> Mat4 { + return mul(translate(origin_x, origin_y), mul(t, translate(-origin_x, -origin_y))); +} + +// ---- Forward projection: surface-local (lx,ly) -> screen point --------------- +// +// Place a surface-local point on the z=0 plane, push it through the element +// transform, perform the perspective divide, and return the on-screen (sx,sy) +// where a finger/cursor would land. This is the point the live path feeds to +// RmlUi's transform-aware pick. +struct ScreenPoint { + double x{}, y{}; +}; + +inline auto project_to_screen(const Mat4& transform, double lx, double ly) -> ScreenPoint { + const Vec4 clip = apply(transform, Vec4{lx, ly, 0.0, 1.0}); + const double inv_w = (std::abs(clip.w) < 1e-12) ? 0.0 : 1.0 / clip.w; + return ScreenPoint{clip.x * inv_w, clip.y * inv_w}; +} + +// ---- Inverse: screen point -> surface-local (lx,ly) -------------------------- +// +// Inverting the projection is a ray/plane intersection (the transform is not +// affine under perspective). We invert the 4x4 transform, take the screen point +// as a clip-space ray (two points at different homogeneous depths), transform +// both back to model space, and intersect the resulting model-space ray with +// the element's own z=0 plane. The intersection's (x,y) is the surface-local +// coordinate. Returns nullopt if the transform is singular or the ray is +// parallel to the plane (degenerate edge-on view). + +// General 4x4 inverse (column-major). nullopt if |det| ~ 0. +inline auto invert(const Mat4& a) -> std::optional<Mat4> { + const std::array<double, 16>& s = a.m; + std::array<double, 16> inv{}; + + inv[0] = s[5] * s[10] * s[15] - s[5] * s[11] * s[14] - s[9] * s[6] * s[15] + + s[9] * s[7] * s[14] + s[13] * s[6] * s[11] - s[13] * s[7] * s[10]; + inv[4] = -s[4] * s[10] * s[15] + s[4] * s[11] * s[14] + s[8] * s[6] * s[15] - + s[8] * s[7] * s[14] - s[12] * s[6] * s[11] + s[12] * s[7] * s[10]; + inv[8] = s[4] * s[9] * s[15] - s[4] * s[11] * s[13] - s[8] * s[5] * s[15] + + s[8] * s[7] * s[13] + s[12] * s[5] * s[11] - s[12] * s[7] * s[9]; + inv[12] = -s[4] * s[9] * s[14] + s[4] * s[10] * s[13] + s[8] * s[5] * s[14] - + s[8] * s[6] * s[13] - s[12] * s[5] * s[10] + s[12] * s[6] * s[9]; + inv[1] = -s[1] * s[10] * s[15] + s[1] * s[11] * s[14] + s[9] * s[2] * s[15] - + s[9] * s[3] * s[14] - s[13] * s[2] * s[11] + s[13] * s[3] * s[10]; + inv[5] = s[0] * s[10] * s[15] - s[0] * s[11] * s[14] - s[8] * s[2] * s[15] + + s[8] * s[3] * s[14] + s[12] * s[2] * s[11] - s[12] * s[3] * s[10]; + inv[9] = -s[0] * s[9] * s[15] + s[0] * s[11] * s[13] + s[8] * s[1] * s[15] - + s[8] * s[3] * s[13] - s[12] * s[1] * s[11] + s[12] * s[3] * s[9]; + inv[13] = s[0] * s[9] * s[14] - s[0] * s[10] * s[13] - s[8] * s[1] * s[14] + + s[8] * s[2] * s[13] + s[12] * s[1] * s[10] - s[12] * s[2] * s[9]; + inv[2] = s[1] * s[6] * s[15] - s[1] * s[7] * s[14] - s[5] * s[2] * s[15] + + s[5] * s[3] * s[14] + s[13] * s[2] * s[7] - s[13] * s[3] * s[6]; + inv[6] = -s[0] * s[6] * s[15] + s[0] * s[7] * s[14] + s[4] * s[2] * s[15] - + s[4] * s[3] * s[14] - s[12] * s[2] * s[7] + s[12] * s[3] * s[6]; + inv[10] = s[0] * s[5] * s[15] - s[0] * s[7] * s[13] - s[4] * s[1] * s[15] + + s[4] * s[3] * s[13] + s[12] * s[1] * s[7] - s[12] * s[3] * s[5]; + inv[14] = -s[0] * s[5] * s[14] + s[0] * s[6] * s[13] + s[4] * s[1] * s[14] - + s[4] * s[2] * s[13] - s[12] * s[1] * s[6] + s[12] * s[2] * s[5]; + inv[3] = -s[1] * s[6] * s[11] + s[1] * s[7] * s[10] + s[5] * s[2] * s[11] - + s[5] * s[3] * s[10] - s[9] * s[2] * s[7] + s[9] * s[3] * s[6]; + inv[7] = s[0] * s[6] * s[11] - s[0] * s[7] * s[10] - s[4] * s[2] * s[11] + + s[4] * s[3] * s[10] + s[8] * s[2] * s[7] - s[8] * s[3] * s[6]; + inv[11] = -s[0] * s[5] * s[11] + s[0] * s[7] * s[9] + s[4] * s[1] * s[11] - + s[4] * s[3] * s[9] - s[8] * s[1] * s[7] + s[8] * s[3] * s[5]; + inv[15] = s[0] * s[5] * s[10] - s[0] * s[6] * s[9] - s[4] * s[1] * s[10] + + s[4] * s[2] * s[9] + s[8] * s[1] * s[6] - s[8] * s[2] * s[5]; + + double det = s[0] * inv[0] + s[1] * inv[4] + s[2] * inv[8] + s[3] * inv[12]; + if (std::abs(det) < 1e-12) { + return std::nullopt; + } + det = 1.0 / det; + Mat4 r; + for (int i = 0; i < 16; ++i) { + r.m[static_cast<std::size_t>(i)] = inv[static_cast<std::size_t>(i)] * det; + } + return r; +} + +struct LocalPoint { + double x{}, y{}; +}; + +// Unproject a screen point through `transform` back onto the element's z=0 +// plane. `transform` is the same forward operator used by project_to_screen +// (RCSS transform about origin). Returns the surface-local (lx,ly). +inline auto unproject_to_local(const Mat4& transform, double sx, double sy) + -> std::optional<LocalPoint> { + const std::optional<Mat4> inv = invert(transform); + if (!inv) { + return std::nullopt; + } + // Two clip-space points along the viewing ray at the screen pixel: clip-z + // is free under an orthographic screen, so pick z=0 and z=1 (homogeneous + // w=1) and map both back to model space, then intersect with model z=0. + const Vec4 a = apply(*inv, Vec4{sx, sy, 0.0, 1.0}); + const Vec4 b = apply(*inv, Vec4{sx, sy, 1.0, 1.0}); + const auto dehom = [](const Vec4& v) -> Vec4 { + const double iw = (std::abs(v.w) < 1e-12) ? 0.0 : 1.0 / v.w; + return Vec4{v.x * iw, v.y * iw, v.z * iw, 1.0}; + }; + const Vec4 pa = dehom(a); + const Vec4 pb = dehom(b); + const double dz = pb.z - pa.z; + if (std::abs(dz) < 1e-12) { + return std::nullopt; // ray parallel to the element plane + } + const double t = (0.0 - pa.z) / dz; // param where the ray crosses z=0 + return LocalPoint{pa.x + (pb.x - pa.x) * t, pa.y + (pb.y - pa.y) * t}; +} + +// ---- Parent-relative child placement (surface trees) ------------------------ +// +// A surface element's subsurfaces/popups are per-subsurface child elements +// positioned at their tree offset (sx,sy in the ROOT surface's pixel space) +// relative to the parent's RESOLVED on-screen box (spike §6 edge note). Given +// the parent <img>'s resolved content box (px) and the parent surface's natural +// pixel size, this maps a child node's (sx,sy,w,h) into the child <img>'s box in +// the SAME document space as the parent (so a moving/transformed parent — whose +// resolved box changes — drags its children's element boxes when this is +// recomputed each frame). Pure: input box+offsets -> output box. +struct ChildBox { + double x{}, y{}, w{}, h{}; +}; + +// `px,py,pw,ph` = parent <img> resolved content box (document px). `surf_w` = +// parent surface natural width in px (the texture width the box renders); +// `surf_h` its height. `sx,sy` = child tree offset in root-surface px; `cw,ch` = +// child natural pixel size. The box is scaled by the parent box / surface size +// so a child sits at the correct fraction of the (possibly resized) parent box. +[[nodiscard]] inline auto place_child_box(double px, double py, double pw, double ph, int surf_w, + int surf_h, int sx, int sy, int cw, int ch) -> ChildBox { + const double scale_x = surf_w > 0 ? pw / static_cast<double>(surf_w) : 1.0; + const double scale_y = surf_h > 0 ? ph / static_cast<double>(surf_h) : 1.0; + return ChildBox{ + px + static_cast<double>(sx) * scale_x, + py + static_cast<double>(sy) * scale_y, + static_cast<double>(cw) * scale_x, + static_cast<double>(ch) * scale_y, + }; +} + +} // namespace unbox::kernel diff --git a/packages/kernel/src/server.cpp b/packages/kernel/src/server.cpp index b79337c..8b67c8a 100644 --- a/packages/kernel/src/server.cpp +++ b/packages/kernel/src/server.cpp @@ -1,5 +1,7 @@ #include "server_impl.hpp" +#include <xkbcommon/xkbcommon.h> // virtual-keyboard test seam keymap + #include <ctime> #include <stdexcept> #include <unistd.h> @@ -152,6 +154,78 @@ auto Server::ui_reload_surface() -> bool { return impl_->substrate != nullptr && impl_->substrate->reload_first_surface(); } +void Server::ui_route_pointer_motion_for_test(double lx, double ly, unsigned int time_msec) { + if (impl_->substrate != nullptr) { + impl_->substrate->route_pointer_motion(lx, ly, time_msec); + } +} + +void Server::ui_route_pointer_button_for_test(double lx, double ly, bool pressed, + unsigned int time_msec) { + if (impl_->substrate != nullptr) { + // A button needs a hover first (the cursor is already at (lx,ly) on a real + // seat); feed a motion so the substrate's pick is current, then the button. + impl_->substrate->route_pointer_motion(lx, ly, time_msec); + (void)impl_->substrate->route_pointer_button(lx, ly, pressed, time_msec); + } +} + +void Server::ui_route_touch_down_for_test(int id, double lx, double ly, unsigned int time_msec) { + if (impl_->substrate != nullptr) { + (void)impl_->substrate->route_touch_down(id, lx, ly, time_msec); + } +} + +void Server::ui_route_touch_up_for_test(int id, unsigned int time_msec) { + if (impl_->substrate != nullptr) { + (void)impl_->substrate->route_touch_up(id, time_msec); + } +} + +namespace { +// A no-op wlr_keyboard impl for the virtual test keyboard (it never produces LED +// updates; the seat only needs a keyboard object + keymap to ship an enter). +const wlr_keyboard_impl kTestKeyboardImpl = { + .name = "unbox-test-keyboard", + .led_update = nullptr, +}; +} // namespace + +void Server::ui_add_test_keyboard() { + if (impl_->test_keyboard != nullptr || impl_->seat == nullptr) { + return; + } + auto* kb = new wlr_keyboard(); + wlr_keyboard_init(kb, &kTestKeyboardImpl, "unbox-test-keyboard"); + xkb_context* ctx = xkb_context_new(XKB_CONTEXT_NO_FLAGS); + xkb_keymap* keymap = xkb_keymap_new_from_names(ctx, nullptr, XKB_KEYMAP_COMPILE_NO_FLAGS); + wlr_keyboard_set_keymap(kb, keymap); + xkb_keymap_unref(keymap); + xkb_context_unref(ctx); + impl_->test_keyboard = kb; + // Advertise the keyboard capability + set it on the seat so an enter ships + // the keymap (mirrors input.cpp::new_keyboard's seat wiring). + wlr_seat_set_capabilities(impl_->seat, WL_SEAT_CAPABILITY_POINTER | + WL_SEAT_CAPABILITY_TOUCH | + WL_SEAT_CAPABILITY_KEYBOARD); + wlr_seat_set_keyboard(impl_->seat, kb); +} + +void Server::ui_send_key_for_test(unsigned int keycode, bool pressed) { + if (impl_->seat == nullptr) { + return; + } + // The post-filter equivalent of input.cpp's key path: the seat forwards the + // key to whatever surface holds keyboard focus (set by focus_keyboard()). + timespec now{}; + clock_gettime(CLOCK_MONOTONIC, &now); + const std::uint32_t t = static_cast<std::uint32_t>(now.tv_sec) * 1000U + + static_cast<std::uint32_t>(now.tv_nsec) / 1000000U; + wlr_seat_keyboard_notify_key(impl_->seat, t, keycode, + pressed ? WL_KEYBOARD_KEY_STATE_PRESSED + : WL_KEYBOARD_KEY_STATE_RELEASED); +} + void Server::ui_set_touch_override(UiTouchOverride ov) { if (impl_->substrate == nullptr) { return; @@ -446,7 +520,7 @@ void Server::Impl::start_substrate() { // isolation path the bus uses (Server::Impl is the DisableSink). The // substrate uses the kernel's ONE shared FileWatcher for (UNBOX_DEV-gated) // asset hot-reload — the same watcher Host::watch_file uses for config. - substrate = Substrate::create(display_egl, allocator, renderer, file_watcher(), + substrate = Substrate::create(display_egl, allocator, renderer, seat, file_watcher(), [this](ExtensionId who) { disable(who); }, [this] { schedule_driver_frame(); }); } @@ -502,6 +576,13 @@ void Server::Impl::shutdown() { test_new_surface.disconnect(); test_surface_commits.clear(); test_last_client_surface = nullptr; + // The virtual test keyboard (if added) is finished + freed before the seat / + // display die (a wlr_keyboard outliving the seat it was set on is UB). + if (test_keyboard != nullptr) { + wlr_keyboard_finish(test_keyboard); + delete test_keyboard; + test_keyboard = nullptr; + } // The ui substrate owns scene nodes + GL objects on a sibling context and // borrows scene/renderer/allocator: tear it down before they die. (Its asset diff --git a/packages/kernel/src/server_impl.hpp b/packages/kernel/src/server_impl.hpp index 0e69760..821ea9e 100644 --- a/packages/kernel/src/server_impl.hpp +++ b/packages/kernel/src/server_impl.hpp @@ -98,6 +98,12 @@ struct Server::Impl : detail::DisableSink { wlr_surface* test_last_client_surface = nullptr; // last surface with a buffer std::unique_ptr<SurfaceElement> test_surface_element; // built on demand by the probe + // ---- surface-element input-back test seam (kernel suite only) ---- + // A minimal virtual keyboard added to the seat on demand (headless has no + // input), so SurfaceElement::focus_keyboard can deliver a wl_keyboard enter + + // a forwarded key. Owned here; finished/freed in shutdown() before the seat. + wlr_keyboard* test_keyboard = nullptr; + // Ordered scene-tree z-bands (SceneLayer order). Created once over // scene->tree in stacking order so background < … < overlay. Extensions // attach nodes via Host::scene_layer(); the kernel owns them. diff --git a/packages/kernel/src/ui_substrate.cpp b/packages/kernel/src/ui_substrate.cpp index 2265c5b..0a4b508 100644 --- a/packages/kernel/src/ui_substrate.cpp +++ b/packages/kernel/src/ui_substrate.cpp @@ -1,6 +1,7 @@ #include "ui_substrate.hpp" #include "file_watcher.hpp" +#include "input_core.hpp" // pure surface-element input-back geometry (place_child_box) #include "listener.hpp" // RAII wl_listener for the surface-element commit hook #include "rmlui_renderer_gl3.h" @@ -28,6 +29,10 @@ #include <GLES2/gl2ext.h> // glEGLImageTargetTexture2DOES #include <GLES3/gl32.h> +// BTN_LEFT for the surface-element input-back pointer-button forward. A pure +// value header (input-event codes), not a wlroots/GL effect surface. +#include <linux/input-event-codes.h> + #include <algorithm> #include <cstdint> #include <cstdlib> @@ -611,43 +616,74 @@ struct PreviewState { bool dmabuf = false; // true once a dmabuf import succeeded }; -// ---- SurfaceElementState ---------------------------------------------------- +// ---- SurfaceElementState (a surface TREE) ----------------------------------- // -// A LIVE surface element: a client wl_surface's CURRENT committed buffer -// imported zero-copy as a sampled GL texture in the RMLUi sibling context and -// registered under an "unbox-surface://N" URI — the live sibling of -// PreviewState. Ports spike `LiveTexture` (src/spike/spike_gl.hpp) into the real -// substrate: the seq-gated re-import (§0d frozen-frame fix) + the double-buffered -// wlr_buffer lock/unlock lifecycle (at most ONE buffer pinned). Re-import + -// frame-done run on the sibling context inside tick_all (the caller makes it -// current); the RAII commit Listener (kernel-private, never crosses the -// contract) marks the element dirty + asks the kernel to schedule a frame. -// Lives in Substrate::Impl::surface_elements (stable addresses). -struct SurfaceElementState { - Substrate::Impl* owner = nullptr; - int id = 0; - std::string uri; - - wlr_surface* surface = nullptr; // BORROW; caller outlives the element (ui.hpp) - int width = 0; +// A LIVE surface element: a client wl_surface ROOT + its subsurfaces + (if it is +// an xdg surface) its xdg popups, each imported zero-copy as its OWN sampled GL +// texture in the RMLUi sibling context and registered under its OWN +// "unbox-surface://N" / "unbox-surface://N.K" URI — the live sibling of +// PreviewState, generalised to a TREE (Wave 1b). Ports spike `LiveTexture` +// (src/spike/spike_gl.hpp) + the per-LiveSurface subsurface/popup model +// (rml_compositing_spike_run.cpp): the seq-gated re-import (§0d frozen-frame fix) +// + the double-buffered wlr_buffer lock/unlock lifecycle (at most ONE buffer +// pinned PER NODE). Re-import + frame-done run on the sibling context inside +// tick_all (the caller makes it current); the RAII commit Listener (kernel- +// private, never crosses the contract) marks the element dirty + asks the kernel +// to schedule a frame. Lives in Substrate::Impl::surface_elements (stable addrs). + +// One node of the surface tree: a single wl_surface's live import (root, a +// subsurface, or a popup's surface). Its `surface` is a BORROW valid only while +// the node exists in the tree — the substrate re-enumerates the live tree each +// frame (adopt_surface_element) and drops a node the instant its surface stops +// appearing in the walk, so it never holds a freed child surface across frames. +struct SurfaceNode { + wlr_surface* surface = nullptr; // BORROW (see above) + std::string uri; // this node's <img src> URI + int width = 0; // node surface natural px (tracks commits) int height = 0; + int sx = 0; // tree offset within the ROOT surface (px) + int sy = 0; + bool is_popup = false; // popups are NOT parent-clipped (spike crit 4) - // The live import (ported from spike LiveTexture). `current` is the buffer we + // The live import (port of spike LiveTexture). `current` is the buffer we // hold LOCKED + have imported; `current_seq` its surface commit seq. wlr_buffer* current = nullptr; std::uint32_t current_seq = 0; bool have_seq = false; // false until the first adopt() EGLImageKHR image = EGL_NO_IMAGE_KHR; GLuint tex = 0; - bool dmabuf = false; // last import took the dmabuf path - int reimports = 0; // REAL re-imports (seq advances) — test probe - int frame_done_sends = 0; // frame-done calls — test probe - // RAII commit hook: a client wl_surface.commit dirties this element + kicks - // the dirty-gate. Destruction (element drop) unsubscribes — never a bare - // wl_listener across the contract (.unbox/rules/listener-lifetime.md). + bool seen = false; // tree-walk mark (this frame) — GC sweep +}; + +struct SurfaceElementState { + Substrate::Impl* owner = nullptr; + int id = 0; + + // The ROOT node. `root.surface` is the wl_surface passed to + // create_surface_element — a BORROW the caller outlives (ui.hpp). `root.uri` + // is the element's source_uri() (the single <img src> a consumer authors); + // child nodes get sibling URIs and the substrate places their <img> elements + // relative to the root's resolved box (parent-relative child placement). + SurfaceNode root; + // Per-subsurface / per-popup child nodes, in TREE (= composite) order. Each + // its own live texture + URI; placed at its tree offset relative to the root. + std::list<SurfaceNode> children; + int next_child_id = 0; // numbers child URIs "unbox-surface://N.K" + + bool dmabuf = false; // last import (any node) took the dmabuf path + int reimports = 0; // REAL re-imports across ALL nodes — test probe + int frame_done_sends = 0; // frame-done calls across ALL nodes — test probe + + // RAII commit hook on the ROOT surface: a client commit dirties this element + // + kicks the dirty-gate (the whole tree is re-walked + re-imported next + // tick). Destruction (element drop) unsubscribes — never a bare wl_listener + // across the contract (.unbox/rules/listener-lifetime.md). Listener commit_l; - bool needs_reimport = true; // a commit happened => re-adopt next tick + bool needs_reimport = true; // a commit happened => re-walk + re-adopt next tick + + // Convenience: the element's stable source_uri() (== root.uri). + [[nodiscard]] auto uri() const -> const std::string& { return root.uri; } }; // ---- Substrate::Impl -------------------------------------------------------- @@ -656,6 +692,11 @@ struct Substrate::Impl { GlBridge gl; wlr_allocator* allocator = nullptr; wlr_renderer* renderer = nullptr; + // The kernel's seat — a BORROW (the seat is destroyed with the wl_display, + // AFTER the substrate, so it outlives every input-back call). Used ONLY by + // surface-element input-back (route_* forwarding a pick to the client via + // wlr_seat_pointer/touch/keyboard_notify_*). May be null (no seat / test). + wlr_seat* seat = nullptr; SubstrateDisableFn disable; // Ask the kernel to schedule an output frame (the dirty-gate kick for live // surface elements: a client commit, or the continuous frame-callback loop @@ -699,13 +740,19 @@ struct Substrate::Impl { bool last_preview_dmabuf = false; // test probe: last import took the dmabuf path int resize_realloc_count = 0; // test probe: # of set_size GL-target reallocs - // Live surface elements (RML compositing Wave 1): stable addresses + // Live surface elements (RML compositing Wave 1/1b): stable addresses // (SurfaceElementHandle borrows a SurfaceElementState*). next_surface_id - // numbers the "unbox-surface://N" URIs. + // numbers the "unbox-surface://N" URIs (root nodes). std::list<SurfaceElementState> surface_elements; int next_surface_id = 0; bool last_surface_element_dmabuf = false; // test probe: last import path + // The wl_surface that currently holds keyboard focus via a surface element + // (set by SurfaceElement::focus_keyboard, cleared when that element/its node + // is destroyed). A BORROW used only to clear focus on destroy; never deref'd + // for routing. Focus POLICY is a later wave — this is only the mechanism. + wlr_surface* keyboard_focus = nullptr; + // Pointer implicit grab: the consumer of the first button press owns the // whole press..release stream (standard seat behavior). `pointer_grab` // (pure) tracks owner + down-count; `pointer_grab_surface` is the ui surface @@ -784,15 +831,55 @@ struct Substrate::Impl { bool import_snapshot(PreviewState& p); void destroy_preview(PreviewState* p); - // Surface elements (RML compositing Wave 1). adopt_surface_element re-imports - // `s`'s surface's CURRENT buffer if (and only if) the commit seq advanced - // (seq-gate); it manages the double-buffered wlr_buffer lock and registers - // the URI. Caller holds the sibling context current. Returns true if the - // sampled texture reflects the current buffer afterwards. destroy_surface_ - // element drops the import (texture/EGLImage/held lock), the URI, and the - // element from the list. - bool adopt_surface_element(SurfaceElementState& s); + // Surface elements (RML compositing Wave 1/1b). + // + // adopt_node re-imports ONE node's surface CURRENT buffer iff its commit seq + // advanced (seq-gate); manages the double-buffered wlr_buffer lock + registers + // the node URI. Caller holds the sibling context current. Returns true if the + // node's sampled texture reflects its current buffer afterwards. + bool adopt_node(SurfaceElementState& el, SurfaceNode& node); + // Re-walk `el`'s live surface TREE (root + subsurfaces + xdg popups): create + // child nodes for newly-appeared surfaces, drop nodes whose surface vanished, + // refresh each node's tree offset, and re-import every node (seq-gated). This + // is the Wave-1b tree generalisation of adopt: composite order = tree order, + // popups NOT parent-clipped. Caller holds the sibling context current. + bool adopt_surface_element(SurfaceElementState& el); + // Drop ONE node's GL import (texture/EGLImage/held lock) + URI registration. + // Caller holds the sibling context current for the GL/URI part. + void destroy_node(SurfaceNode& node); + // Drop the whole element: every node's import + the root commit listener, and + // clear keyboard focus if it pointed at any of this element's surfaces. void destroy_surface_element(SurfaceElementState* s); + // Place the per-node child <img> elements in EVERY ui surface document that + // hosts this element's root <img src=root.uri>, at each child's tree offset + // relative to the root's resolved box (parent-relative placement). Removes a + // child <img> whose node is gone. Pure DOM glue (no GL); called from tick_all + // after the tree was re-walked. Idempotent per frame. + void layout_surface_element_children(SurfaceElementState& el); + + // ---- surface-element input-back ---- + // Resolve a hovered RmlUi element to the surface-element NODE it samples (its + // <img src> is a node URI), walking up parents. Returns null if the pick is + // not over a surface element (then normal ui routing / data-events apply). + // `out_el` receives the owning element. Borrows valid only during the call. + auto node_for_hovered(Rml::Element* hovered, SurfaceElementState*& out_el) -> SurfaceNode*; + // Forward a transform-aware pick on `surf`'s context at surface-local doc + // coords (lx-s.x, ly-s.y) to the client surface under it, via the seat, at + // surface-LOCAL px (Element::Project through the node <img>'s real transform, + // then box->surface scale — the spike route_point fix). Returns true if a + // client surface was found + entered (so the caller knows a client was hit). + // `pressed`/`is_button` drive the pointer button + implicit-grab semantics. + enum class PointerKind { motion, button_down, button_up }; + auto forward_pointer_to_client(Surface& surf, double lx, double ly, PointerKind kind, + std::uint32_t button, std::uint32_t time_msec) -> bool; + auto forward_touch_to_client(Surface& surf, double lx, double ly, std::int32_t id, + bool down, std::uint32_t time_msec) -> bool; + // Keyboard-focus PRIMITIVE (mechanism only — focus POLICY is a later wave). + // Give `el`'s ROOT client surface seat keyboard focus + send the enter with + // the active keyboard's pressed keys + modifiers, so the kernel's existing + // key passthrough (input.cpp) routes subsequent keys to it. Idempotent. + // Clearing focus on element/node destroy is handled in destroy_*. + void focus_keyboard(SurfaceElementState& el); // Forward a synthesized pointer event into a surface's Rml context. Returns // whether RmlUi (or our hit-test) treats it as consumed. @@ -1443,7 +1530,7 @@ void Substrate::Impl::destroy_preview(PreviewState* p) { // ---- Surface elements: seq-gated live import (port of spike LiveTexture) ----- // -// The client surface's CURRENT committed buffer is `surface->buffer` — a +// Each NODE's CURRENT committed buffer is `node.surface->buffer` — a // wlr_client_buffer (the renderer-side import; wlroots has ALREADY released the // client's underlying pool wl_buffer, so reading/locking it can never starve the // client). We re-import it ONLY when the surface commit SEQUENCE advances @@ -1452,47 +1539,47 @@ void Substrate::Impl::destroy_preview(PreviewState* p) { // re-commits the SAME wlr_buffer pointer with NEW contents and a bumped seq => // re-import (the §0d frozen-frame fix). The buffer we import is LOCKED for the // import+sample lifetime and the PREVIOUS one unlocked once the new import is -// live — double-buffered, at most one buffer pinned, balanced in EVERY case incl. -// the pooled same-pointer re-commit (prev == buf: net +1 then -1). +// live — double-buffered, at most one buffer pinned PER NODE, balanced in EVERY +// case incl. the pooled same-pointer re-commit (prev == buf: net +1 then -1). -bool Substrate::Impl::adopt_surface_element(SurfaceElementState& s) { - if (s.surface == nullptr) { - return s.tex != 0; +bool Substrate::Impl::adopt_node(SurfaceElementState& el, SurfaceNode& node) { + if (node.surface == nullptr) { + return node.tex != 0; } wlr_buffer* buf = nullptr; - if (s.surface->buffer != nullptr) { - buf = &s.surface->buffer->base; + if (node.surface->buffer != nullptr) { + buf = &node.surface->buffer->base; } if (buf == nullptr) { // No buffer committed yet (e.g. a 0x0 configure-ack commit) — nothing to // import; keep any prior texture. Not a failure. - return s.tex != 0; + return node.tex != 0; } - const std::uint32_t seq = s.surface->current.seq; - if (!surface_element_needs_reimport(s.have_seq, s.current_seq, seq, buf == s.current, - s.tex != 0)) { + const std::uint32_t seq = node.surface->current.seq; + if (!surface_element_needs_reimport(node.have_seq, node.current_seq, seq, buf == node.current, + node.tex != 0)) { return true; // truly unchanged surface state: zero re-import, zero copy } // Lock the buffer we are about to sample (its dmabuf FDs / shm storage must // stay valid for the whole import+sample); release the PREVIOUS one once the - // new import is live (double-buffered: at most one buffer pinned). - wlr_buffer* prev = s.current; + // new import is live (double-buffered: at most one buffer pinned per node). + wlr_buffer* prev = node.current; wlr_buffer_lock(buf); auto commit = [&](bool is_dmabuf) { - s.current = buf; - s.current_seq = seq; - s.have_seq = true; - s.dmabuf = is_dmabuf; + node.current = buf; + node.current_seq = seq; + node.have_seq = true; + el.dmabuf = is_dmabuf; last_surface_element_dmabuf = is_dmabuf; - ++s.reimports; + ++el.reimports; if (prev != nullptr) { wlr_buffer_unlock(prev); // prev may == buf (pooled re-commit): net +1/-1 } if (gl.render_iface) { - gl.render_iface->register_preview_texture(s.uri, s.tex, - Rml::Vector2i(s.width, s.height)); + gl.render_iface->register_preview_texture(node.uri, node.tex, + Rml::Vector2i(node.width, node.height)); } }; @@ -1513,24 +1600,24 @@ bool Substrate::Impl::adopt_surface_element(SurfaceElementState& s) { gl.egl_create_image(gl.egl_display, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, nullptr, ia); if (img != EGL_NO_IMAGE_KHR) { // Drop the old GL objects, build the new texture from the EGLImage. - if (s.tex != 0) { - glDeleteTextures(1, &s.tex); - s.tex = 0; + if (node.tex != 0) { + glDeleteTextures(1, &node.tex); + node.tex = 0; } - if (s.image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { - gl.egl_destroy_image(gl.egl_display, s.image); + if (node.image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { + gl.egl_destroy_image(gl.egl_display, node.image); } - glGenTextures(1, &s.tex); - glBindTexture(GL_TEXTURE_2D, s.tex); + glGenTextures(1, &node.tex); + glBindTexture(GL_TEXTURE_2D, node.tex); gl.gl_image_target_texture(GL_TEXTURE_2D, static_cast<GLeglImageOES>(img)); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glBindTexture(GL_TEXTURE_2D, 0); - s.image = img; - s.width = attribs.width; - s.height = attribs.height; + node.image = img; + node.width = attribs.width; + node.height = attribs.height; commit(true); return true; } @@ -1544,18 +1631,18 @@ bool Substrate::Impl::adopt_surface_element(SurfaceElementState& s) { if (!wlr_buffer_begin_data_ptr_access(buf, WLR_BUFFER_DATA_PTR_ACCESS_READ, &data, &fmt, &stride)) { wlr_buffer_unlock(buf); // import failed: drop the lock we just took - return s.tex != 0; + return node.tex != 0; } - if (s.tex != 0) { - glDeleteTextures(1, &s.tex); - s.tex = 0; + if (node.tex != 0) { + glDeleteTextures(1, &node.tex); + node.tex = 0; } - if (s.image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { - gl.egl_destroy_image(gl.egl_display, s.image); - s.image = EGL_NO_IMAGE_KHR; + if (node.image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { + gl.egl_destroy_image(gl.egl_display, node.image); + node.image = EGL_NO_IMAGE_KHR; } - glGenTextures(1, &s.tex); - glBindTexture(GL_TEXTURE_2D, s.tex); + glGenTextures(1, &node.tex); + glBindTexture(GL_TEXTURE_2D, node.tex); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_SWIZZLE_R, GL_BLUE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_SWIZZLE_B, GL_RED); glPixelStorei(GL_UNPACK_ROW_LENGTH, static_cast<GLint>(stride / 4)); @@ -1568,45 +1655,429 @@ bool Substrate::Impl::adopt_surface_element(SurfaceElementState& s) { glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glBindTexture(GL_TEXTURE_2D, 0); wlr_buffer_end_data_ptr_access(buf); - s.width = buf->width; - s.height = buf->height; + node.width = buf->width; + node.height = buf->height; commit(false); return true; } +namespace { +// Tree-walk scratch: collect every (surface, tree-offset, is_popup) under a +// root, in tree (= composite) order, so adopt_surface_element can reconcile it +// against the element's existing nodes. wlr_surface_for_each_surface visits the +// surface + all subsurfaces root->leaves (z-order); xdg popups are walked +// separately (each its own tree under the toplevel), exactly as the spike's +// send_frame_done_to_clients walks them. The ROOT itself (sx=sy=0) is the FIRST +// visited subsurface entry and is handled as el.root, so it is skipped here. +struct TreeEntry { + wlr_surface* surface = nullptr; + int sx = 0; + int sy = 0; + bool is_popup = false; +}; +struct TreeWalk { + wlr_surface* root = nullptr; + std::vector<TreeEntry>* out = nullptr; + bool popup_pass = false; + int popup_ox = 0; // popup tree offset within the root (toplevel coords) + int popup_oy = 0; +}; +void tree_walk_cb(wlr_surface* surface, int sx, int sy, void* data) { + auto* w = static_cast<TreeWalk*>(data); + if (!w->popup_pass && surface == w->root) { + return; // the root is el.root, not a child node + } + w->out->push_back(TreeEntry{surface, w->popup_ox + sx, w->popup_oy + sy, w->popup_pass}); +} +} // namespace + +// A stable DOM id for a child node's <img> in a hosting document: derived from +// the node URI (unique per node, stable for the node's life), prefixed so it +// never collides with author ids. The substrate creates/removes the child <img> +// under this id; consumers do NOT author or address it (the element represents +// the whole tree — ui.hpp). Defined here so destroy_surface_element + +// layout_surface_element_children agree on the id. +namespace { +auto child_dom_id(const SurfaceElementState& /*el*/, const SurfaceNode& node) -> std::string { + // node.uri is "unbox-surface://N.K"; turn the scheme-y chars into id-safe + // ones so RmlUi's id lookup is well-formed. + std::string id = "unbox-se-child-"; + for (char ch : node.uri) { + id.push_back((ch == ':' || ch == '/' || ch == '.') ? '-' : ch); + } + return id; +} +} // namespace + +bool Substrate::Impl::adopt_surface_element(SurfaceElementState& el) { + // 1) Re-import the ROOT node first (seq-gated). + bool any = adopt_node(el, el.root); + + if (el.root.surface == nullptr) { + return any; + } + + // 2) Walk the live tree: subsurfaces (any surface) + xdg popups (if the root + // is an xdg surface). Collect entries in tree order, then reconcile. + std::vector<TreeEntry> entries; + TreeWalk walk; + walk.root = el.root.surface; + walk.out = &entries; + wlr_surface_for_each_surface(el.root.surface, tree_walk_cb, &walk); + + if (wlr_xdg_surface* xdg = wlr_xdg_surface_try_from_wlr_surface(el.root.surface)) { + // Popups: each popup's surface (+ its own subsurfaces) at coords relative + // to the toplevel. wlr_xdg_surface_for_each_popup_surface gives the + // popup's surface-tree with sx/sy already in the parent xdg surface's + // coordinate space (== root surface space here). Popups are NOT parent- + // clipped — they become their own absolutely-placed child <img> (crit 4). + walk.popup_pass = true; + wlr_xdg_surface_for_each_popup_surface(xdg, tree_walk_cb, &walk); + } + + // 3) Reconcile entries -> el.children (match by wlr_surface identity). Mark + // every existing child unseen, walk entries (create/refresh), then sweep + // unseen children (their surface vanished — drop the node + its import). + for (SurfaceNode& c : el.children) { + c.seen = false; + } + for (const TreeEntry& e : entries) { + SurfaceNode* node = nullptr; + for (SurfaceNode& c : el.children) { + if (c.surface == e.surface) { + node = &c; + break; + } + } + if (node == nullptr) { + el.children.emplace_back(); + node = &el.children.back(); + node->surface = e.surface; + node->uri = el.root.uri + "." + std::to_string(++el.next_child_id); + } + node->sx = e.sx; + node->sy = e.sy; + node->is_popup = e.is_popup; + node->seen = true; + any = adopt_node(el, *node) || any; + } + for (auto it = el.children.begin(); it != el.children.end();) { + if (it->seen) { + ++it; + continue; + } + // This child's surface left the tree (subsurface/popup destroyed): drop + // its import + URI. If it held keyboard focus, clear it (mechanism). + if (keyboard_focus == it->surface) { + if (seat != nullptr) { + wlr_seat_keyboard_notify_clear_focus(seat); + } + keyboard_focus = nullptr; + } + destroy_node(*it); + it = el.children.erase(it); + } + return any; +} + +void Substrate::Impl::destroy_node(SurfaceNode& node) { + if (gl.render_iface) { + gl.render_iface->unregister_preview_texture(node.uri); + } + if (node.tex != 0) { + glDeleteTextures(1, &node.tex); + node.tex = 0; + } + if (node.image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { + gl.egl_destroy_image(gl.egl_display, node.image); + node.image = EGL_NO_IMAGE_KHR; + } + if (node.current != nullptr) { + wlr_buffer_unlock(node.current); // release the buffer we held locked + node.current = nullptr; + } + node.have_seq = false; +} + void Substrate::Impl::destroy_surface_element(SurfaceElementState* s) { s->commit_l.disconnect(); // stop dirtying after the element is gone - const bool cur = gl.make_current(); - if (gl.render_iface) { - gl.render_iface->unregister_preview_texture(s->uri); + // Clear keyboard focus if it pointed at any of this element's surfaces (the + // mechanism's clean-up half — never leave the seat focused on a dead surface). + auto owns_focus = [&](wlr_surface* f) { + if (s->root.surface == f) { + return true; + } + for (const SurfaceNode& c : s->children) { + if (c.surface == f) { + return true; + } + } + return false; + }; + if (keyboard_focus != nullptr && owns_focus(keyboard_focus)) { + if (seat != nullptr) { + wlr_seat_keyboard_notify_clear_focus(seat); + } + keyboard_focus = nullptr; } - if (s->tex != 0) { - glDeleteTextures(1, &s->tex); - s->tex = 0; + // Remove this element's child <img> elements from every hosting document + // (before the URIs are unregistered) so no stale <img src> survives. + for (Surface& surf : surfaces) { + if (surf.document == nullptr) { + continue; + } + for (const SurfaceNode& c : s->children) { + if (Rml::Element* el = surf.document->GetElementById(child_dom_id(*s, c))) { + if (Rml::Element* parent = el->GetParentNode()) { + parent->RemoveChild(el); + } + } + } } - if (s->image != EGL_NO_IMAGE_KHR && gl.egl_destroy_image != nullptr) { - gl.egl_destroy_image(gl.egl_display, s->image); - s->image = EGL_NO_IMAGE_KHR; + const bool cur = gl.make_current(); + destroy_node(s->root); + for (SurfaceNode& c : s->children) { + destroy_node(c); } if (cur) { gl.restore_current(); } - if (s->current != nullptr) { - wlr_buffer_unlock(s->current); // release the buffer we held locked - s->current = nullptr; - } - s->have_seq = false; + s->children.clear(); surface_elements.remove_if([s](const SurfaceElementState& e) { return &e == s; }); } +// ---- Surface-element child placement (parent-relative, Wave 1b) -------------- + +namespace { +// Find the <img> in `doc` whose src is exactly `uri` (the root surface-element +// img the consumer authored, or a child img the substrate created). nullptr if +// the document does not host it. Cheap linear scan over <img> (documents have a +// handful). `src` resolution: RmlUi keeps the raw attribute, so an exact string +// match identifies the surface element's image robustly across reloads. +auto find_img_by_src(Rml::ElementDocument* doc, const std::string& uri) -> Rml::Element* { + if (doc == nullptr) { + return nullptr; + } + Rml::ElementList imgs; + doc->GetElementsByTagName(imgs, "img"); + for (Rml::Element* img : imgs) { + if (img->GetAttribute<Rml::String>("src", "") == uri) { + return img; + } + } + return nullptr; +} +} // namespace + +void Substrate::Impl::layout_surface_element_children(SurfaceElementState& el) { + // Pure DOM glue (no GL). For every ui surface document that hosts this + // element's ROOT <img src=root.uri>, ensure each child node has its own + // <img> placed at the child's tree offset relative to the root img's resolved + // box (place_child_box — parent-relative placement so a moving/resized parent + // drags its children). Composite order = tree order: we append child imgs in + // el.children order (= the tree walk order) AFTER the root img, so later + // (higher z) nodes draw on top. Popups are NOT parent-clipped — the child img + // is an absolutely-positioned sibling of the root img (no overflow clip). + for (Surface& surf : surfaces) { + Rml::ElementDocument* doc = surf.document; + Rml::Element* root_img = find_img_by_src(doc, el.root.uri); + if (root_img == nullptr) { + continue; // this document does not host the element + } + Rml::Element* container = root_img->GetParentNode(); + if (container == nullptr) { + continue; + } + // The root img's resolved box (content area, the texture's drawn box) in + // the container's coordinate space: place children relative to THIS. + const Rml::Vector2f roff = root_img->GetAbsoluteOffset(Rml::BoxArea::Content); + const Rml::Vector2f coff = container->GetAbsoluteOffset(Rml::BoxArea::Content); + const double px = roff.x - coff.x; + const double py = roff.y - coff.y; + const double pw = root_img->GetClientWidth(); + const double ph = root_img->GetClientHeight(); + + for (const SurfaceNode& node : el.children) { + const std::string id = child_dom_id(el, node); + Rml::Element* img = doc->GetElementById(id); + if (img == nullptr) { + Rml::ElementPtr created = doc->CreateElement("img"); + created->SetId(id); + created->SetAttribute("src", node.uri); + created->SetProperty("position", "absolute"); + created->SetProperty("display", "block"); + img = container->AppendChild(std::move(created)); + } + if (img == nullptr) { + continue; + } + const ChildBox box = place_child_box(px, py, pw, ph, el.root.width, el.root.height, + node.sx, node.sy, node.width, node.height); + img->SetProperty("left", std::to_string(box.x) + "px"); + img->SetProperty("top", std::to_string(box.y) + "px"); + img->SetProperty("width", std::to_string(box.w) + "px"); + img->SetProperty("height", std::to_string(box.h) + "px"); + } + } +} + +// ---- Surface-element input-back (pick -> surface-local -> wl_seat) ----------- +// +// Port of the spike's route_point: the substrate already feeds pointer/touch +// into a ui surface's Rml context (for bind_event/bind_drag). After a move, the +// hovered element is RmlUi's transform-aware pick. If that element is a surface +// element's <img> (root OR a child node), map the point THROUGH the img's own +// transform with Element::Project() (a no-op when untransformed; the spike fix), +// then box->surface-local scale, and forward to THAT client surface via the seat +// at surface-LOCAL px. A pick that lands on normal RML is untouched (data-events +// fire as before) — only surface-element picks forward to clients. + +auto Substrate::Impl::node_for_hovered(Rml::Element* hovered, SurfaceElementState*& out_el) + -> SurfaceNode* { + for (Rml::Element* el = hovered; el != nullptr; el = el->GetParentNode()) { + const Rml::String src = el->GetAttribute<Rml::String>("src", ""); + if (src.empty()) { + continue; + } + for (SurfaceElementState& se : surface_elements) { + if (se.root.uri == src) { + out_el = &se; + return &se.root; + } + for (SurfaceNode& c : se.children) { + if (c.uri == src) { + out_el = &se; + return &c; + } + } + } + } + out_el = nullptr; + return nullptr; +} + +auto Substrate::Impl::forward_pointer_to_client(Surface& surf, double lx, double ly, PointerKind kind, + std::uint32_t button, std::uint32_t time_msec) + -> bool { + if (seat == nullptr || surf.context == nullptr) { + return false; + } + // The caller has ALREADY fed this surface's context the move (ctx_motion), so + // RmlUi's transform-aware hover pick is current; read it. + SurfaceElementState* el = nullptr; + SurfaceNode* node = node_for_hovered(surf.context->GetHoverElement(), el); + if (node == nullptr || node->surface == nullptr) { + // Not over a client surface (over plain RML / a gap): clear client + // pointer focus so a stale client does not keep the pointer. The ui + // surface still got the move above (data-events / hover unaffected). + wlr_seat_pointer_notify_clear_focus(seat); + return false; + } + // Project the screen point onto the node img's OWN (possibly 3D-transformed) + // plane FIRST (Element::Project: the spike fix; no-op when untransformed), + // then box->surface-local scale. The point fed to the context was in + // surface-local doc coords; Project()/GetAbsoluteOffset work in the same + // (untransformed) document space, so we project the SAME doc point. + Rml::Element* img = find_img_by_src(surf.document, node->uri); + if (img == nullptr) { + return false; + } + Rml::Vector2f p(static_cast<float>(lx - surf.x), static_cast<float>(ly - surf.y)); + if (!img->Project(p)) { + return false; // edge-on: ray parallel to the element plane, no valid hit + } + const Rml::Vector2f off = img->GetAbsoluteOffset(Rml::BoxArea::Content); + const float bw = img->GetClientWidth(); + const float bh = img->GetClientHeight(); + if (bw <= 0 || bh <= 0) { + return false; + } + const double fx = std::clamp((p.x - off.x) / bw, 0.0F, 1.0F); + const double fy = std::clamp((p.y - off.y) / bh, 0.0F, 1.0F); + const double sx = fx * node->width; + const double sy = fy * node->height; + // Implicit-grab discipline (mirror src/input.cpp / the spike): enter before + // button, frame after. notify_enter is idempotent for the already-entered + // surface, so calling it each event keeps focus + sends the up-to-date coords. + wlr_seat_pointer_notify_enter(seat, node->surface, sx, sy); + if (kind == PointerKind::motion) { + wlr_seat_pointer_notify_motion(seat, time_msec, sx, sy); + } else { + wlr_seat_pointer_notify_button( + seat, time_msec, button, + kind == PointerKind::button_down ? WL_POINTER_BUTTON_STATE_PRESSED + : WL_POINTER_BUTTON_STATE_RELEASED); + } + wlr_seat_pointer_notify_frame(seat); + return true; +} + +auto Substrate::Impl::forward_touch_to_client(Surface& surf, double lx, double ly, std::int32_t id, + bool down, std::uint32_t time_msec) -> bool { + if (seat == nullptr || surf.context == nullptr) { + return false; + } + // The caller has ALREADY fed this surface's context the move (ctx_motion). + SurfaceElementState* el = nullptr; + SurfaceNode* node = node_for_hovered(surf.context->GetHoverElement(), el); + if (node == nullptr || node->surface == nullptr) { + return false; + } + Rml::Element* img = find_img_by_src(surf.document, node->uri); + if (img == nullptr) { + return false; + } + Rml::Vector2f p(static_cast<float>(lx - surf.x), static_cast<float>(ly - surf.y)); + if (!img->Project(p)) { + return false; + } + const Rml::Vector2f off = img->GetAbsoluteOffset(Rml::BoxArea::Content); + const float bw = img->GetClientWidth(); + const float bh = img->GetClientHeight(); + if (bw <= 0 || bh <= 0) { + return false; + } + const double sx = std::clamp((p.x - off.x) / bw, 0.0F, 1.0F) * node->width; + const double sy = std::clamp((p.y - off.y) / bh, 0.0F, 1.0F) * node->height; + if (down) { + wlr_seat_touch_notify_down(seat, node->surface, time_msec, id, sx, sy); + } else { + wlr_seat_touch_notify_motion(seat, time_msec, id, sx, sy); + } + return true; +} + +void Substrate::Impl::focus_keyboard(SurfaceElementState& el) { + // Mechanism only (focus POLICY is Wave 3). Mirror src/input.cpp / + // ext-xdg-shell discipline: pick the seat's current keyboard, set it on the + // seat (so the enter ships the keymap), and notify_enter the root client + // surface with the keyboard's currently-pressed keys + modifiers. The + // kernel's key handler (input.cpp) already forwards subsequent keys to + // whatever surface holds seat keyboard focus, so this is the whole primitive. + if (seat == nullptr || el.root.surface == nullptr) { + return; + } + wlr_keyboard* kb = wlr_seat_get_keyboard(seat); + if (kb == nullptr) { + // No keyboard device on the seat yet: enter with no keys/mods so the + // client still receives focus (the keymap follows when a keyboard is set). + wlr_seat_keyboard_notify_enter(seat, el.root.surface, nullptr, 0, nullptr); + } else { + wlr_seat_keyboard_notify_enter(seat, el.root.surface, kb->keycodes, kb->num_keycodes, + &kb->modifiers); + } + keyboard_focus = el.root.surface; +} + // ---- Substrate (private surface) -------------------------------------------- auto Substrate::create(EGLDisplay egl_display, wlr_allocator* allocator, wlr_renderer* renderer, - FileWatcher* watcher, SubstrateDisableFn disable, SubstrateScheduleFn schedule) - -> std::unique_ptr<Substrate> { + wlr_seat* seat, FileWatcher* watcher, SubstrateDisableFn disable, + SubstrateScheduleFn schedule) -> std::unique_ptr<Substrate> { auto impl = std::make_unique<Impl>(); impl->allocator = allocator; impl->renderer = renderer; + impl->seat = seat; // borrow: surface-element input-back forwards picks here impl->disable = std::move(disable); impl->schedule = std::move(schedule); // dirty-gate kick for live surface elements impl->watcher = watcher; // shared kernel-owned file watcher (asset hot-reload) @@ -1750,12 +2221,12 @@ auto Substrate::create_surface_element(wlr_surface* client) -> std::unique_ptr<S SurfaceElementState& s = impl_->surface_elements.back(); s.owner = impl_.get(); s.id = ++impl_->next_surface_id; - s.uri = surface_element_uri(s.id); - s.surface = client; + s.root.surface = client; + s.root.uri = surface_element_uri(s.id); // == source_uri() - // Import the client's current buffer now (so source_uri()/width()/height() - // are valid immediately, like create_preview). The sibling context must be - // current for the EGLImage/texture/RmlUi-registration work. + // Import the client's current buffer + walk its tree now (so source_uri()/ + // width()/height() are valid immediately, like create_preview). The sibling + // context must be current for the EGLImage/texture/RmlUi-registration work. if (!impl_->gl.make_current()) { impl_->destroy_surface_element(&s); return nullptr; @@ -1763,7 +2234,7 @@ auto Substrate::create_surface_element(wlr_surface* client) -> std::unique_ptr<S const bool imported = impl_->adopt_surface_element(s); impl_->gl.restore_current(); s.needs_reimport = false; - if (!imported || s.tex == 0) { + if (!imported || s.root.tex == 0) { // The surface may simply have no buffer yet (pre-first-commit); that is a // valid live element that will import on its first commit. Only fail if // there is no GL path to ever import on — but available() already gated @@ -1795,19 +2266,40 @@ auto Substrate::has_surface_elements() const -> bool { return !impl_->surface_elements.empty(); } +namespace { +// Frame-done tree-walk (spike §0c send_frame_done_to_clients): visit the root + +// every subsurface + every xdg popup tree, calling wlr_surface_send_frame_done +// on each mapped surface. wlr_surface_for_each_surface covers the root + +// subsurfaces; xdg popups are walked separately, as the spike does. +struct FrameDoneWalk { + timespec* now = nullptr; + int* counter = nullptr; +}; +void frame_done_cb(wlr_surface* surface, int /*sx*/, int /*sy*/, void* data) { + auto* w = static_cast<FrameDoneWalk*>(data); + wlr_surface_send_frame_done(surface, w->now); + ++*w->counter; +} +} // namespace + void Substrate::send_frame_done_to_surface_elements(const timespec& now) { // Frame-callback duty (the stuck-frame fix, spike §0c // send_frame_done_to_clients): tell every live-element-backing surface "now // is a good time to draw your next frame", so the client keeps producing - // buffers. Wave 1 is single-surface — one wl_surface per element — so we send - // frame-done to that surface directly (subsurface/popup tree-walking is Wave - // 1b). Sent UNCONDITIONALLY per composited output frame (like + // buffers. Wave 1b walks the WHOLE TREE (root + subsurfaces + + // xdg popups), exactly as the spike does — not just the root. Sent + // UNCONDITIONALLY per composited output frame (like // wlr_scene_output_send_frame_done), NOT gated by re-render: the client needs // callbacks to progress regardless of whether WE re-rendered. + timespec t = now; for (SurfaceElementState& s : impl_->surface_elements) { - if (s.surface != nullptr) { - wlr_surface_send_frame_done(s.surface, const_cast<timespec*>(&now)); - ++s.frame_done_sends; + if (s.root.surface == nullptr) { + continue; + } + FrameDoneWalk w{&t, &s.frame_done_sends}; + wlr_surface_for_each_surface(s.root.surface, frame_done_cb, &w); + if (wlr_xdg_surface* xdg = wlr_xdg_surface_try_from_wlr_surface(s.root.surface)) { + wlr_xdg_surface_for_each_popup_surface(xdg, frame_done_cb, &w); } } } @@ -1870,6 +2362,8 @@ void Substrate::tick_all() { for (SurfaceElementState& s : impl_->surface_elements) { if (s.needs_reimport) { const int before = s.reimports; + // Re-walk the tree + re-import every node (seq-gated). A subsurface/ + // popup added or removed since the last tick is reconciled here. impl_->adopt_surface_element(s); // Clear the flag only once the seq actually caught up: adopt is // seq-gated, so if it did nothing (no new buffer) leaving the flag set @@ -1877,10 +2371,16 @@ void Substrate::tick_all() { // (or a no-op on an unchanged seq with a live texture) means we are // current, so clear it. Keep it set only when there is still no // texture (pre-first-buffer) so the first real buffer is picked up. - if (s.reimports != before || s.tex != 0) { + if (s.reimports != before || s.root.tex != 0) { s.needs_reimport = false; } } + // Place per-node child <img> elements in every hosting document at their + // tree offset relative to the root img's resolved box (parent-relative + // placement). Pure DOM glue; cheap (no-op when there are no children or + // no hosting document yet). Done EVERY tick so a moved/resized parent + // drags its children, and a child created this tick gets an <img>. + impl_->layout_surface_element_children(s); } for (Surface& s : impl_->surfaces) { if (s.is_visible) { @@ -1915,13 +2415,24 @@ void Substrate::route_pointer_motion(double lx, double ly, std::uint32_t time_ms s.context->ProcessMouseLeave(); } } + // Surface-element input-back: if the move's transform-aware pick on the target + // ui surface lands on a surface element <img>, forward the motion to THAT + // client at surface-local coords via the seat (the move was just fed above). + // A pick on normal RML clears client pointer focus (handled inside) — the ui + // surface's own data-events/hover already fired, so this only ADDS the client + // forward, never regresses existing routing. + if (target != nullptr) { + (void)impl_->forward_pointer_to_client(*target, lx, ly, + Substrate::Impl::PointerKind::motion, 0, time_msec); + } } -auto Substrate::route_pointer_button(double lx, double ly, bool pressed, std::uint32_t /*time*/) +auto Substrate::route_pointer_button(double lx, double ly, bool pressed, std::uint32_t time_msec) -> bool { if (!impl_->available()) { return false; } + using PK = Substrate::Impl::PointerKind; if (pressed) { // The press decides (or joins) the grab. Owner is fixed at the first // press of the stream; this press routes to that owner. @@ -1936,6 +2447,11 @@ auto Substrate::route_pointer_button(double lx, double ly, bool pressed, std::ui if (impl_->pointer_grab_surface != nullptr) { impl_->ctx_motion(*impl_->pointer_grab_surface, lx, ly); impl_->ctx_button(*impl_->pointer_grab_surface, true); + // Surface-element input-back: forward the press to the client under the + // pick (enter-before-button, frame-after — done inside the helper), + // mirroring the spike / src/input.cpp implicit-grab discipline. + (void)impl_->forward_pointer_to_client(*impl_->pointer_grab_surface, lx, ly, + PK::button_down, BTN_LEFT, time_msec); } return true; // consumed by the substrate } @@ -1948,6 +2464,8 @@ auto Substrate::route_pointer_button(double lx, double ly, bool pressed, std::ui if (impl_->pointer_grab_surface != nullptr) { impl_->ctx_motion(*impl_->pointer_grab_surface, lx, ly); impl_->ctx_button(*impl_->pointer_grab_surface, false); + (void)impl_->forward_pointer_to_client(*impl_->pointer_grab_surface, lx, ly, PK::button_up, + BTN_LEFT, time_msec); } if (!impl_->pointer_grab.active()) { impl_->pointer_grab_surface = nullptr; // grab ended @@ -1984,6 +2502,9 @@ auto Substrate::route_touch_down(std::int32_t id, double lx, double ly, std::uin impl_->touch_capture[id] = hit; impl_->ctx_motion(*hit, lx, ly); impl_->ctx_button(*hit, true); + // Surface-element input-back: forward the touch-down to the client under the + // pick at surface-local coords (the move was just fed above). + (void)impl_->forward_touch_to_client(*hit, lx, ly, id, /*down=*/true, time_msec); return true; } @@ -1998,10 +2519,11 @@ auto Substrate::route_touch_motion(std::int32_t id, double lx, double ly, std::u } impl_->touch_mode_tracker.on_touch(time_msec); impl_->ctx_motion(*it->second, lx, ly); + (void)impl_->forward_touch_to_client(*it->second, lx, ly, id, /*down=*/false, time_msec); return true; } -auto Substrate::route_touch_up(std::int32_t id, std::uint32_t /*time*/) -> bool { +auto Substrate::route_touch_up(std::int32_t id, std::uint32_t time_msec) -> bool { if (!impl_->available()) { return false; } @@ -2010,6 +2532,12 @@ auto Substrate::route_touch_up(std::int32_t id, std::uint32_t /*time*/) -> bool return false; } impl_->ctx_button(*it->second, false); + // Surface-element input-back: end the touch point on the client too. The seat + // tracks the down's surface per id, so a bare notify_up is enough (it is a + // no-op for a point that was never forwarded to a client). + if (impl_->seat != nullptr) { + wlr_seat_touch_notify_up(impl_->seat, time_msec, id); + } impl_->touch_capture.erase(it); return true; } @@ -2582,8 +3110,10 @@ SurfaceElementHandle::~SurfaceElementHandle() { substrate_->impl_->destroy_surface_element(state_); } -auto SurfaceElementHandle::source_uri() const -> std::string { return state_->uri; } -auto SurfaceElementHandle::width() const -> int { return state_->width; } -auto SurfaceElementHandle::height() const -> int { return state_->height; } +auto SurfaceElementHandle::source_uri() const -> std::string { return state_->root.uri; } +auto SurfaceElementHandle::width() const -> int { return state_->root.width; } +auto SurfaceElementHandle::height() const -> int { return state_->root.height; } + +void SurfaceElementHandle::focus_keyboard() { substrate_->impl_->focus_keyboard(*state_); } } // namespace unbox::kernel diff --git a/packages/kernel/src/ui_substrate.hpp b/packages/kernel/src/ui_substrate.hpp index 8eee390..2c2ffe3 100644 --- a/packages/kernel/src/ui_substrate.hpp +++ b/packages/kernel/src/ui_substrate.hpp @@ -108,6 +108,7 @@ public: [[nodiscard]] auto source_uri() const -> std::string override; [[nodiscard]] auto width() const -> int override; [[nodiscard]] auto height() const -> int override; + void focus_keyboard() override; private: Substrate* substrate_; @@ -172,7 +173,7 @@ public: // FileWatcher, used (dev only, UNBOX_DEV-gated) for asset hot-reload; pass // nullptr to disable watching. Never throws. static auto create(EGLDisplay egl_display, wlr_allocator* allocator, - wlr_renderer* renderer, FileWatcher* watcher, + wlr_renderer* renderer, wlr_seat* seat, FileWatcher* watcher, SubstrateDisableFn disable, SubstrateScheduleFn schedule) -> std::unique_ptr<Substrate>; diff --git a/packages/kernel/tests/test_kernel.cpp b/packages/kernel/tests/test_kernel.cpp index f33e31a..0bf6349 100644 --- a/packages/kernel/tests/test_kernel.cpp +++ b/packages/kernel/tests/test_kernel.cpp @@ -4,6 +4,7 @@ #include <unbox/kernel/extension.hpp> #include <unbox/kernel/hooks.hpp> #include <unbox/kernel/host.hpp> +#include <unbox/kernel/listener.hpp> // RAII wl_listener for the Wave-1b tree test extension #include <unbox/kernel/kernel.hpp> #include <unbox/kernel/server.hpp> #include <unbox/kernel/surface_registry.hpp> @@ -13,6 +14,10 @@ // state machine, implicit-grab ownership, hit-test geometry) are doctest-ed // directly, no wlroots. #include "../src/ui_core.hpp" +// The PRODUCTION surface-element input-back PURE core (the port of the spike's +// screen->surface-local inversion + the parent-relative child-placement helper), +// doctest-ed here as the strict-core half of Wave 1b. No wlroots/GL/RMLUi. +#include "../src/input_core.hpp" // The VT-switch escape hatch's pure core (keysym -> VT number), no wlroots. #include "../src/vt_core.hpp" // SPIKE (rml-compositing, Phase 0): the throwaway spike's PURE input-inversion @@ -40,6 +45,14 @@ // compositor, never a client) — see packages/kernel/meson.build. #include <wayland-client.h> +// Wave-1b surface-tree test: a real xdg popup is a tree child, so the test needs +// CLIENT-side xdg-shell bindings (generated; see packages/kernel/meson.build) and +// the SERVER side runs xdg-shell in a TEST extension via the kernel's wlr wrapper +// (xdg-shell stays a feature, provided by an extension — the kernel names none). +#include <unbox/kernel/wlr.hpp> +#include "xdg-shell-client-protocol.h" +#include "xdg-shell-client-protocol-code.h" // private code: included by ONE TU only + #include <atomic> #include <chrono> #include <cstring> @@ -2670,6 +2683,82 @@ TEST_CASE("spike(rml-compositing): an edge-on (90deg) transform collapses the el } // ============================================================================ +// RML compositing Wave 1b: surface-element input-back PURE CORE (src/input_core. +// hpp) — the PRODUCTION port of the spike's screen->surface-local inversion the +// live wl_seat translation rides on, doctest-ed here (criterion-3 round-trip) +// plus the parent-relative child-placement helper. Strict core, zero mocks. +// ============================================================================ + +namespace { +namespace ic = unbox::kernel; + +// Forward-project a surface-local point through `t`, invert, assert recovery. +auto kernel_roundtrip_err(const ic::Mat4& t, double lx, double ly) -> double { + const ic::ScreenPoint s = ic::project_to_screen(t, lx, ly); + const auto back = ic::unproject_to_local(t, s.x, s.y); + if (!back) { + return 1e9; + } + return std::hypot(back->x - lx, back->y - ly); +} +} // namespace + +TEST_CASE("input-back core: screen->surface-local inverts perspective+rotateY (<0.01px)") { + // The criterion-3 case in the PRODUCTION core: a 256x256 surface element with + // perspective(800) + rotateY about the 50% origin (what RCSS computes). The + // inverse is a ray/plane intersection (non-affine); recovery must be sub- + // 0.01px across the element — the geometry the live Element::Project()-based + // forward + wl_seat surface-local notify depends on. + const double origin = 128.0; + for (double deg : {15.0, 35.0, 60.0, -45.0}) { + const ic::Mat4 t = ic::rcss_transform_about_origin( + ic::mul(ic::perspective(800.0), ic::rotate_y(deg * std::numbers::pi / 180.0)), origin, + origin); + CHECK(kernel_roundtrip_err(t, 128.0, 128.0) < 1e-6); // center: on the axis + CHECK(kernel_roundtrip_err(t, 32.0, 64.0) < 0.01); // near edge (foreshortened) + CHECK(kernel_roundtrip_err(t, 224.0, 200.0) < 0.01); // far edge + CHECK(kernel_roundtrip_err(t, 64.0, 96.0) < 0.01); // arbitrary interior + } + // A plain translate (affine): exact everywhere. + const ic::Mat4 tr = ic::translate(120.0, -40.0); + CHECK(kernel_roundtrip_err(tr, 0.0, 0.0) < 1e-9); + CHECK(kernel_roundtrip_err(tr, 200.0, 150.0) < 1e-9); +} + +TEST_CASE("input-back core: place_child_box maps a tree offset into the parent's resolved box") { + using unbox::kernel::place_child_box; + // Parent <img> resolved box (px) == surface natural size (1:1 scale): a child + // at tree offset (10,20) sized 30x40 lands at exactly (10,20,30,40). + { + const auto b = place_child_box(/*px*/ 0, /*py*/ 0, /*pw*/ 200, /*ph*/ 100, + /*surf_w*/ 200, /*surf_h*/ 100, /*sx*/ 10, /*sy*/ 20, + /*cw*/ 30, /*ch*/ 40); + CHECK(b.x == doctest::Approx(10.0)); + CHECK(b.y == doctest::Approx(20.0)); + CHECK(b.w == doctest::Approx(30.0)); + CHECK(b.h == doctest::Approx(40.0)); + } + // A parent rendered at HALF its natural size (a resized window): the offset + + // child size scale by 0.5, and the parent's box origin is added (a moving + // parent drags the child). surf 200x100 drawn into a 100x50 box at (40,30). + { + const auto b = place_child_box(/*px*/ 40, /*py*/ 30, /*pw*/ 100, /*ph*/ 50, + /*surf_w*/ 200, /*surf_h*/ 100, /*sx*/ 20, /*sy*/ 40, + /*cw*/ 60, /*ch*/ 20); + CHECK(b.x == doctest::Approx(50.0)); // 40 + 20*0.5 + CHECK(b.y == doctest::Approx(50.0)); // 30 + 40*0.5 + CHECK(b.w == doctest::Approx(30.0)); // 60*0.5 + CHECK(b.h == doctest::Approx(10.0)); // 20*0.5 + } + // Degenerate (zero surface size): no NaN — falls back to a 1:1 scale. + { + const auto b = place_child_box(5, 5, 0, 0, 0, 0, 7, 8, 9, 10); + CHECK(b.x == doctest::Approx(12.0)); // 5 + 7 + CHECK(b.y == doctest::Approx(13.0)); // 5 + 8 + } +} + +// ============================================================================ // RML compositing Wave 1: surface-element PURE CORES (ui_core.hpp). The URI // minting and the seq-gate decision predicate are pure (no wlroots/GL), so they // are doctest-ed here with nothing running — the strict-core half of the @@ -2965,3 +3054,646 @@ TEST_CASE("surface-element: live import + seq-gate + frame-done against a real c client.join(); unsetenv("WLR_HEADLESS_OUTPUTS"); } + +// ============================================================================ +// RML compositing Wave 1b: surface-TREE + INPUT-BACK + KEYBOARD-FOCUS headless +// integration test. A real client maps an xdg toplevel ROOT with a SUBSURFACE +// and an xdg POPUP; a test extension builds a SurfaceElement from the root and +// hosts it in a ui surface's <img src=root_uri>. The suite asserts (via the +// public Host::ui() path + kernel test seams, since headless has no input +// devices): (A) the subsurface + popup become per-node child <img> elements and +// frame-done reaches EVERY node (the tree-walk); (B) a pointer motion/button (+ +// a touch down) over the element forwards to the client at the EXPECTED surface- +// LOCAL coords through the element's transform (Element::Project); (C) focusing +// the element delivers a wl_keyboard enter + a forwarded key. xdg-shell is +// provided by the TEST extension (the kernel names no shell), exactly as +// ext-xdg-shell will in Wave 2. +// ============================================================================ + +namespace { + +// Server-side: a test extension that runs xdg-shell (via the kernel's wlr +// wrapper) + the subcompositor is the kernel's own (it always creates one), and +// turns the first mapped toplevel into a SurfaceElement shown in a ui surface. +class TreeTestExtension : public unbox::kernel::Extension { +public: + auto manifest() const -> const Manifest& override { return manifest_; } + + void activate(Host& host) override { + host_ = &host; + xdg_shell_ = wlr_xdg_shell_create(host.display(), 3); + if (xdg_shell_ == nullptr) { + return; + } + new_toplevel_.connect(xdg_shell_->events.new_toplevel, [this](void* data) { + on_new_toplevel(static_cast<wlr_xdg_toplevel*>(data)); + }); + new_popup_.connect(xdg_shell_->events.new_popup, [this](void* data) { + on_new_popup(static_cast<wlr_xdg_popup*>(data)); + }); + } + + // Drive the surface element + ui surface once the root toplevel maps. The ui + // surface hosts <img id=se src=root_uri> filling its box 1:1 with the + // toplevel buffer, at a NON-ZERO layout origin (proves coords are surface- + // local). `transform_deg` (set before map by the test) tilts the hosting img. + void on_map() { + if (root_surface_ == nullptr || element_ != nullptr) { + return; + } + element_ = host_->ui().create_surface_element(root_surface_); + if (element_ == nullptr) { + return; + } + std::string xform; + if (transform_deg_ != 0.0) { + xform = "#se { transform: perspective(800px) rotateY(" + + std::to_string(transform_deg_) + + "deg); transform-origin: 50% 50%; }"; + } + std::string rml = + "<rml><head><style>body{margin:0px;background-color:transparent;" + "width:200px;height:200px;} #se{display:block;position:absolute;" + "left:0px;top:0px;width:200px;height:200px;} " + + xform + "</style></head><body data-model=\"ui\">" + "<img id=\"se\" src=\"" + + element_->source_uri() + "\"/></body></rml>"; + UiSurfaceSpec spec; + spec.rml_inline = rml; + spec.x = kSurfX; + spec.y = kSurfY; + spec.width = 200; + spec.height = 200; + spec.layer = unbox::kernel::SceneLayer::overlay; + spec.visible = true; + surface_ = host_->ui().create_surface(spec); + } + + void set_transform(double deg) { transform_deg_ = deg; } + void focus() { + if (element_ != nullptr) { + element_->focus_keyboard(); + } + } + [[nodiscard]] auto has_element() const -> bool { return element_ != nullptr; } + [[nodiscard]] auto has_surface() const -> bool { return surface_ != nullptr; } + + static constexpr int kSurfX = 40; + static constexpr int kSurfY = 30; + +private: + void on_new_toplevel(wlr_xdg_toplevel* toplevel) { + wlr_xdg_surface* xdg = toplevel->base; + root_surface_ = xdg->surface; + map_.connect(xdg->surface->events.map, [this](void*) { on_map(); }); + commit_.connect(xdg->surface->events.commit, [this, xdg](void*) { + if (xdg->initial_commit) { + wlr_xdg_toplevel_set_size(xdg->toplevel, 0, 0); + } + }); + } + void on_new_popup(wlr_xdg_popup* popup) { + wlr_xdg_surface* xdg = popup->base; + popup_commit_.connect(xdg->surface->events.commit, [xdg](void*) { + if (xdg->initial_commit) { + wlr_xdg_surface_schedule_configure(xdg); + } + }); + } + + Manifest manifest_{"tree-test", Tier::standard, {}}; + Host* host_ = nullptr; + wlr_xdg_shell* xdg_shell_ = nullptr; + wlr_surface* root_surface_ = nullptr; + double transform_deg_ = 0.0; + unbox::kernel::Listener new_toplevel_, new_popup_, map_, commit_, popup_commit_; + std::unique_ptr<unbox::kernel::SurfaceElement> element_; + std::unique_ptr<UiSurface> surface_; +}; + +// Client-side: a real Wayland client that maps an xdg toplevel + a subsurface + +// an xdg popup, and records what its wl_pointer / wl_touch / wl_keyboard receive +// (so the test can assert the input-back surface-local coords). On its own thread. +struct TreeClient { + std::thread thread; + std::atomic<bool> ready{false}; // toplevel + subsurface + popup committed + std::atomic<bool> stop{false}; + std::string socket; + + // Recorded client input (atomics: read from the test thread). + std::atomic<int> ptr_enters{0}; + std::atomic<int> ptr_motions{0}; + std::atomic<int> ptr_buttons{0}; + std::atomic<int> touch_downs{0}; + std::atomic<int> kbd_enters{0}; + std::atomic<int> keys{0}; + std::atomic<double> last_ptr_x{-1.0}; + std::atomic<double> last_ptr_y{-1.0}; + std::atomic<double> last_touch_x{-1.0}; + std::atomic<double> last_touch_y{-1.0}; + // Which of our surfaces the pointer/touch entered (so we can assert it hit + // the EXPECTED node — root vs subsurface vs popup). + std::atomic<int> ptr_enter_surface{-1}; // 0=root 1=subsurface 2=popup -1=none + + explicit TreeClient(std::string sock) : socket(std::move(sock)) {} + void start() { thread = std::thread([this] { run(); }); } + void join() { + stop = true; + if (thread.joinable()) { + thread.join(); + } + } + + wl_display* dpy = nullptr; + wl_registry* registry = nullptr; + wl_compositor* compositor = nullptr; + wl_subcompositor* subcompositor = nullptr; + wl_shm* shm = nullptr; + wl_seat* seat = nullptr; + xdg_wm_base* wm_base = nullptr; + wl_pointer* pointer = nullptr; + wl_touch* touch = nullptr; + wl_keyboard* keyboard = nullptr; + + wl_surface* root = nullptr; // the toplevel surface (node 0) + wl_surface* sub = nullptr; // the subsurface (node 1) + wl_surface* pop = nullptr; // the popup surface (node 2) + xdg_surface* xsurf = nullptr; + xdg_toplevel* xtop = nullptr; + xdg_surface* xpopsurf = nullptr; + xdg_popup* xpop = nullptr; + bool configured = false; + + auto surface_index(wl_surface* s) const -> int { + if (s == root) { + return 0; + } + if (s == sub) { + return 1; + } + if (s == pop) { + return 2; + } + return -1; + } + + static auto make_buffer(wl_shm* shm, int w, int h, uint32_t argb) -> wl_buffer* { + const int stride = w * 4; + const int size = stride * h; + int fd = memfd_create("unbox-tree-test", MFD_CLOEXEC); + if (fd < 0) { + return nullptr; + } + if (ftruncate(fd, size) < 0) { + close(fd); + return nullptr; + } + auto* px = static_cast<uint32_t*>( + mmap(nullptr, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)); + if (px == MAP_FAILED) { + close(fd); + return nullptr; + } + for (int i = 0; i < w * h; ++i) { + px[i] = argb; + } + munmap(px, size); + wl_shm_pool* pool = wl_shm_create_pool(shm, fd, size); + wl_buffer* buf = wl_shm_pool_create_buffer(pool, 0, w, h, stride, WL_SHM_FORMAT_ARGB8888); + wl_shm_pool_destroy(pool); + close(fd); + return buf; + } + + // --- listeners --- + static void reg_global(void* data, wl_registry* reg, uint32_t name, const char* iface, + uint32_t ver) { + auto* self = static_cast<TreeClient*>(data); + if (std::strcmp(iface, "wl_compositor") == 0) { + self->compositor = static_cast<wl_compositor*>( + wl_registry_bind(reg, name, &wl_compositor_interface, 4)); + } else if (std::strcmp(iface, "wl_subcompositor") == 0) { + self->subcompositor = static_cast<wl_subcompositor*>( + wl_registry_bind(reg, name, &wl_subcompositor_interface, 1)); + } else if (std::strcmp(iface, "wl_shm") == 0) { + self->shm = static_cast<wl_shm*>(wl_registry_bind(reg, name, &wl_shm_interface, 1)); + } else if (std::strcmp(iface, "wl_seat") == 0) { + self->seat = static_cast<wl_seat*>( + wl_registry_bind(reg, name, &wl_seat_interface, std::min<uint32_t>(ver, 5))); + } else if (std::strcmp(iface, "xdg_wm_base") == 0) { + self->wm_base = static_cast<xdg_wm_base*>( + wl_registry_bind(reg, name, &xdg_wm_base_interface, 1)); + } + } + static void reg_remove(void*, wl_registry*, uint32_t) {} + + static void wm_ping(void*, xdg_wm_base* b, uint32_t serial) { xdg_wm_base_pong(b, serial); } + + static void xsurf_configure(void* data, xdg_surface* s, uint32_t serial) { + auto* self = static_cast<TreeClient*>(data); + xdg_surface_ack_configure(s, serial); + self->configured = true; + } + static void xtop_configure(void*, xdg_toplevel*, int32_t, int32_t, wl_array*) {} + static void xtop_close(void*, xdg_toplevel*) {} + + static void xpopsurf_configure(void* data, xdg_surface* s, uint32_t serial) { + xdg_surface_ack_configure(s, serial); + (void)data; + } + static void xpop_configure(void*, xdg_popup*, int32_t, int32_t, int32_t, int32_t) {} + static void xpop_done(void*, xdg_popup*) {} + + // pointer + static void p_enter(void* data, wl_pointer*, uint32_t, wl_surface* surf, wl_fixed_t sx, + wl_fixed_t sy) { + auto* self = static_cast<TreeClient*>(data); + ++self->ptr_enters; + self->ptr_enter_surface = self->surface_index(surf); + self->last_ptr_x = wl_fixed_to_double(sx); + self->last_ptr_y = wl_fixed_to_double(sy); + } + static void p_leave(void*, wl_pointer*, uint32_t, wl_surface*) {} + static void p_motion(void* data, wl_pointer*, uint32_t, wl_fixed_t sx, wl_fixed_t sy) { + auto* self = static_cast<TreeClient*>(data); + ++self->ptr_motions; + self->last_ptr_x = wl_fixed_to_double(sx); + self->last_ptr_y = wl_fixed_to_double(sy); + } + static void p_button(void* data, wl_pointer*, uint32_t, uint32_t, uint32_t, uint32_t) { + ++static_cast<TreeClient*>(data)->ptr_buttons; + } + static void p_axis(void*, wl_pointer*, uint32_t, uint32_t, wl_fixed_t) {} + static void p_frame(void*, wl_pointer*) {} + static void p_axis_source(void*, wl_pointer*, uint32_t) {} + static void p_axis_stop(void*, wl_pointer*, uint32_t, uint32_t) {} + static void p_axis_discrete(void*, wl_pointer*, uint32_t, int32_t) {} + static void p_axis_value120(void*, wl_pointer*, uint32_t, int32_t) {} + static void p_axis_relative_direction(void*, wl_pointer*, uint32_t, uint32_t) {} + + // touch + static void t_down(void* data, wl_touch*, uint32_t, uint32_t, wl_surface*, int32_t, + wl_fixed_t x, wl_fixed_t y) { + auto* self = static_cast<TreeClient*>(data); + ++self->touch_downs; + self->last_touch_x = wl_fixed_to_double(x); + self->last_touch_y = wl_fixed_to_double(y); + } + static void t_up(void*, wl_touch*, uint32_t, uint32_t, int32_t) {} + static void t_motion(void*, wl_touch*, uint32_t, int32_t, wl_fixed_t, wl_fixed_t) {} + static void t_frame(void*, wl_touch*) {} + static void t_cancel(void*, wl_touch*) {} + static void t_shape(void*, wl_touch*, int32_t, wl_fixed_t, wl_fixed_t) {} + static void t_orientation(void*, wl_touch*, int32_t, wl_fixed_t) {} + + // keyboard + static void k_keymap(void*, wl_keyboard*, uint32_t, int32_t fd, uint32_t) { + if (fd >= 0) { + close(fd); + } + } + static void k_enter(void* data, wl_keyboard*, uint32_t, wl_surface*, wl_array*) { + ++static_cast<TreeClient*>(data)->kbd_enters; + } + static void k_leave(void*, wl_keyboard*, uint32_t, wl_surface*) {} + static void k_key(void* data, wl_keyboard*, uint32_t, uint32_t, uint32_t, uint32_t) { + ++static_cast<TreeClient*>(data)->keys; + } + static void k_mods(void*, wl_keyboard*, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {} + static void k_repeat(void*, wl_keyboard*, int32_t, int32_t) {} + + std::atomic<uint32_t> seat_caps{0}; + static void seat_caps_cb(void* data, wl_seat*, uint32_t caps) { + static_cast<TreeClient*>(data)->seat_caps = caps; + } + static void seat_name_cb(void*, wl_seat*, const char*) {} + + // Bind pointer/touch/keyboard only once the seat advertises the capability + // (newer libwayland enforces it). The kernel advertises POINTER|TOUCH| + // KEYBOARD via the ui_add_test_keyboard seam before the client connects. + void bind_seat() { + static const wl_seat_listener sl = {seat_caps_cb, seat_name_cb}; + wl_seat_add_listener(seat, &sl, this); + wl_display_roundtrip(dpy); // deliver the capabilities event + static const wl_pointer_listener pl = { + p_enter, p_leave, p_motion, p_button, p_axis, p_frame, p_axis_source, + p_axis_stop, p_axis_discrete, p_axis_value120, p_axis_relative_direction}; + static const wl_touch_listener tl = {t_down, t_up, t_motion, t_frame, + t_cancel, t_shape, t_orientation}; + static const wl_keyboard_listener kl = {k_keymap, k_enter, k_leave, k_key, k_mods, k_repeat}; + const uint32_t caps = seat_caps.load(); + if ((caps & WL_SEAT_CAPABILITY_POINTER) != 0) { + pointer = wl_seat_get_pointer(seat); + wl_pointer_add_listener(pointer, &pl, this); + } + if ((caps & WL_SEAT_CAPABILITY_TOUCH) != 0) { + touch = wl_seat_get_touch(seat); + wl_touch_add_listener(touch, &tl, this); + } + if ((caps & WL_SEAT_CAPABILITY_KEYBOARD) != 0) { + keyboard = wl_seat_get_keyboard(seat); + wl_keyboard_add_listener(keyboard, &kl, this); + } + } + + void run() { + dpy = wl_display_connect(socket.c_str()); + if (dpy == nullptr) { + return; + } + registry = wl_display_get_registry(dpy); + static const wl_registry_listener reg_l = {reg_global, reg_remove}; + wl_registry_add_listener(registry, ®_l, this); + wl_display_roundtrip(dpy); // bind globals + // wl_seat capabilities arrive async; roundtrip again so get_keyboard works + // once the kernel advertises a (test) keyboard. + if (compositor == nullptr || shm == nullptr || subcompositor == nullptr || + wm_base == nullptr || seat == nullptr) { + wl_display_disconnect(dpy); + return; + } + static const xdg_wm_base_listener wm_l = {wm_ping}; + xdg_wm_base_add_listener(wm_base, &wm_l, this); + bind_seat(); + + // --- toplevel root --- + root = wl_compositor_create_surface(compositor); + xsurf = xdg_wm_base_get_xdg_surface(wm_base, root); + static const xdg_surface_listener xs_l = {xsurf_configure}; + xdg_surface_add_listener(xsurf, &xs_l, this); + xtop = xdg_surface_get_toplevel(xsurf); + static const xdg_toplevel_listener xt_l = {xtop_configure, xtop_close}; + xdg_toplevel_add_listener(xtop, &xt_l, this); + wl_surface_commit(root); // initial commit -> server sends configure + while (!configured && wl_display_dispatch(dpy) != -1) { + } + wl_buffer* root_buf = make_buffer(shm, 200, 200, 0xff2060c0); + wl_surface_attach(root, root_buf, 0, 0); + wl_surface_damage(root, 0, 0, 200, 200); + + // --- subsurface (node 1): 40x40 at tree offset (20,30) --- + sub = wl_compositor_create_surface(compositor); + wl_subsurface* subsurface = wl_subcompositor_get_subsurface(subcompositor, sub, root); + wl_subsurface_set_position(subsurface, 20, 30); + wl_subsurface_set_desync(subsurface); + wl_buffer* sub_buf = make_buffer(shm, 40, 40, 0xff60c020); + wl_surface_attach(sub, sub_buf, 0, 0); + wl_surface_damage(sub, 0, 0, 40, 40); + wl_surface_commit(sub); + wl_surface_commit(root); // apply the subsurface + wl_display_roundtrip(dpy); + + // --- popup (node 2): a 60x50 popup positioned at (80,90) off the root --- + pop = wl_compositor_create_surface(compositor); + xpopsurf = xdg_wm_base_get_xdg_surface(wm_base, pop); + static const xdg_surface_listener xps_l = {xpopsurf_configure}; + xdg_surface_add_listener(xpopsurf, &xps_l, this); + xdg_positioner* pos = xdg_wm_base_create_positioner(wm_base); + xdg_positioner_set_size(pos, 60, 50); + xdg_positioner_set_anchor_rect(pos, 80, 90, 1, 1); + xpop = xdg_surface_get_popup(xpopsurf, xsurf, pos); + xdg_positioner_destroy(pos); + static const xdg_popup_listener xp_l = {xpop_configure, xpop_done}; + xdg_popup_add_listener(xpop, &xp_l, this); + static const xdg_surface_listener xps2 = {xpopsurf_configure}; + (void)xps2; + wl_surface_commit(pop); // initial popup commit -> configure + wl_display_roundtrip(dpy); + wl_buffer* pop_buf = make_buffer(shm, 60, 50, 0xff2080e0); + wl_surface_attach(pop, pop_buf, 0, 0); + wl_surface_damage(pop, 0, 0, 60, 50); + wl_surface_commit(pop); + + // Map the root last so the extension's map handler builds the element with + // the subsurface + popup already in the tree. + wl_surface_commit(root); + wl_display_flush(dpy); + ready = true; + + while (!stop) { + if (wl_display_dispatch_pending(dpy) == -1) { + break; + } + wl_display_flush(dpy); + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + } + + // Teardown: destroy every proxy before disconnect (asan leak-clean). + if (pop_buf != nullptr) { + wl_buffer_destroy(pop_buf); + } + if (sub_buf != nullptr) { + wl_buffer_destroy(sub_buf); + } + if (root_buf != nullptr) { + wl_buffer_destroy(root_buf); + } + if (xpop != nullptr) { + xdg_popup_destroy(xpop); + } + if (xpopsurf != nullptr) { + xdg_surface_destroy(xpopsurf); + } + if (pop != nullptr) { + wl_surface_destroy(pop); + } + if (subsurface != nullptr) { + wl_subsurface_destroy(subsurface); + } + if (sub != nullptr) { + wl_surface_destroy(sub); + } + if (xtop != nullptr) { + xdg_toplevel_destroy(xtop); + } + if (xsurf != nullptr) { + xdg_surface_destroy(xsurf); + } + if (root != nullptr) { + wl_surface_destroy(root); + } + if (pointer != nullptr) { + wl_pointer_destroy(pointer); + } + if (touch != nullptr) { + wl_touch_destroy(touch); + } + if (keyboard != nullptr) { + wl_keyboard_destroy(keyboard); + } + if (seat != nullptr) { + wl_seat_destroy(seat); + } + if (wm_base != nullptr) { + xdg_wm_base_destroy(wm_base); + } + if (subcompositor != nullptr) { + wl_subcompositor_destroy(subcompositor); + } + if (compositor != nullptr) { + wl_compositor_destroy(compositor); + } + if (shm != nullptr) { + wl_shm_destroy(shm); + } + if (registry != nullptr) { + wl_registry_destroy(registry); + } + wl_display_flush(dpy); + wl_display_disconnect(dpy); + } +}; + +} // namespace + +TEST_CASE("surface-element: tree (subsurface + popup) + input-back + keyboard focus") { + setenv("WLR_BACKENDS", "headless", 1); + setenv("WLR_RENDERER", "gles2", 1); + setenv("WLR_HEADLESS_OUTPUTS", "1", 1); + unsetenv("UNBOX_UI_SUBSTRATE_FORCE_SHM"); + + auto server = unbox::kernel::Server::create({}); + auto* ext = new TreeTestExtension(); + server->install(std::unique_ptr<unbox::kernel::Extension>(ext)); + server->activate_extensions(); + // A virtual keyboard on the seat (headless has none) so focus_keyboard can + // deliver a wl_keyboard enter; added before the client binds the seat. + server->ui_add_test_keyboard(); + + TreeClient client(server->socket_name()); + client.start(); + + const bool ok = pump_until_se(*server, [&] { return client.ready.load(); }, 800) && + pump_until_se(*server, [&] { return ext->has_surface(); }, 800); + if (!ok || !ext->has_element() || server->ui_frame_count() == 0) { + // No GL path on this box, or the client could not map: skip (mirrors the + // other GL-gated tests). The pure-core input math is asserted separately. + client.join(); + return; + } + pump(*server, 40); // let the tree re-walk + child <img> placement settle + + // (A) TREE: the root + the subsurface + the popup compose as per-node <img> + // elements (root authored; subsurface + popup created by the substrate). + INFO("img count = ", server->ui_element_count("img")); + CHECK(server->ui_element_count("img") >= 3); + + // (A) FRAME-DONE walks the WHOLE tree: the count climbs by MORE than one per + // composited frame (root + subsurface + popup each get a frame-done). + const int fd0 = server->ui_surface_element_frame_done_count(); + pump(*server, 20); + const int fd1 = server->ui_surface_element_frame_done_count(); + CHECK(fd1 > fd0); + CHECK(fd1 - fd0 >= 3); // >= one per node (root + subsurface + popup) + + // (B) INPUT-BACK pointer: drive a motion at a known layout point over the + // ROOT region of the element. Surface at (40,30); the root img fills 200x200 + // 1:1 with the 200x200 buffer, so layout (40+50, 30+60) => surface-local + // (50,60) on the root client surface. + using DK = unbox::kernel::Server::UiTouchOverride; // (unused; keep includes warm) + (void)DK::automatic; + server->ui_route_pointer_motion_for_test(TreeTestExtension::kSurfX + 50.0, + TreeTestExtension::kSurfY + 60.0, 1000); + pump_until_se(*server, [&] { return client.ptr_enters.load() > 0; }, 200); + CHECK(client.ptr_enters.load() > 0); + CHECK(client.ptr_enter_surface.load() == 0); // hit the ROOT node + CHECK(client.last_ptr_x.load() == doctest::Approx(50.0).epsilon(0.05)); + CHECK(client.last_ptr_y.load() == doctest::Approx(60.0).epsilon(0.05)); + + // (B) INPUT-BACK pointer over the SUBSURFACE node: the subsurface is 40x40 at + // tree offset (20,30); a point at layout (40+30, 30+45) => surface-local + // (30,45) on the root, which lands inside the subsurface (its <img> spans + // (20,30)..(60,70)). The pick must hit the SUBSURFACE node and report coords + // LOCAL TO THE SUBSURFACE: (30-20, 45-30) = (10,15). + server->ui_route_pointer_motion_for_test(TreeTestExtension::kSurfX + 30.0, + TreeTestExtension::kSurfY + 45.0, 1010); + pump(*server, 5); + CHECK(client.ptr_enter_surface.load() == 1); // the subsurface node + CHECK(client.last_ptr_x.load() == doctest::Approx(10.0).epsilon(0.1)); + CHECK(client.last_ptr_y.load() == doctest::Approx(15.0).epsilon(0.1)); + + // (B) INPUT-BACK button: a press over the root forwards a wl_pointer button. + const int btn0 = client.ptr_buttons.load(); + server->ui_route_pointer_button_for_test(TreeTestExtension::kSurfX + 50.0, + TreeTestExtension::kSurfY + 60.0, true, 1020); + server->ui_route_pointer_button_for_test(TreeTestExtension::kSurfX + 50.0, + TreeTestExtension::kSurfY + 60.0, false, 1021); + pump_until_se(*server, [&] { return client.ptr_buttons.load() > btn0; }, 200); + CHECK(client.ptr_buttons.load() > btn0); + + // (B) INPUT-BACK touch: a touch-down over the root forwards a wl_touch down at + // surface-local coords. + server->ui_route_touch_down_for_test(7, TreeTestExtension::kSurfX + 50.0, + TreeTestExtension::kSurfY + 60.0, 1030); + pump_until_se(*server, [&] { return client.touch_downs.load() > 0; }, 200); + CHECK(client.touch_downs.load() > 0); + CHECK(client.last_touch_x.load() == doctest::Approx(50.0).epsilon(0.05)); + CHECK(client.last_touch_y.load() == doctest::Approx(60.0).epsilon(0.05)); + server->ui_route_touch_up_for_test(7, 1031); + + // (C) KEYBOARD FOCUS: focusing the element delivers a wl_keyboard enter, then + // a forwarded key reaches the client. + ext->focus(); + pump_until_se(*server, [&] { return client.kbd_enters.load() > 0; }, 200); + CHECK(client.kbd_enters.load() > 0); + const int keys0 = client.keys.load(); + server->ui_send_key_for_test(/*KEY_A*/ 30, true); + server->ui_send_key_for_test(/*KEY_A*/ 30, false); + pump_until_se(*server, [&] { return client.keys.load() > keys0; }, 200); + CHECK(client.keys.load() > keys0); + + client.join(); + unsetenv("WLR_HEADLESS_OUTPUTS"); +} + +// ============================================================================ +// RML compositing Wave 1b: surface-element input-back through a TRANSFORMED +// hosting element. A point on the rotation AXIS (the element centre) projects to +// the surface centre regardless of the rotateY, so we can assert EXACT surface- +// local coords even under a 3D transform (Element::Project inverts it). This is +// the live analogue of the pure-core round-trip test above. +// ============================================================================ + +TEST_CASE("surface-element: input-back through a 3D-transformed hosting element") { + setenv("WLR_BACKENDS", "headless", 1); + setenv("WLR_RENDERER", "gles2", 1); + setenv("WLR_HEADLESS_OUTPUTS", "1", 1); + unsetenv("UNBOX_UI_SUBSTRATE_FORCE_SHM"); + + auto server = unbox::kernel::Server::create({}); + auto* ext = new TreeTestExtension(); + ext->set_transform(35.0); // perspective + rotateY(35deg) about 50% origin + server->install(std::unique_ptr<unbox::kernel::Extension>(ext)); + server->activate_extensions(); + // Advertise seat pointer/touch capabilities (headless adds no input devices, + // so update_seat_capabilities never runs); the seam does this as a side effect. + server->ui_add_test_keyboard(); + + TreeClient client(server->socket_name()); + client.start(); + + const bool ok = pump_until_se(*server, [&] { return client.ready.load(); }, 800) && + pump_until_se(*server, [&] { return ext->has_surface(); }, 800); + if (!ok || !ext->has_element() || server->ui_frame_count() == 0) { + client.join(); + return; + } + pump(*server, 40); + + // The element box is 200x200 at layout (40,30); its centre is layout + // (40+100, 30+100). Under perspective+rotateY about the 50% origin the centre + // sits on the rotation axis, so it projects to surface-local (100,100) — the + // transform-aware Element::Project recovers the centre exactly. (An untilted + // build would give the same answer; the point is the tilt does NOT shift the + // axis point, proving the projection — not a naive axis-aligned map — runs.) + server->ui_route_pointer_motion_for_test(TreeTestExtension::kSurfX + 100.0, + TreeTestExtension::kSurfY + 100.0, 2000); + pump_until_se(*server, [&] { return client.ptr_enters.load() > 0; }, 200); + REQUIRE(client.ptr_enters.load() > 0); + CHECK(client.ptr_enter_surface.load() == 0); // the root node + CHECK(client.last_ptr_x.load() == doctest::Approx(100.0).epsilon(0.03)); + CHECK(client.last_ptr_y.load() == doctest::Approx(100.0).epsilon(0.03)); + + client.join(); + unsetenv("WLR_HEADLESS_OUTPUTS"); +} @@ -26,16 +26,24 @@ out of the shipped binary. **CONTRACT DECISION (user): RCSS is the single source of truth for ALL layout + animation; C++ drives the document via a TYPED substrate API.** PHASE 2 on `feat/rml-compositing` (off main; spike sources carried as in-tree reference, `build_by_default:false`, deleted when the waves land). -**Wave 1 DONE + verified**: kernel `SurfaceElement` (live sibling of `Preview`) — -zero-copy seq-gated import, frame-callback duty, dirty-gate; public contract in -`ui.hpp` (`create_surface_element(wlr_surface*)`); kernel suite + asan green -(test-only `wayland-client` dep accepted, scoped to kernel-tests). Wave plan -refined: **Wave 1** = the live primitive (done); **Wave 1b** = input-back -(pick→surface-local→wl_seat via `Element::Project`) + subsurface/popup child -trees; **Wave 2** = ext-xdg-shell (`Toplevel::wl_surface()`, retire scene -compositing) + ext-layer-shell; **Wave 3** = NEW `ext-window-field` (window list + -RCSS layout); **Wave 4** = ext-stage-dock; **Wave 5** = damage limiting (Option B) -+ scanout bypass. NEXT ACTION: **Wave 1b** (kernel input-back + surface trees). +**Waves 1 + 1b DONE + verified** (kernel): `SurfaceElement` (live sibling of +`Preview`) — zero-copy seq-gated import, frame-callback duty, dirty-gate, public +`create_surface_element(wlr_surface*)`; **surface trees** (subsurface/popup child +elements, whole-tree frame-done, parent-relative placement); **input-back** +(pointer/touch → surface-local via `Element::Project`, pure inversion core +doctested) + `SurfaceElement::focus_keyboard()` primitive. Kernel suite (72c/375a) ++ asan green; test-only `wayland-client`/xdg-shell-client codegen accepted +(kernel-tests scope). Wave plan: **W1/W1b** done; **W2** = ext-xdg-shell (ADD +`Toplevel::wl_surface()`; keep scene compositing for now — retire behind the flag +in W3) + ext-layer-shell (expose its surface); **W3** = NEW `ext-window-field` +(window list + RCSS layout + focus policy via `focus_keyboard()`, flips the flag); +**W4** = ext-stage-dock; **W5** = damage limiting (Option B) + scanout bypass. +NEXT ACTION: **Wave 2** (ext-xdg-shell + ext-layer-shell, disjoint/parallel). +TRACKED BUG (pre-existing, baseline-confirmed, NOT from these waves): the +`ext-stage-dock-glue` + `ext-xdg-shell-client` test suites abort at teardown on a +wlroots `wl_list_empty(...commit.listener_list)` assertion (a listener-lifetime +teardown-order bug in those units' tests) — fix `ext-xdg-shell-client` in W2, +`ext-stage-dock-glue` in W4. Tiling (slice 7) is DEFERRED behind this (becomes RCSS over surface elements; pure layout core in `notes/tiling-spec.md` carries over). Stage dock (slice 10) real-seat feel check is paused under this pivot. |
