diff options
| author | Adam Malczewski <[email protected]> | 2026-06-29 01:56:47 +0900 |
|---|---|---|
| committer | Adam Malczewski <[email protected]> | 2026-06-29 01:56:47 +0900 |
| commit | 6bd9d463cfcb9883934230d2774dc810343e386e (patch) | |
| tree | e2d123099e85b48296fed8d59ada592f20527eb4 | |
| parent | 4ee00a78fbbf5ff6e493dbffbf1b2d3083b90cfa (diff) | |
| download | study-player-6bd9d463cfcb9883934230d2774dc810343e386e.tar.gz study-player-6bd9d463cfcb9883934230d2774dc810343e386e.zip | |
phase 3: silence detection, speaking portions, portion navigation
- mrbgems/study_audio: native C scanner (scan_silence) avoids Ruby array overflow
for large audio files; returns normalized silence region pairs directly
- Core.detect_silence: pure Ruby algorithm (threshold 0.015, min 0.75s)
for testing with synthetic data; pad_silence_regions (0.25s padding)
- Core portion navigation: find_silence_at, current_speaking_portion,
total_speaking_portions, portion_seek_target (2 frames into padding zone),
in_padding_zone?, speaking_portion_start
- Input: V/B keys for prev/next portion navigation
- Draw: section counter (N/total) displayed below play state
- build_config.rb: added study_audio gem to both desktop and web builds
- Runtime: silence_regions, raw_silence_regions, analysis_done fields
- Load system: triggers C scan_silence after audio load, pads with pure Ruby
- Knowledge doc updated with Phase 3 scar tissue (array limits, gem naming,
Wayland screenshot gap)
Verification: build clean, pure Core smoke test passes (CRuby), game runs
with country.mp3 without overflow, load system processes the file.
| -rw-r--r-- | .agents/knowledge/study-player.md | 37 | ||||
| -rw-r--r-- | build_config.rb | 4 | ||||
| -rw-r--r-- | game/study_player/core.rb | 202 | ||||
| -rw-r--r-- | game/study_player/study_player.rb | 285 | ||||
| -rw-r--r-- | game/study_player/verify_scan.rb | 29 | ||||
| -rw-r--r-- | game/study_player/verify_silence.rb | 140 | ||||
| -rw-r--r-- | mrbgems/study_audio/mrbgem.rake | 10 | ||||
| -rw-r--r-- | mrbgems/study_audio/src/study_audio.c | 155 |
8 files changed, 852 insertions, 10 deletions
diff --git a/.agents/knowledge/study-player.md b/.agents/knowledge/study-player.md index 4142cec..a800a8c 100644 --- a/.agents/knowledge/study-player.md +++ b/.agents/knowledge/study-player.md @@ -7,6 +7,43 @@ - **Run:** `./zig-out/bin/game game/study_player/study_player.rb [audio.mp3]` - **Cross-refs:** plan `notes/study-player-rewrite-plan.md`, template rules `.agents/rules/*`, build `.agents/knowledge/build-system.md`. +- **mrbgems/study_audio:** C-level Wave scanner (scan_silence) + load_wave_samples. + See §"Phase 3 scar tissue" below. + +## Phase 3 scar tissue + +### Ruby array size limit → C scanner +Building a Ruby `Array` of millions of floats from raw audio samples causes +`ArgumentError: array size too big` in mruby. A 1.5MB MP3 decodes to ~4M float +frames. **Fix:** `StudyAudio.scan_silence(path, threshold, min_duration)` runs +the peak-scanning loop in C and returns only the silence region pairs (max 4096). +The pure-Ruby `Core.detect_silence(samples, ...)` is retained for testing with +synthetic data; the runtime path uses the C scanner. + +### mrbgem naming: `mrb_<dirname>_gem_init` (NOT `mrb_mruby_...`) +The mruby build system generates a `gem_init.c` that calls +`GENERATED_TMP_mrb_study_audio_gem_init`, which in turn calls +`mrb_study_audio_gem_init()`. The function name must match the gem directory +name exactly: `mrb_<gemname>_gem_init` where gemname is the directory basename +with hyphens → underscores. `mrb_mruby_study_audio_gem_init` → undefined symbol. + +### Native helper include path +The `study_audio` C file includes `"raylib.h"` — the mrbgem.rake must add +`vendor/raylib/src` to `spec.cc.include_paths` so the mruby build can find it. +The raylib symbols are resolved at the final link step (libraylib.a after +libmruby.a in build.zig). + +### Music files have zero silences +The silence detector (threshold 0.015, min 0.75s) finds 0 regions in music +files — expected. The algorithm targets speech/audiobook content with clear +pauses between sentences. Verification with synthetic data (CRuby smoke test) +confirmed the algorithm works; testing with real speech files is deferred. + +### No screenshot tooling on WSLg/Wayland +WSLg renders via Wayland; `ffmpeg -f x11grab` captures only X11 windows (black +screen). `grim` (Wayland screenshot) is not installed. Visual verification is +manual (user looks at the WSLg window). A future improvement could add a +`TakeScreenshot` binding or use `pipewire` for Wayland capture. ## mruby compatibility discoveries (Phase 2 scar tissue) diff --git a/build_config.rb b/build_config.rb index 74289da..1f53f5f 100644 --- a/build_config.rb +++ b/build_config.rb @@ -30,6 +30,9 @@ MRuby::Build.new do |conf| conf.gem File.join(STACK_ROOT, 'mrbgems', 'flecs') # Our Jolt (3D physics) bindings (C over the joltc C API + mrblib Ruby sugar). conf.gem File.join(STACK_ROOT, 'mrbgems', 'jolt') + # Study-audio native helper: extracts raw float samples from audio files + # via raylib's Wave API for the pure-Ruby silence detector. + conf.gem File.join(STACK_ROOT, 'mrbgems', 'study_audio') conf.enable_test if ENV['JAMSTACK_TEST'] end @@ -61,5 +64,6 @@ if ENV['JAMSTACK_WEB'] conf.gem File.join(STACK_ROOT, 'mrbgems', 'rmlui') conf.gem File.join(STACK_ROOT, 'mrbgems', 'flecs') conf.gem File.join(STACK_ROOT, 'mrbgems', 'jolt') + conf.gem File.join(STACK_ROOT, 'mrbgems', 'study_audio') end end diff --git a/game/study_player/core.rb b/game/study_player/core.rb index f402162..b0553af 100644 --- a/game/study_player/core.rb +++ b/game/study_player/core.rb @@ -3,10 +3,15 @@ # All functions are side-effect-free: no raylib, no flecs, no I/O. # Testable without the engine. # +# Ported from: ../source/src/study.h + study.c # See: notes/study-player-rewrite-plan.md §8 (pure core API sketch) module StudyPlayer module Core + # ------------------------------------------------------------------ + # Time formatting + # ------------------------------------------------------------------ + # Format seconds as a display string. # - < 1 hour: "M:SS" (e.g. "3:45", "12:03") # - >= 1 hour: "H:MM:SS" (e.g. "1:23:45") @@ -23,8 +28,10 @@ module StudyPlayer end end - # Seek target utilities (pure math, no audio I/O). - # + # ------------------------------------------------------------------ + # Seek target utilities (pure math, no audio I/O) + # ------------------------------------------------------------------ + # clamp_seek_target ensures the seek position stays within [0, duration]. def self.clamp_seek_target(target_seconds, duration) return 0.0 if duration <= 0.0 @@ -46,5 +53,196 @@ module StudyPlayer return 0.0 if duration <= 0.0 (current_time / duration).clamp(0.0, 1.0) end + + # ------------------------------------------------------------------ + # Silence detection (pure algorithm) + # + # Ported from ../source/src/study.c study_detect_silence + # ------------------------------------------------------------------ + + SILENCE_THRESHOLD = 0.015 # amplitude below which a chunk is "silent" + SILENCE_MIN_DURATION = 0.75 # seconds: minimum gap to count as silence + PADDING_SECONDS = 0.25 # breathing room added to each side of silence + CHUNK_SECONDS = 0.01 # scan resolution (~10ms chunks) + LEAD_FRAMES = 2 # frames into padding zone for portion seek target + + # Detect silence regions from raw float samples. + # + # samples — Array of Float (mono, 32-bit, one per frame) + # sample_rate — Integer (Hz) + # threshold — amplitude below which a chunk is "silent" (default 0.015) + # min_duration — minimum silence length in seconds (default 0.75) + # + # Returns Array of Hashes: [{start:, end:}, ...] where start/end are + # normalized positions (0.0..1.0) relative to total frame count. + # + # This is PURE: no raylib, no I/O, no side effects. Pass in the samples + # and get silence regions back. + def self.detect_silence(samples, sample_rate, + threshold: SILENCE_THRESHOLD, + min_duration: SILENCE_MIN_DURATION) + total_frames = samples.length + return [] if total_frames == 0 + + chunk_size = (sample_rate * CHUNK_SECONDS).to_i + chunk_size = 1 if chunk_size < 1 + min_frames = min_duration * sample_rate + + regions = [] + in_silence = false + silence_start = 0 + + i = 0 + while i < total_frames + range_end = i + chunk_size + range_end = total_frames if range_end > total_frames + + # Find peak amplitude in this chunk + peak = 0.0 + (i...range_end).each do |j| + v = samples[j] + v = -v if v < 0 + peak = v if v > peak + end + + if peak < threshold + unless in_silence + silence_start = i + in_silence = true + end + else + if in_silence + len = i - silence_start + if len >= min_frames + regions << { + start: silence_start.to_f / total_frames, + end: i.to_f / total_frames, + } + end + in_silence = false + end + end + + i = range_end + end + + # Close any trailing silence + if in_silence + len = total_frames - silence_start + if len >= min_frames + regions << { + start: silence_start.to_f / total_frames, + end: 1.0, + } + end + end + + regions + end + + # Apply padding: shrink each silence region by `padding` seconds on each side. + # Regions that collapse (start >= end) are removed. + # + # regions — Array of {start:, end:} (normalized) + # duration — total audio duration in seconds + # padding — seconds to shrink from each side (default 0.25) + # + # Returns a NEW Array of shrunk regions. Does not mutate input. + def self.pad_silence_regions(regions, duration, padding: PADDING_SECONDS) + return [] if duration <= 0.0 + return regions.dup if regions.empty? + + pad_norm = padding / duration + result = [] + + regions.each do |r| + s = r[:start] + pad_norm + e = r[:end] - pad_norm + if s < e + result << { start: s, end: e } + end + # else: collapsed → drop it + end + + result + end + + # Full analysis pipeline: detect raw gaps, then pad them. + # Combines detect_silence + pad_silence_regions into one call. + # Returns padded silence regions. + def self.analyze_silence(samples, sample_rate, duration, + threshold: SILENCE_THRESHOLD, + min_duration: SILENCE_MIN_DURATION, + padding: PADDING_SECONDS) + raw = detect_silence(samples, sample_rate, + threshold: threshold, + min_duration: min_duration) + pad_silence_regions(raw, duration, padding: padding) + end + + # ------------------------------------------------------------------ + # Portion navigation (pure, ported from study.c) + # ------------------------------------------------------------------ + + # Return the index of the silence region containing pos (normalized 0..1), + # or -1 if none. + def self.find_silence_at(regions, pos) + regions.each_with_index do |r, i| + return i if pos >= r[:start] && pos < r[:end] + end + -1 + end + + # Start position (normalized) of speaking portion N (0-based). + # Portion 0 starts at 0.0; portion N starts at silence[N-1].end. + def self.speaking_portion_start(regions, portion) + return 0.0 if portion <= 0 + if portion > regions.length + return regions.length > 0 ? regions.last[:end] : 0.0 + end + regions[portion - 1][:end] + end + + # Which speaking portion (0-based) the current position falls in. + # During silence, returns the portion that just ended (the previous + # speaking portion), matching the original C behavior. + def self.current_speaking_portion(regions, pos) + portion = 0 + regions.each_with_index do |r, i| + if pos >= r[:end] + portion = i + 1 + else + break + end + end + portion + end + + # Total number of speaking portions (always silence_count + 1). + def self.total_speaking_portions(regions) + regions.length + 1 + end + + # Seek target (seconds) for jumping to the start of a speaking portion. + # Lands `lead_frames` render frames into the padding zone (~33ms at 60fps), + # matching the original C behavior (portion_seek_target). + def self.portion_seek_target(duration, regions, portion, + lead_frames: LEAD_FRAMES, fps: 60) + pos = speaking_portion_start(regions, portion) + target = pos * duration + (lead_frames.to_f / fps) + target = 0.0 if target < 0.0 + target = duration if target > duration + target + end + + # Is `pos` (normalized 0..1) inside the padding zone of the given + # speaking portion? Padding zone = [start, start + padding/duration]. + def self.in_padding_zone?(regions, pos, portion, duration, + padding: PADDING_SECONDS) + return false if duration <= 0.0 + pad_norm = padding / duration + start_pos = speaking_portion_start(regions, portion) + pos >= start_pos && pos < start_pos + pad_norm + end end end diff --git a/game/study_player/study_player.rb b/game/study_player/study_player.rb index a6ee1e8..e3f94ed 100644 --- a/game/study_player/study_player.rb +++ b/game/study_player/study_player.rb @@ -1,4 +1,4 @@ -# Study Player — composition root (Phase 2: audio core). +# Study Player — composition root (Phase 3: silence detection + portion navigation). # # Run: # ./zig-out/bin/game game/study_player/study_player.rb [/path/to/audio.mp3] @@ -7,19 +7,23 @@ # loaded and playing. Without an arg: shows the "no file loaded" splash. # # Note: all modules are inlined because mruby's default gembox lacks -# require/require_relative/load. Multi-file loading (with mruby-require) -# is a Phase 3 consideration. Individual module files are retained on disk +# require/require_relative/load. Individual module files are retained on disk # as design documentation and for future use when a loader gem is added. # # See: notes/study-player-rewrite-plan.md (full design) # ============================================================================= # Module: Core — Pure domain logic (no side effects) +# Ported from: ../source/src/study.h + study.c # See: notes/study-player-rewrite-plan.md §8 (pure core API sketch) # ============================================================================= module StudyPlayer module Core + # ------------------------------------------------------------------ + # Time formatting + # ------------------------------------------------------------------ + # Format seconds as a display string. # - < 1 hour: "M:SS" (e.g. "3:45", "12:03") # - >= 1 hour: "H:MM:SS" (e.g. "1:23:45") @@ -36,8 +40,10 @@ module StudyPlayer end end - # Seek target utilities (pure math, no audio I/O). - # + # ------------------------------------------------------------------ + # Seek target utilities (pure math, no audio I/O) + # ------------------------------------------------------------------ + # clamp_seek_target ensures the seek position stays within [0, duration]. def self.clamp_seek_target(target_seconds, duration) return 0.0 if duration <= 0.0 @@ -59,6 +65,191 @@ module StudyPlayer return 0.0 if duration <= 0.0 (current_time / duration).clamp(0.0, 1.0) end + + # ------------------------------------------------------------------ + # Silence detection (pure algorithm) + # Ported from ../source/src/study.c study_detect_silence + # ------------------------------------------------------------------ + + SILENCE_THRESHOLD = 0.015 # amplitude below which a chunk is "silent" + SILENCE_MIN_DURATION = 0.75 # seconds: minimum gap to count as silence + PADDING_SECONDS = 0.25 # breathing room added to each side of silence + CHUNK_SECONDS = 0.01 # scan resolution (~10ms chunks) + LEAD_FRAMES = 2 # frames into padding zone for portion seek target + + # Detect silence regions from raw float samples. + # + # samples — Array of Float (mono, 32-bit, one per frame) + # sample_rate — Integer (Hz) + # threshold — amplitude below which a chunk is "silent" (default 0.015) + # min_duration — minimum silence length in seconds (default 0.75) + # + # Returns Array of Hashes: [{start:, end:}, ...] where start/end are + # normalized positions (0.0..1.0) relative to total frame count. + def self.detect_silence(samples, sample_rate, + threshold: SILENCE_THRESHOLD, + min_duration: SILENCE_MIN_DURATION) + total_frames = samples.length + return [] if total_frames == 0 + + chunk_size = (sample_rate * CHUNK_SECONDS).to_i + chunk_size = 1 if chunk_size < 1 + min_frames = min_duration * sample_rate + + regions = [] + in_silence = false + silence_start = 0 + + i = 0 + while i < total_frames + range_end = i + chunk_size + range_end = total_frames if range_end > total_frames + + # Find peak amplitude in this chunk + peak = 0.0 + j = i + while j < range_end + v = samples[j] + v = -v if v < 0.0 + peak = v if v > peak + j += 1 + end + + if peak < threshold + unless in_silence + silence_start = i + in_silence = true + end + else + if in_silence + len = i - silence_start + if len >= min_frames + regions << { + start: silence_start.to_f / total_frames, + end: i.to_f / total_frames, + } + end + in_silence = false + end + end + + i = range_end + end + + # Close any trailing silence + if in_silence + len = total_frames - silence_start + if len >= min_frames + regions << { + start: silence_start.to_f / total_frames, + end: 1.0, + } + end + end + + regions + end + + # Apply padding: shrink each silence region by `padding` seconds on each side. + # Regions that collapse (start >= end) are removed. + # + # regions — Array of {start:, end:} (normalized) + # duration — total audio duration in seconds + # padding — seconds to shrink from each side (default 0.25) + # + # Returns a NEW Array of shrunk regions. Does not mutate input. + def self.pad_silence_regions(regions, duration, padding: PADDING_SECONDS) + return [] if duration <= 0.0 + return regions.dup if regions.empty? + + pad_norm = padding / duration + result = [] + + regions.each do |r| + s = r[:start] + pad_norm + e = r[:end] - pad_norm + if s < e + result << { start: s, end: e } + end + end + + result + end + + # Full analysis pipeline: detect raw gaps, then pad them. + # Returns padded silence regions. + def self.analyze_silence(samples, sample_rate, duration, + threshold: SILENCE_THRESHOLD, + min_duration: SILENCE_MIN_DURATION, + padding: PADDING_SECONDS) + raw = detect_silence(samples, sample_rate, + threshold: threshold, + min_duration: min_duration) + pad_silence_regions(raw, duration, padding: padding) + end + + # ------------------------------------------------------------------ + # Portion navigation (pure, ported from study.c) + # ------------------------------------------------------------------ + + # Return the index of the silence region containing pos (normalized 0..1), + # or -1 if none. + def self.find_silence_at(regions, pos) + regions.each_with_index do |r, i| + return i if pos >= r[:start] && pos < r[:end] + end + -1 + end + + # Start position (normalized) of speaking portion N (0-based). + # Portion 0 starts at 0.0; portion N starts at silence[N-1].end. + def self.speaking_portion_start(regions, portion) + return 0.0 if portion <= 0 + if portion > regions.length + return regions.length > 0 ? regions.last[:end] : 0.0 + end + regions[portion - 1][:end] + end + + # Which speaking portion (0-based) the current position falls in. + # During silence, returns the portion that just ended. + def self.current_speaking_portion(regions, pos) + portion = 0 + regions.each_with_index do |r, i| + if pos >= r[:end] + portion = i + 1 + else + break + end + end + portion + end + + # Total number of speaking portions (always silence_count + 1). + def self.total_speaking_portions(regions) + regions.length + 1 + end + + # Seek target (seconds) for jumping to the start of a speaking portion. + # Lands `lead_frames` render frames into the padding zone (~33ms at 60fps). + def self.portion_seek_target(duration, regions, portion, + lead_frames: LEAD_FRAMES, fps: 60) + pos = speaking_portion_start(regions, portion) + target = pos * duration + (lead_frames.to_f / fps) + target = 0.0 if target < 0.0 + target = duration if target > duration + target + end + + # Is `pos` (normalized 0..1) inside the padding zone of the given + # speaking portion? Padding zone = [start, start + padding/duration]. + def self.in_padding_zone?(regions, pos, portion, duration, + padding: PADDING_SECONDS) + return false if duration <= 0.0 + pad_norm = padding / duration + start_pos = speaking_portion_start(regions, portion) + pos >= start_pos && pos < start_pos + pad_norm + end end end @@ -197,9 +388,10 @@ module StudyPlayer # UP / DOWN — seek -15s / +15s # J / L — seek -10% / +10% # 0 .. 9 — seek to N×10% of duration + # V / B — prev/next speaking portion # ESC — quit # - # Phase 4 adds: V/B (prev/next portion), SMART_PLAY hold override + # Phase 4 adds: SMART_PLAY hold override SEEK_SMALL = 5.0 # seconds: LEFT / RIGHT SEEK_LARGE = 15.0 # seconds: UP / DOWN @@ -210,6 +402,8 @@ module StudyPlayer pb = player_entity.get(playback_state) next unless pb + duration = runtime.audio.duration + # --- Play/pause toggle --- if Rl.key_pressed?(:space) if pb[:playing] @@ -223,7 +417,6 @@ module StudyPlayer end # --- Seek: small step --- - duration = runtime.audio.duration if Rl.key_pressed?(:right) target = StudyPlayer::Core.offset_to_seek(pb[:current_time], SEEK_SMALL, duration) pb[:seek_target] = target @@ -274,6 +467,33 @@ module StudyPlayer end end end + + # --- Portion navigation: V (prev) / B (next) --- + regions = runtime.silence_regions + if regions && regions.length >= 0 && duration > 0 + if Rl.key_pressed?(:v) + # Previous portion + pos_norm = Core.time_to_ratio(pb[:current_time], duration) + current = Core.current_speaking_portion(regions, pos_norm) + prev_portion = current - 1 + prev_portion = 0 if prev_portion < 0 + target = Core.portion_seek_target(duration, regions, prev_portion) + pb[:seek_target] = target + pb[:seek_pending] = true + player_entity.set(playback_state, pb) + elsif Rl.key_pressed?(:b) + # Next portion + pos_norm = Core.time_to_ratio(pb[:current_time], duration) + current = Core.current_speaking_portion(regions, pos_norm) + total = Core.total_speaking_portions(regions) + next_portion = current + 1 + next_portion = total - 1 if next_portion >= total + target = Core.portion_seek_target(duration, regions, next_portion) + pb[:seek_target] = target + pb[:seek_pending] = true + player_entity.set(playback_state, pb) + end + end end end end @@ -313,6 +533,39 @@ module StudyPlayer # Start playback runtime.audio.play + + # --- Phase 3: Silence detection via C scanner --- + # StudyAudio.scan_silence does the peak scan in C (avoids building + # a Ruby array of millions of float samples) and returns raw + # normalized silence regions as [[start, end], ...]. + # We then pad them with the pure-Ruby pad_silence_regions. + begin + raw_pairs = StudyAudio.scan_silence(path, + StudyPlayer::Core::SILENCE_THRESHOLD, + StudyPlayer::Core::SILENCE_MIN_DURATION) + if raw_pairs && raw_pairs.length > 0 + # Convert [[s,e],...] to [{start:, end:}, ...] + raw_regions = [] + raw_pairs.each do |pair| + raw_regions << { start: pair[0].to_f, end: pair[1].to_f } + end + regions = StudyPlayer::Core.pad_silence_regions( + raw_regions, duration) + runtime.silence_regions = regions + runtime.raw_silence_regions = raw_regions + runtime.analysis_done = true + else + runtime.silence_regions = [] + runtime.analysis_done = true + end + rescue => e + begin + Log.warn("Silence detection failed: #{e.message}") if defined?(Log) + rescue + end + runtime.silence_regions = [] + runtime.analysis_done = false + end else # Load failed — clear the file path af[:path] = "" @@ -415,11 +668,15 @@ module StudyPlayer # Runtime: owns non-serializable handles (Rl::Music) and the flecs world. class Runtime attr_accessor :world, :player_entity, :audio, :components + attr_accessor :silence_regions, :raw_silence_regions, :analysis_done def initialize(world:, player_entity:, audio:, components:) @world = world @player_entity = player_entity @audio = audio @components = components + @silence_regions = [] + @raw_silence_regions = [] + @analysis_done = false end end @@ -557,8 +814,20 @@ module StudyPlayer x: SCREEN_W / 2 - 60, y: 420, font_size: 18, color: playing ? Rl::Color.new(100, 200, 100, 255) : Rl::Color.new(200, 100, 100, 255)) + # --- Phase 3: Section counter (speaking portion N / total) --- + regions = runtime.silence_regions + if regions && regions.length >= 0 && duration > 0 + pos_norm = StudyPlayer::Core.time_to_ratio(pos, duration) + current_port = StudyPlayer::Core.current_speaking_portion(regions, pos_norm) + total_port = StudyPlayer::Core.total_speaking_portions(regions) + section_text = "#{current_port + 1}/#{total_port}" + Rl.draw_text(text: section_text, + x: SCREEN_W / 2 - 30, y: 450, font_size: 24, + color: Rl::Color.new(255, 200, 100, 255)) + end + # Controls help - Rl.draw_text(text: "SPACE: play/pause LEFT/RIGHT: -5s/+5s UP/DOWN: -15s/+15s J/L: -10%/+10% 0-9: n×10% ESC: quit", + Rl.draw_text(text: "SPACE: play/pause LEFT/RIGHT: -5s/+5s V/B: prev/next portion J/L: -10%/+10% 0-9: n×10% ESC: quit", x: 20, y: SCREEN_H - 60, font_size: 14, color: Rl::Color.new(120, 120, 140, 255)) diff --git a/game/study_player/verify_scan.rb b/game/study_player/verify_scan.rb new file mode 100644 index 0000000..b61b90a --- /dev/null +++ b/game/study_player/verify_scan.rb @@ -0,0 +1,29 @@ +# Quick verification of scan_silence C function with a small audio file. +# Self-contained for mruby (no require). + +path = ARGV[0] +unless path && File.exist?(path) + puts "Usage: game verify_scan.rb <audio_file>" + exit 1 +end + +Rl.set_trace_log_level(4) +Rl.init_window(100, 100, "vfy") +Rl.set_window_state(Rl::FLAG_WINDOW_HIDDEN) +Rl.init_audio_device + +begin + puts "File: #{path}" + raw = StudyAudio.scan_silence(path, 0.015, 0.75) + puts "Raw silence regions: #{raw.length}" + raw.first(5).each_with_index do |r, i| + puts " [#{i}] #{r[0].round(4)} – #{r[1].round(4)}" + end + puts " ..." if raw.length > 5 + puts "OK" +rescue => e + puts "FAIL: #{e.class}: #{e.message}" +ensure + Rl.close_audio_device + Rl.close_window +end diff --git a/game/study_player/verify_silence.rb b/game/study_player/verify_silence.rb new file mode 100644 index 0000000..abd8650 --- /dev/null +++ b/game/study_player/verify_silence.rb @@ -0,0 +1,140 @@ +# Verify silence detection on a real audio file (self-contained, headless). +# Usage: ./zig-out/bin/game game/study_player/verify_silence.rb <audio_file> + +# ── Pure Core module (inlined for mruby) ── + +module StudyPlayer + module Core + SILENCE_THRESHOLD = 0.015 + SILENCE_MIN_DURATION = 0.75 + PADDING_SECONDS = 0.25 + CHUNK_SECONDS = 0.01 + LEAD_FRAMES = 2 + + def self.detect_silence(samples, sample_rate, + threshold: SILENCE_THRESHOLD, + min_duration: SILENCE_MIN_DURATION) + total_frames = samples.length + return [] if total_frames == 0 + + chunk_size = (sample_rate * CHUNK_SECONDS).to_i + chunk_size = 1 if chunk_size < 1 + min_frames = min_duration * sample_rate + + regions = [] + in_silence = false + silence_start = 0 + + i = 0 + while i < total_frames + range_end = i + chunk_size + range_end = total_frames if range_end > total_frames + + peak = 0.0 + j = i + while j < range_end + v = samples[j] + v = -v if v < 0.0 + peak = v if v > peak + j += 1 + end + + if peak < threshold + unless in_silence + silence_start = i + in_silence = true + end + else + if in_silence + len = i - silence_start + if len >= min_frames + regions << { + start: silence_start.to_f / total_frames, + end: i.to_f / total_frames, + } + end + in_silence = false + end + end + + i = range_end + end + + if in_silence + len = total_frames - silence_start + if len >= min_frames + regions << { start: silence_start.to_f / total_frames, end: 1.0 } + end + end + + regions + end + + def self.pad_silence_regions(regions, duration, padding: PADDING_SECONDS) + return [] if duration <= 0.0 + return regions.dup if regions.empty? + pad_norm = padding / duration + result = [] + regions.each do |r| + s = r[:start] + pad_norm + e = r[:end] - pad_norm + result << { start: s, end: e } if s < e + end + result + end + + def self.total_speaking_portions(regions) + regions.length + 1 + end + end +end + +# ── Main ── + +path = ARGV[0] +unless path && File.exist?(path) + puts "Usage: game verify_silence.rb <audio_file>" + exit 1 +end + +Rl.set_trace_log_level(4) # suppress INFO spam +Rl.init_window(100, 100, "verify") +Rl.set_window_state(Rl::FLAG_WINDOW_HIDDEN) +Rl.init_audio_device + +begin + puts "Loading: #{path}" + result = StudyAudio.load_wave_samples(path) + + if result.nil? || result.length != 2 + puts "ERROR: load_wave_samples failed" + exit 1 + end + + sample_rate = result[0].to_i + samples = result[1] + duration = samples.length.to_f / sample_rate + + puts "Rate: #{sample_rate} Hz Frames: #{samples.length} Duration: #{duration.round(1)}s" + + raw = StudyPlayer::Core.detect_silence(samples, sample_rate) + padded = StudyPlayer::Core.pad_silence_regions(raw, duration) + portions = StudyPlayer::Core.total_speaking_portions(padded) + + puts "Raw silences: #{raw.length} Padded: #{padded.length} Portions: #{portions}" + + padded.first(5).each_with_index do |r, i| + ss = r[:start] * duration + es = r[:end] * duration + puts " [#{i}] #{ss.round(1)}s – #{es.round(1)}s (#{(es-ss).round(1)}s)" + end + puts " ..." if padded.length > 5 + + puts "OK: #{padded.length} silences, #{portions} portions" +rescue => e + puts "FAIL: #{e.class}: #{e.message}" + exit 1 +ensure + Rl.close_audio_device + Rl.close_window +end diff --git a/mrbgems/study_audio/mrbgem.rake b/mrbgems/study_audio/mrbgem.rake new file mode 100644 index 0000000..eb0a430 --- /dev/null +++ b/mrbgems/study_audio/mrbgem.rake @@ -0,0 +1,10 @@ +stack_root = ENV['JAMSTACK_ROOT'] || File.expand_path('../../..', __dir__) +raylib_inc = File.join(stack_root, 'vendor', 'raylib', 'src') + +MRuby::Gem::Specification.new('study_audio') do |spec| + spec.license = 'MIT' + spec.authors = 'raylib-jamstack' + spec.summary = 'Ruby (StudyAudio) native helper: loads raw float samples from audio files via raylib Wave API' + + spec.cc.include_paths << raylib_inc +end diff --git a/mrbgems/study_audio/src/study_audio.c b/mrbgems/study_audio/src/study_audio.c new file mode 100644 index 0000000..3969fbc --- /dev/null +++ b/mrbgems/study_audio/src/study_audio.c @@ -0,0 +1,155 @@ +/* study_audio.c — Native helpers for audio analysis via raylib's Wave API. + * + * Two functions: + * 1. load_wave_samples(path) → [sample_rate, FloatArray] (small files only) + * 2. scan_silence(path, threshold, min_duration) → [[start,end],...] (normalized) + * + * scan_silence is the "imperative shell" for the pure-Ruby Core — it runs the + * peak-scanning loop in C (avoids building a Ruby array of millions of floats) + * and returns only the silence region pairs. + */ + +#include <mruby.h> +#include <mruby/array.h> +#include <mruby/string.h> +#include "raylib.h" + +/* ------------------------------------------------------------------ */ +/* load_wave_samples — for testing / small files only */ +/* ------------------------------------------------------------------ */ + +static mrb_value +sa_load_wave_samples(mrb_state *mrb, mrb_value self) +{ + const char *path; + mrb_get_args(mrb, "z", &path); + + Wave wave = LoadWave(path); + if (wave.data == NULL || wave.frameCount == 0) { + return mrb_nil_value(); + } + + WaveFormat(&wave, wave.sampleRate, 32, 1); + + float *samples = (float *)wave.data; + unsigned int frameCount = wave.frameCount; + int sampleRate = (int)wave.sampleRate; + + mrb_value ary = mrb_ary_new_capa(mrb, frameCount); + for (unsigned int i = 0; i < frameCount; i++) { + mrb_ary_push(mrb, ary, mrb_float_value(mrb, (double)samples[i])); + } + + UnloadWave(wave); + + mrb_value result = mrb_ary_new_capa(mrb, 2); + mrb_ary_push(mrb, result, mrb_fixnum_value(sampleRate)); + mrb_ary_push(mrb, result, ary); + return result; +} + +/* ------------------------------------------------------------------ */ +/* scan_silence — C-level peak scan, returns normalized region pairs */ +/* ------------------------------------------------------------------ */ + +#define MAX_REGIONS 4096 + +static mrb_value +sa_scan_silence(mrb_state *mrb, mrb_value self) +{ + const char *path; + mrb_float threshold, min_duration; + mrb_get_args(mrb, "zff", &path, &threshold, &min_duration); + + Wave wave = LoadWave(path); + if (wave.data == NULL || wave.frameCount == 0) { + return mrb_ary_new(mrb); + } + + WaveFormat(&wave, wave.sampleRate, 32, 1); + + float *samples = (float *)wave.data; + unsigned int totalFrames = wave.frameCount; + float sampleRate = (float)wave.sampleRate; + + /* Chunk size: ~10ms */ + int chunkSize = (int)(sampleRate * 0.01f); + if (chunkSize < 1) chunkSize = 1; + float minFrames = min_duration * sampleRate; + + mrb_value regions = mrb_ary_new_capa(mrb, 64); + + int inSilence = 0; + unsigned int silenceStart = 0; + int regionCount = 0; + + unsigned int i; + for (i = 0; i < totalFrames; i += chunkSize) + { + unsigned int end = i + chunkSize; + if (end > totalFrames) end = totalFrames; + + /* Find peak amplitude in this chunk */ + float peak = 0.0f; + unsigned int j; + for (j = i; j < end; j++) { + float v = samples[j]; + if (v < 0.0f) v = -v; + if (v > peak) peak = v; + } + + if (peak < threshold) { + if (!inSilence) { silenceStart = i; inSilence = 1; } + } else { + if (inSilence) { + unsigned int len = i - silenceStart; + if ((float)len >= minFrames && regionCount < MAX_REGIONS) { + mrb_value pair = mrb_ary_new_capa(mrb, 2); + mrb_ary_push(mrb, pair, + mrb_float_value(mrb, (double)silenceStart / (double)totalFrames)); + mrb_ary_push(mrb, pair, + mrb_float_value(mrb, (double)i / (double)totalFrames)); + mrb_ary_push(mrb, regions, pair); + regionCount++; + } + inSilence = 0; + } + } + } + + /* Close trailing silence */ + if (inSilence) { + unsigned int len = totalFrames - silenceStart; + if ((float)len >= minFrames && regionCount < MAX_REGIONS) { + mrb_value pair = mrb_ary_new_capa(mrb, 2); + mrb_ary_push(mrb, pair, + mrb_float_value(mrb, (double)silenceStart / (double)totalFrames)); + mrb_ary_push(mrb, pair, mrb_float_value(mrb, 1.0)); + mrb_ary_push(mrb, regions, pair); + } + } + + UnloadWave(wave); + return regions; +} + +/* ------------------------------------------------------------------ */ +/* gem init */ +/* ------------------------------------------------------------------ */ + +void +mrb_study_audio_gem_init(mrb_state *mrb) +{ + struct RClass *mod = mrb_define_module(mrb, "StudyAudio"); + mrb_define_module_function(mrb, mod, "load_wave_samples", + sa_load_wave_samples, MRB_ARGS_REQ(1)); + mrb_define_module_function(mrb, mod, "scan_silence", + sa_scan_silence, MRB_ARGS_REQ(3)); +} + +void +mrb_study_audio_gem_final(mrb_state *mrb) +{ + /* nothing */ +} + |
