summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authorAdam Malczewski <[email protected]>2026-04-28 14:29:29 +0900
committerAdam Malczewski <[email protected]>2026-04-28 14:29:29 +0900
commit3f9836fda60f26d856e3621a36ce1f4555c69f4c (patch)
tree0a16399e689f5b4687d74169ab817d380f69bc88
downloaddispatch-adapter-tester-3f9836fda60f26d856e3621a36ce1f4555c69f4c.tar.gz
dispatch-adapter-tester-3f9836fda60f26d856e3621a36ce1f4555c69f4c.zip
changes
-rw-r--r--.gitignore17
-rw-r--r--.rspec3
-rw-r--r--.rubocop.yml44
-rw-r--r--APP_INTEGRATION.md66
-rw-r--r--Gemfile14
-rw-r--r--Gemfile.lock133
-rw-r--r--LICENSE.txt21
-rw-r--r--README.md124
-rw-r--r--Rakefile12
-rwxr-xr-xbin/check32
-rwxr-xr-xbin/console11
-rwxr-xr-xbin/install23
-rwxr-xr-xbin/setup8
-rw-r--r--dispatch-adapter-tester.gemspec33
-rw-r--r--lib/dispatch/adapter/tester.rb15
-rw-r--r--lib/dispatch/adapter/tester/errors.rb57
-rw-r--r--lib/dispatch/adapter/tester/playbook.rb196
-rw-r--r--lib/dispatch/adapter/tester/step.rb125
-rw-r--r--lib/dispatch/adapter/tester/version.rb9
-rw-r--r--sig/dispatch/adapter/tester.rbs8
-rw-r--r--spec/dispatch/adapter/tester/errors_spec.rb54
-rw-r--r--spec/dispatch/adapter/tester/playbook_spec.rb360
-rw-r--r--spec/dispatch/adapter/tester/step_spec.rb185
-rw-r--r--spec/dispatch/adapter/tester_spec.rb22
-rw-r--r--spec/spec_helper.rb13
25 files changed, 1585 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..c411e15
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,17 @@
+/.bundle/
+/.yardoc
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+# rspec failure tracking
+.rspec_status
+
+# Test results
+test_results.txt
+
+# Built gems
+*.gem
diff --git a/.rspec b/.rspec
new file mode 100644
index 0000000..34c5164
--- /dev/null
+++ b/.rspec
@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper
diff --git a/.rubocop.yml b/.rubocop.yml
new file mode 100644
index 0000000..f520914
--- /dev/null
+++ b/.rubocop.yml
@@ -0,0 +1,44 @@
+AllCops:
+ TargetRubyVersion: 3.2
+ NewCops: enable
+
+Style/StringLiterals:
+ EnforcedStyle: double_quotes
+
+Style/StringLiteralsInInterpolation:
+ EnforcedStyle: double_quotes
+
+Style/FrozenStringLiteralComment:
+ Enabled: true
+ EnforcedStyle: always
+
+Metrics/BlockLength:
+ Exclude:
+ - "spec/**/*"
+
+Metrics/MethodLength:
+ Max: 40
+
+Metrics/AbcSize:
+ Enabled: false
+
+Metrics/CyclomaticComplexity:
+ Enabled: false
+
+Metrics/PerceivedComplexity:
+ Enabled: false
+
+Style/Documentation:
+ Enabled: false
+
+Naming/MethodParameterName:
+ Enabled: false
+
+Metrics/ClassLength:
+ Enabled: false
+
+Metrics/ParameterLists:
+ Enabled: false
+
+Style/MultilineBlockChain:
+ Enabled: false
diff --git a/APP_INTEGRATION.md b/APP_INTEGRATION.md
new file mode 100644
index 0000000..041f804
--- /dev/null
+++ b/APP_INTEGRATION.md
@@ -0,0 +1,66 @@
+# App Integration Requirements
+
+This document describes the changes required in the main `dispatch-api` application
+to support `dispatch-adapter-tester` as a drop-in replacement for
+`dispatch-adapter-copilot`.
+
+---
+
+## 1. Make the Adapter Class Configurable
+
+`AgentJob#build_adapter` currently hardcodes `Dispatch::Adapter::Copilot`. This needs
+to be changed so the adapter class is resolved dynamically — either from a
+configuration setting, an environment variable, or a method that can be overridden
+in the test environment.
+
+The tester adapter's class is `Dispatch::Adapter::Tester::Playbook`. It accepts the
+same constructor keyword arguments as `Copilot` (model, max_tokens,
+min_request_interval, rate_limit) and silently absorbs any it does not use.
+
+---
+
+## 2. Add the Gem to the Gemfile
+
+The gem needs to be added to the application's `Gemfile`, scoped to the test group,
+with a local path reference.
+
+---
+
+## 3. Provide a Way to Pass the Playbook Script (Per-Agent)
+
+The `Playbook` adapter requires a `steps_json:` keyword argument containing the JSON
+script. Each agent instance needs its **own** playbook — multiple agents running
+concurrently will each have independent `Playbook` instances with separate step
+lists, indices, and call logs.
+
+The app's adapter construction path needs a per-agent mechanism to supply the
+playbook JSON. Some approaches:
+
+- An attribute on the `Agent` model (e.g. `playbook_json`) that is only populated
+ in the test environment.
+- A test helper registry keyed by agent ID that maps each agent to its playbook.
+- A factory/fixture that associates a playbook with each agent before the test runs.
+
+The chosen mechanism must ensure that when `build_adapter` constructs a `Playbook`
+for agent A, it receives agent A's script — not agent B's.
+
+---
+
+## 4. No Changes to the Agent Loop
+
+The agent loop (`AgentJob#run_loop`) does **not** need any changes. The `Playbook`
+adapter returns the same `Dispatch::Adapter::Response` struct with the same fields
+(`content`, `tool_calls`, `model`, `stop_reason`, `usage`), so the existing loop
+logic will work unmodified.
+
+---
+
+## 5. Test Verification Helpers
+
+After a test run, call `adapter.verify_all_consumed!` to assert that the full
+playbook script was exercised. This raises
+`Dispatch::Adapter::Tester::UnconsumedStepsError` if any steps were not consumed,
+which helps catch cases where the agent loop exited early unexpectedly.
+
+The `adapter.call_log` array can also be inspected to verify what messages, system
+prompts, and tools were passed to each `chat` call.
diff --git a/Gemfile b/Gemfile
new file mode 100644
index 0000000..8e6f2b1
--- /dev/null
+++ b/Gemfile
@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gemspec
+
+gem "dispatch-adapter-interface", path: "../dispatch-adapter-interface"
+
+gem "irb"
+gem "rake", "~> 13.0"
+
+gem "rspec", "~> 3.0"
+
+gem "rubocop", "~> 1.21"
diff --git a/Gemfile.lock b/Gemfile.lock
new file mode 100644
index 0000000..842e0e8
--- /dev/null
+++ b/Gemfile.lock
@@ -0,0 +1,133 @@
+PATH
+ remote: ../dispatch-adapter-interface
+ specs:
+ dispatch-adapter-interface (0.1.0)
+
+PATH
+ remote: .
+ specs:
+ dispatch-adapter-tester (0.1.0)
+ dispatch-adapter-interface (~> 0.1)
+
+GEM
+ remote: https://rubygems.org/
+ specs:
+ ast (2.4.3)
+ date (3.5.1)
+ diff-lcs (1.6.2)
+ erb (6.0.2)
+ io-console (0.8.2)
+ irb (1.17.0)
+ pp (>= 0.6.0)
+ prism (>= 1.3.0)
+ rdoc (>= 4.0.0)
+ reline (>= 0.4.2)
+ json (2.19.3)
+ language_server-protocol (3.17.0.5)
+ lint_roller (1.1.0)
+ parallel (2.0.1)
+ parser (3.3.11.1)
+ ast (~> 2.4.1)
+ racc
+ pp (0.6.3)
+ prettyprint
+ prettyprint (0.2.0)
+ prism (1.9.0)
+ psych (5.3.1)
+ date
+ stringio
+ racc (1.8.1)
+ rainbow (3.1.1)
+ rake (13.3.1)
+ rdoc (7.2.0)
+ erb
+ psych (>= 4.0.0)
+ tsort
+ regexp_parser (2.12.0)
+ reline (0.6.3)
+ io-console (~> 0.5)
+ rspec (3.13.2)
+ rspec-core (~> 3.13.0)
+ rspec-expectations (~> 3.13.0)
+ rspec-mocks (~> 3.13.0)
+ rspec-core (3.13.6)
+ rspec-support (~> 3.13.0)
+ rspec-expectations (3.13.5)
+ diff-lcs (>= 1.2.0, < 2.0)
+ rspec-support (~> 3.13.0)
+ rspec-mocks (3.13.8)
+ diff-lcs (>= 1.2.0, < 2.0)
+ rspec-support (~> 3.13.0)
+ rspec-support (3.13.7)
+ rubocop (1.86.1)
+ json (~> 2.3)
+ language_server-protocol (~> 3.17.0.2)
+ lint_roller (~> 1.1.0)
+ parallel (>= 1.10)
+ parser (>= 3.3.0.2)
+ rainbow (>= 2.2.2, < 4.0)
+ regexp_parser (>= 2.9.3, < 3.0)
+ rubocop-ast (>= 1.49.0, < 2.0)
+ ruby-progressbar (~> 1.7)
+ unicode-display_width (>= 2.4.0, < 4.0)
+ rubocop-ast (1.49.1)
+ parser (>= 3.3.7.2)
+ prism (~> 1.7)
+ ruby-progressbar (1.13.0)
+ stringio (3.2.0)
+ tsort (0.2.0)
+ unicode-display_width (3.2.0)
+ unicode-emoji (~> 4.1)
+ unicode-emoji (4.2.0)
+
+PLATFORMS
+ ruby
+ x86_64-linux
+
+DEPENDENCIES
+ dispatch-adapter-interface!
+ dispatch-adapter-tester!
+ irb
+ rake (~> 13.0)
+ rspec (~> 3.0)
+ rubocop (~> 1.21)
+
+CHECKSUMS
+ ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383
+ date (3.5.1) sha256=750d06384d7b9c15d562c76291407d89e368dda4d4fff957eb94962d325a0dc0
+ diff-lcs (1.6.2) sha256=9ae0d2cba7d4df3075fe8cd8602a8604993efc0dfa934cff568969efb1909962
+ dispatch-adapter-interface (0.1.0)
+ dispatch-adapter-tester (0.1.0)
+ erb (6.0.2) sha256=9fe6264d44f79422c87490a1558479bd0e7dad4dd0e317656e67ea3077b5242b
+ io-console (0.8.2) sha256=d6e3ae7a7cc7574f4b8893b4fca2162e57a825b223a177b7afa236c5ef9814cc
+ irb (1.17.0) sha256=168c4ddb93d8a361a045c41d92b2952c7a118fa73f23fe14e55609eb7a863aae
+ json (2.19.3) sha256=289b0bb53052a1fa8c34ab33cc750b659ba14a5c45f3fcf4b18762dc67c78646
+ language_server-protocol (3.17.0.5) sha256=fd1e39a51a28bf3eec959379985a72e296e9f9acfce46f6a79d31ca8760803cc
+ lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87
+ parallel (2.0.1) sha256=337782d3e39f4121e67563bf91dd8ece67f48923d90698614773a0ec9a5b2c7d
+ parser (3.3.11.1) sha256=d17ace7aabe3e72c3cc94043714be27cc6f852f104d81aa284c2281aecc65d54
+ pp (0.6.3) sha256=2951d514450b93ccfeb1df7d021cae0da16e0a7f95ee1e2273719669d0ab9df6
+ prettyprint (0.2.0) sha256=2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193
+ prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85
+ psych (5.3.1) sha256=eb7a57cef10c9d70173ff74e739d843ac3b2c019a003de48447b2963d81b1974
+ racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f
+ rainbow (3.1.1) sha256=039491aa3a89f42efa1d6dec2fc4e62ede96eb6acd95e52f1ad581182b79bc6a
+ rake (13.3.1) sha256=8c9e89d09f66a26a01264e7e3480ec0607f0c497a861ef16063604b1b08eb19c
+ rdoc (7.2.0) sha256=8650f76cd4009c3b54955eb5d7e3a075c60a57276766ebf36f9085e8c9f23192
+ regexp_parser (2.12.0) sha256=35a916a1d63190ab5c9009457136ae5f3c0c7512d60291d0d1378ba18ce08ebb
+ reline (0.6.3) sha256=1198b04973565b36ec0f11542ab3f5cfeeec34823f4e54cebde90968092b1835
+ rspec (3.13.2) sha256=206284a08ad798e61f86d7ca3e376718d52c0bc944626b2349266f239f820587
+ rspec-core (3.13.6) sha256=a8823c6411667b60a8bca135364351dda34cd55e44ff94c4be4633b37d828b2d
+ rspec-expectations (3.13.5) sha256=33a4d3a1d95060aea4c94e9f237030a8f9eae5615e9bd85718fe3a09e4b58836
+ rspec-mocks (3.13.8) sha256=086ad3d3d17533f4237643de0b5c42f04b66348c28bf6b9c2d3f4a3b01af1d47
+ rspec-support (3.13.7) sha256=0640e5570872aafefd79867901deeeeb40b0c9875a36b983d85f54fb7381c47c
+ rubocop (1.86.1) sha256=44415f3f01d01a21e01132248d2fd0867572475b566ca188a0a42133a08d4531
+ rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035
+ ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
+ stringio (3.2.0) sha256=c37cb2e58b4ffbd33fe5cd948c05934af997b36e0b6ca6fdf43afa234cf222e1
+ tsort (0.2.0) sha256=9650a793f6859a43b6641671278f79cfead60ac714148aabe4e3f0060480089f
+ unicode-display_width (3.2.0) sha256=0cdd96b5681a5949cdbc2c55e7b420facae74c4aaf9a9815eee1087cb1853c42
+ unicode-emoji (4.2.0) sha256=519e69150f75652e40bf736106cfbc8f0f73aa3fb6a65afe62fefa7f80b0f80f
+
+BUNDLED WITH
+ 4.0.9
diff --git a/LICENSE.txt b/LICENSE.txt
new file mode 100644
index 0000000..974325c
--- /dev/null
+++ b/LICENSE.txt
@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Adam Malczewski
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..47f228b
--- /dev/null
+++ b/README.md
@@ -0,0 +1,124 @@
+# dispatch-adapter-tester
+
+A deterministic playbook adapter for integration testing Dispatch agent flows.
+
+`dispatch-adapter-tester` is a drop-in replacement for `dispatch-adapter-copilot`
+that replays a scripted JSON sequence of AI responses, enabling deterministic
+end-to-end testing of the full agent loop without calling any real LLM API.
+
+## Installation
+
+Add to your Gemfile (test group):
+
+```ruby
+group :test do
+ gem "dispatch-adapter-tester", path: "reference/dispatch-adapter-tester"
+end
+```
+
+## Usage
+
+### Define a Playbook
+
+A playbook is a JSON array of steps. Each step represents one response from the
+fake AI:
+
+```json
+[
+ {
+ "step": 1,
+ "type": "message",
+ "content": "I will read the file now."
+ },
+ {
+ "step": 2,
+ "type": "tool_calls",
+ "content": null,
+ "tool_calls": [
+ {
+ "id": "tc_read_001",
+ "name": "read_file",
+ "arguments": { "path": "/some/file.rb" }
+ }
+ ]
+ },
+ {
+ "step": 3,
+ "type": "message",
+ "content": "The file contains a Ruby class."
+ }
+]
+```
+
+### Step Schema
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `step` | integer | yes | Unique step identifier for debug output |
+| `type` | string | yes | `"message"` or `"tool_calls"` |
+| `content` | string/null | yes for message, optional for tool_calls | Text content of the response |
+| `tool_calls` | array | yes for tool_calls | Array of tool call objects |
+
+Each tool call object:
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `id` | string | yes | Tool call ID (for tracing/debugging) |
+| `name` | string | yes | Name of the tool to invoke |
+| `arguments` | object | yes | Arguments to pass to the tool |
+
+### Create the Adapter
+
+```ruby
+adapter = Dispatch::Adapter::Tester::Playbook.new(
+ steps_json: steps_json, # JSON string or Ruby array
+ model: "gpt-4", # optional, for interface compat
+ max_tokens: 200_000, # optional, absorbed but unused
+ min_request_interval: 0, # optional, absorbed but unused
+ rate_limit: nil # optional, absorbed but unused
+)
+```
+
+### Use in Tests
+
+```ruby
+# Each call to chat consumes the next step
+response = adapter.chat(messages, system: system_prompt, tools: registry.to_a)
+
+# After the test, verify all steps were consumed
+adapter.verify_all_consumed!
+
+# Inspect what was passed to each chat call
+adapter.call_log.each do |entry|
+ puts entry[:step] # the Step object
+ puts entry[:system] # system prompt passed
+ puts entry[:messages] # messages array passed
+ puts entry[:tools] # tools array passed
+end
+```
+
+### Test Helpers
+
+| Method | Description |
+|--------|-------------|
+| `finished?` | Returns true when all steps have been consumed |
+| `remaining_steps` | Number of unconsumed steps |
+| `verify_all_consumed!` | Raises `UnconsumedStepsError` if steps remain |
+| `reset!` | Resets to step 0 and clears the call log |
+| `call_log` | Array of recorded chat call details |
+| `current_index` | Current position in the playbook |
+
+### Error Types
+
+| Error | When |
+|-------|------|
+| `InvalidPlaybookError` | Malformed JSON, missing fields, invalid types |
+| `PlaybookExhaustedError` | `chat` called after all steps consumed |
+| `UnconsumedStepsError` | `verify_all_consumed!` called with steps remaining |
+
+All errors include step IDs for debugging.
+
+## See Also
+
+- `APP_INTEGRATION.md` — Required changes in the host application
+- `dispatch-adapter-copilot` — The real adapter this replaces
diff --git a/Rakefile b/Rakefile
new file mode 100644
index 0000000..cca7175
--- /dev/null
+++ b/Rakefile
@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]
diff --git a/bin/check b/bin/check
new file mode 100755
index 0000000..c4385f7
--- /dev/null
+++ b/bin/check
@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+RESULTS_FILE="test_results.txt"
+
+{
+ echo "============================================"
+ echo "dispatch-adapter-tester — $(date)"
+ echo "============================================"
+ echo ""
+
+ echo "--- bundle install ---"
+ bundle install 2>&1
+ echo ""
+
+ echo "--- rubocop --autocorrect-all ---"
+ bundle exec rubocop --autocorrect-all 2>&1 || true
+ echo ""
+
+ echo "--- rspec ---"
+ bundle exec rspec 2>&1 || true
+ echo ""
+
+ echo "============================================"
+ echo "Done."
+ echo "============================================"
+} | tee "$RESULTS_FILE"
+
+echo ""
+echo "Results written to $RESULTS_FILE"
diff --git a/bin/console b/bin/console
new file mode 100755
index 0000000..3c2bbcb
--- /dev/null
+++ b/bin/console
@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "dispatch/adapter/tester"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+require "irb"
+IRB.start(__FILE__)
diff --git a/bin/install b/bin/install
new file mode 100755
index 0000000..ac91156
--- /dev/null
+++ b/bin/install
@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+GEM_NAME="dispatch-adapter-tester"
+
+echo "--- Building $GEM_NAME ---"
+gem build "$GEM_NAME.gemspec"
+
+GEM_FILE=$(ls -t "$GEM_NAME"-*.gem 2>/dev/null | head -1)
+
+if [ -z "$GEM_FILE" ]; then
+ echo "ERROR: No .gem file found after build."
+ exit 1
+fi
+
+echo ""
+echo "--- Installing $GEM_FILE ---"
+gem install "$GEM_FILE" --local
+
+echo ""
+echo "Done. $GEM_NAME installed."
diff --git a/bin/setup b/bin/setup
new file mode 100755
index 0000000..dce67d8
--- /dev/null
+++ b/bin/setup
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here
diff --git a/dispatch-adapter-tester.gemspec b/dispatch-adapter-tester.gemspec
new file mode 100644
index 0000000..9b6017d
--- /dev/null
+++ b/dispatch-adapter-tester.gemspec
@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require_relative "lib/dispatch/adapter/tester/version"
+
+Gem::Specification.new do |spec|
+ spec.name = "dispatch-adapter-tester"
+ spec.version = Dispatch::Adapter::Tester::VERSION
+ spec.authors = ["Adam Malczewski"]
+ spec.email = ["[email protected]"]
+
+ spec.summary = "Deterministic playbook adapter for testing Dispatch agent flows"
+ spec.description = "A drop-in replacement for any Dispatch adapter that replays a scripted JSON " \
+ "sequence of AI responses, enabling deterministic end-to-end testing of the full agent loop."
+ spec.homepage = "https://github.com/realtradam/dispatch-adapter-tester"
+ spec.license = "MIT"
+ spec.required_ruby_version = ">= 3.2.0"
+ spec.metadata["homepage_uri"] = spec.homepage
+ spec.metadata["source_code_uri"] = spec.homepage
+ spec.metadata["rubygems_mfa_required"] = "true"
+
+ gemspec = File.basename(__FILE__)
+ spec.files = IO.popen(%w[git ls-files -z], chdir: __dir__, err: IO::NULL) do |ls|
+ ls.readlines("\x0", chomp: true).reject do |f|
+ (f == gemspec) ||
+ f.start_with?(*%w[bin/ Gemfile .gitignore .rspec spec/ .rubocop.yml])
+ end
+ end
+ spec.bindir = "exe"
+ spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+ spec.require_paths = ["lib"]
+
+ spec.add_dependency "dispatch-adapter-interface", "~> 0.1"
+end
diff --git a/lib/dispatch/adapter/tester.rb b/lib/dispatch/adapter/tester.rb
new file mode 100644
index 0000000..966ef43
--- /dev/null
+++ b/lib/dispatch/adapter/tester.rb
@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require "dispatch/adapter/interface"
+
+require_relative "tester/version"
+require_relative "tester/errors"
+require_relative "tester/step"
+require_relative "tester/playbook"
+
+module Dispatch
+ module Adapter
+ module Tester
+ end
+ end
+end
diff --git a/lib/dispatch/adapter/tester/errors.rb b/lib/dispatch/adapter/tester/errors.rb
new file mode 100644
index 0000000..e3a9a6b
--- /dev/null
+++ b/lib/dispatch/adapter/tester/errors.rb
@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+module Dispatch
+ module Adapter
+ module Tester
+ class Error < StandardError; end
+
+ # Raised when the playbook JSON schema is invalid
+ class InvalidPlaybookError < Error; end
+
+ # Raised when chat is called but no more steps remain
+ class PlaybookExhaustedError < Error
+ attr_reader :total_steps, :calls_made
+
+ def initialize(total_steps:, calls_made:)
+ @total_steps = total_steps
+ @calls_made = calls_made
+ super(
+ "Playbook exhausted: all #{total_steps} steps have been consumed, " \
+ "but chat was called a #{ordinalize(calls_made)} time"
+ )
+ end
+
+ private
+
+ def ordinalize(n)
+ suffix = case n % 100
+ when 11, 12, 13 then "th"
+ else
+ case n % 10
+ when 1 then "st"
+ when 2 then "nd"
+ when 3 then "rd"
+ else "th"
+ end
+ end
+ "#{n}#{suffix}"
+ end
+ end
+
+ # Raised when not all steps were consumed after a test run
+ class UnconsumedStepsError < Error
+ attr_reader :total_steps, :consumed, :remaining_step_ids
+
+ def initialize(total_steps:, consumed:, remaining_step_ids:)
+ @total_steps = total_steps
+ @consumed = consumed
+ @remaining_step_ids = remaining_step_ids
+ super(
+ "Playbook has unconsumed steps: #{consumed}/#{total_steps} steps consumed. " \
+ "Remaining steps: #{remaining_step_ids.inspect}"
+ )
+ end
+ end
+ end
+ end
+end
diff --git a/lib/dispatch/adapter/tester/playbook.rb b/lib/dispatch/adapter/tester/playbook.rb
new file mode 100644
index 0000000..bf18bf0
--- /dev/null
+++ b/lib/dispatch/adapter/tester/playbook.rb
@@ -0,0 +1,196 @@
+# frozen_string_literal: true
+
+require "json"
+
+module Dispatch
+ module Adapter
+ module Tester
+ # A deterministic, scriptable adapter for integration testing.
+ #
+ # Playbook is a drop-in replacement for Dispatch::Adapter::Copilot.
+ # Instead of calling a real LLM API, it replays a pre-defined sequence
+ # of steps from a JSON script. Each call to #chat consumes the next step
+ # and returns a Dispatch::Adapter::Response built from it.
+ #
+ # Usage:
+ # steps_json = '[{"step":1,"type":"message","content":"Hello"}]'
+ # adapter = Dispatch::Adapter::Tester::Playbook.new(steps_json: steps_json)
+ # response = adapter.chat(messages, system: "...", tools: [...])
+ # response.content # => "Hello"
+ #
+ class Playbook < Dispatch::Adapter::Base
+ MODEL_NAME = "tester-playbook"
+ PROVIDER_NAME = "Tester Playbook"
+ MAX_CONTEXT_TOKENS = 1_000_000
+
+ attr_reader :steps, :current_index, :call_log
+
+ # @param steps_json [String] JSON string containing an array of step objects
+ # @param model [String] model name to report (default: "tester-playbook")
+ # @param max_tokens [Integer] reported max tokens (unused, for interface compat)
+ # @param kwargs [Hash] absorbs any extra keyword arguments for drop-in compat
+ # (e.g. min_request_interval, rate_limit, etc.)
+ def initialize(steps_json: "[]", model: MODEL_NAME, max_tokens: 200_000, **_kwargs)
+ super()
+ @model = model
+ @max_tokens = max_tokens
+ @steps = parse_steps(steps_json)
+ @current_index = 0
+ @call_log = []
+ @mutex = Mutex.new
+
+ validate_step_ids_unique!
+ end
+
+ # Consume the next step and return a Response.
+ #
+ # Accepts the same signature as Dispatch::Adapter::Base#chat.
+ # The messages, system, tools, stream, max_tokens, and thinking
+ # parameters are recorded in the call_log for assertion purposes
+ # but do not affect the response (which is entirely driven by the script).
+ #
+ # @return [Dispatch::Adapter::Response]
+ def chat(messages, system: nil, tools: [], stream: false, max_tokens: nil, thinking: nil, &_block)
+ @mutex.synchronize do
+ if @current_index >= @steps.length
+ raise PlaybookExhaustedError.new(
+ total_steps: @steps.length,
+ calls_made: @current_index + 1
+ )
+ end
+
+ step = @steps[@current_index]
+ @current_index += 1
+
+ @call_log << {
+ step: step,
+ messages: messages,
+ system: system,
+ tools: tools,
+ stream: stream,
+ max_tokens: max_tokens,
+ thinking: thinking
+ }
+
+ build_response_from_step(step)
+ end
+ end
+
+ def model_name
+ @model
+ end
+
+ def provider_name
+ PROVIDER_NAME
+ end
+
+ def max_context_tokens
+ MAX_CONTEXT_TOKENS
+ end
+
+ def list_models
+ [
+ Dispatch::Adapter::ModelInfo.new(
+ id: MODEL_NAME,
+ name: "Tester Playbook",
+ max_context_tokens: MAX_CONTEXT_TOKENS,
+ supports_vision: false,
+ supports_tool_use: true,
+ supports_streaming: false
+ )
+ ]
+ end
+
+ # --- Test helper methods ---
+
+ # Check if all steps have been consumed.
+ # @return [Boolean]
+ def finished?
+ @current_index >= @steps.length
+ end
+
+ # Returns the number of remaining unconsumed steps.
+ # @return [Integer]
+ def remaining_steps
+ @steps.length - @current_index
+ end
+
+ # Raises UnconsumedStepsError if there are steps left.
+ # Call this at the end of a test to ensure the full script was exercised.
+ def verify_all_consumed!
+ return if finished?
+
+ remaining_ids = @steps[@current_index..].map(&:step_id)
+ raise UnconsumedStepsError.new(
+ total_steps: @steps.length,
+ consumed: @current_index,
+ remaining_step_ids: remaining_ids
+ )
+ end
+
+ # Resets the playbook to the beginning.
+ # Useful if you need to replay the same script.
+ def reset!
+ @mutex.synchronize do
+ @current_index = 0
+ @call_log.clear
+ end
+ end
+
+ private
+
+ def parse_steps(steps_json)
+ raw = if steps_json.is_a?(String)
+ parsed = JSON.parse(steps_json)
+ unless parsed.is_a?(Array)
+ raise InvalidPlaybookError, "Playbook JSON must be an array, got #{parsed.class}"
+ end
+
+ parsed
+ elsif steps_json.is_a?(Array)
+ steps_json
+ else
+ raise InvalidPlaybookError,
+ "steps_json must be a JSON string or Array, got #{steps_json.class}"
+ end
+
+ raw.map { |data| Step.new(data) }
+ rescue JSON::ParserError => e
+ raise InvalidPlaybookError, "Failed to parse playbook JSON: #{e.message}"
+ end
+
+ def validate_step_ids_unique!
+ ids = @steps.map(&:step_id)
+ duplicates = ids.group_by(&:itself).select { |_, v| v.size > 1 }.keys
+ return if duplicates.empty?
+
+ raise InvalidPlaybookError,
+ "Duplicate step IDs found: #{duplicates.inspect}. Each step must have a unique 'step' value."
+ end
+
+ def build_response_from_step(step)
+ tool_calls = step.tool_calls.map do |tc|
+ Dispatch::Adapter::ToolUseBlock.new(
+ id: tc["id"],
+ name: tc["name"],
+ arguments: tc["arguments"]
+ )
+ end
+
+ stop_reason = tool_calls.any? ? :tool_use : :end_turn
+
+ Dispatch::Adapter::Response.new(
+ content: step.content,
+ tool_calls: tool_calls,
+ model: @model,
+ stop_reason: stop_reason,
+ usage: Dispatch::Adapter::Usage.new(
+ input_tokens: 0,
+ output_tokens: 0
+ )
+ )
+ end
+ end
+ end
+ end
+end
diff --git a/lib/dispatch/adapter/tester/step.rb b/lib/dispatch/adapter/tester/step.rb
new file mode 100644
index 0000000..bfd5fce
--- /dev/null
+++ b/lib/dispatch/adapter/tester/step.rb
@@ -0,0 +1,125 @@
+# frozen_string_literal: true
+
+module Dispatch
+ module Adapter
+ module Tester
+ # Immutable value object representing a single step in a playbook.
+ # Validated at parse time so errors surface early.
+ class Step
+ VALID_TYPES = %w[message tool_calls].freeze
+
+ attr_reader :step_id, :type, :content, :tool_calls
+
+ def initialize(data)
+ validate_and_assign!(data)
+ freeze
+ end
+
+ def message?
+ @type == "message"
+ end
+
+ def tool_calls?
+ @type == "tool_calls"
+ end
+
+ def to_s
+ "Step ##{@step_id} (#{@type})"
+ end
+
+ private
+
+ def validate_and_assign!(data)
+ validate_hash!(data)
+ @step_id = extract_step_id!(data)
+ @type = extract_type!(data)
+ @content = data["content"]
+
+ if tool_calls?
+ @tool_calls = extract_tool_calls!(data)
+ else
+ @tool_calls = []
+ validate_message_has_content!
+ end
+ end
+
+ def validate_hash!(data)
+ return if data.is_a?(Hash)
+
+ raise InvalidPlaybookError, "Each step must be a JSON object, got #{data.class}"
+ end
+
+ def extract_step_id!(data)
+ step_id = data["step"]
+ raise InvalidPlaybookError, "Step is missing required field 'step'" if step_id.nil?
+ unless step_id.is_a?(Integer)
+ raise InvalidPlaybookError, "Step 'step' field must be an integer, got #{step_id.inspect}"
+ end
+
+ step_id
+ end
+
+ def extract_type!(data)
+ type = data["type"]
+ raise InvalidPlaybookError, "Step ##{@step_id} is missing required field 'type'" if type.nil?
+ unless VALID_TYPES.include?(type)
+ raise InvalidPlaybookError,
+ "Step ##{@step_id} has invalid type #{type.inspect}. " \
+ "Must be one of: #{VALID_TYPES.join(", ")}"
+ end
+ type
+ end
+
+ def extract_tool_calls!(data)
+ raw = data["tool_calls"]
+ if raw.nil? || !raw.is_a?(Array) || raw.empty?
+ raise InvalidPlaybookError,
+ "Step ##{@step_id} (tool_calls) requires a non-empty 'tool_calls' array"
+ end
+
+ raw.map.with_index do |tc, idx|
+ validate_tool_call!(tc, idx)
+ end
+ end
+
+ def validate_tool_call!(tc, idx)
+ unless tc.is_a?(Hash)
+ raise InvalidPlaybookError,
+ "Step ##{@step_id}, tool_call[#{idx}] must be a JSON object"
+ end
+
+ %w[id name arguments].each do |field|
+ if tc[field].nil?
+ raise InvalidPlaybookError,
+ "Step ##{@step_id}, tool_call[#{idx}] is missing required field '#{field}'"
+ end
+ end
+
+ unless tc["id"].is_a?(String)
+ raise InvalidPlaybookError,
+ "Step ##{@step_id}, tool_call[#{idx}] 'id' must be a string"
+ end
+
+ unless tc["name"].is_a?(String)
+ raise InvalidPlaybookError,
+ "Step ##{@step_id}, tool_call[#{idx}] 'name' must be a string"
+ end
+
+ unless tc["arguments"].is_a?(Hash)
+ raise InvalidPlaybookError,
+ "Step ##{@step_id}, tool_call[#{idx}] 'arguments' must be a JSON object"
+ end
+
+ tc
+ end
+
+ def validate_message_has_content!
+ return if @content.is_a?(String) && [email protected]?
+
+ raise InvalidPlaybookError,
+ "Step ##{@step_id} (message) requires a non-empty 'content' string"
+ end
+ end
+ end
+ end
+end
diff --git a/lib/dispatch/adapter/tester/version.rb b/lib/dispatch/adapter/tester/version.rb
new file mode 100644
index 0000000..62e6918
--- /dev/null
+++ b/lib/dispatch/adapter/tester/version.rb
@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+module Dispatch
+ module Adapter
+ module Tester
+ VERSION = "0.1.0"
+ end
+ end
+end
diff --git a/sig/dispatch/adapter/tester.rbs b/sig/dispatch/adapter/tester.rbs
new file mode 100644
index 0000000..87ca87a
--- /dev/null
+++ b/sig/dispatch/adapter/tester.rbs
@@ -0,0 +1,8 @@
+module Dispatch
+ module Adapter
+ module Tester
+ VERSION: String
+ # See the writing guide of rbs: https://github.com/ruby/rbs#guides
+ end
+ end
+end
diff --git a/spec/dispatch/adapter/tester/errors_spec.rb b/spec/dispatch/adapter/tester/errors_spec.rb
new file mode 100644
index 0000000..cb712cd
--- /dev/null
+++ b/spec/dispatch/adapter/tester/errors_spec.rb
@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+RSpec.describe Dispatch::Adapter::Tester do
+ describe "error hierarchy" do
+ it "Error inherits from StandardError" do
+ expect(Dispatch::Adapter::Tester::Error.superclass).to eq(StandardError)
+ end
+
+ it "InvalidPlaybookError inherits from Error" do
+ expect(Dispatch::Adapter::Tester::InvalidPlaybookError.superclass).to eq(Dispatch::Adapter::Tester::Error)
+ end
+
+ it "PlaybookExhaustedError inherits from Error" do
+ expect(Dispatch::Adapter::Tester::PlaybookExhaustedError.superclass).to eq(Dispatch::Adapter::Tester::Error)
+ end
+
+ it "UnconsumedStepsError inherits from Error" do
+ expect(Dispatch::Adapter::Tester::UnconsumedStepsError.superclass).to eq(Dispatch::Adapter::Tester::Error)
+ end
+ end
+
+ describe Dispatch::Adapter::Tester::PlaybookExhaustedError do
+ it "includes step details in the message" do
+ error = described_class.new(total_steps: 3, calls_made: 4)
+
+ expect(error.total_steps).to eq(3)
+ expect(error.calls_made).to eq(4)
+ expect(error.message).to include("all 3 steps")
+ expect(error.message).to include("4th time")
+ end
+
+ it "handles ordinal suffixes correctly" do
+ expect(described_class.new(total_steps: 1, calls_made: 1).message).to include("1st")
+ expect(described_class.new(total_steps: 1, calls_made: 2).message).to include("2nd")
+ expect(described_class.new(total_steps: 1, calls_made: 3).message).to include("3rd")
+ expect(described_class.new(total_steps: 1, calls_made: 11).message).to include("11th")
+ expect(described_class.new(total_steps: 1, calls_made: 12).message).to include("12th")
+ expect(described_class.new(total_steps: 1, calls_made: 13).message).to include("13th")
+ expect(described_class.new(total_steps: 1, calls_made: 21).message).to include("21st")
+ end
+ end
+
+ describe Dispatch::Adapter::Tester::UnconsumedStepsError do
+ it "includes remaining step details in the message" do
+ error = described_class.new(total_steps: 5, consumed: 2, remaining_step_ids: [3, 4, 5])
+
+ expect(error.total_steps).to eq(5)
+ expect(error.consumed).to eq(2)
+ expect(error.remaining_step_ids).to eq([3, 4, 5])
+ expect(error.message).to include("2/5 steps consumed")
+ expect(error.message).to include("[3, 4, 5]")
+ end
+ end
+end
diff --git a/spec/dispatch/adapter/tester/playbook_spec.rb b/spec/dispatch/adapter/tester/playbook_spec.rb
new file mode 100644
index 0000000..fc3ecd9
--- /dev/null
+++ b/spec/dispatch/adapter/tester/playbook_spec.rb
@@ -0,0 +1,360 @@
+# frozen_string_literal: true
+
+RSpec.describe Dispatch::Adapter::Tester::Playbook do
+ let(:message_step) do
+ { "step" => 1, "type" => "message", "content" => "Hello from the AI" }
+ end
+
+ let(:tool_call_step) do
+ {
+ "step" => 2,
+ "type" => "tool_calls",
+ "content" => nil,
+ "tool_calls" => [
+ {
+ "id" => "tc_read_001",
+ "name" => "read_file",
+ "arguments" => { "path" => "/some/file.rb" }
+ }
+ ]
+ }
+ end
+
+ let(:tool_call_with_content_step) do
+ {
+ "step" => 3,
+ "type" => "tool_calls",
+ "content" => "I will read the file for you.",
+ "tool_calls" => [
+ {
+ "id" => "tc_read_002",
+ "name" => "read_file",
+ "arguments" => { "path" => "/other/file.rb" }
+ }
+ ]
+ }
+ end
+
+ let(:multi_tool_step) do
+ {
+ "step" => 4,
+ "type" => "tool_calls",
+ "content" => nil,
+ "tool_calls" => [
+ {
+ "id" => "tc_write_001",
+ "name" => "write_file",
+ "arguments" => { "path" => "/a.rb", "content" => "hello" }
+ },
+ {
+ "id" => "tc_write_002",
+ "name" => "write_file",
+ "arguments" => { "path" => "/b.rb", "content" => "world" }
+ }
+ ]
+ }
+ end
+
+ def build_adapter(steps, **kwargs)
+ described_class.new(steps_json: JSON.generate(steps), **kwargs)
+ end
+
+ describe "#initialize" do
+ it "parses a valid JSON string of steps" do
+ adapter = build_adapter([message_step])
+ expect(adapter.steps.length).to eq(1)
+ expect(adapter.steps.first.step_id).to eq(1)
+ end
+
+ it "accepts an array directly instead of a JSON string" do
+ adapter = described_class.new(steps_json: [message_step])
+ expect(adapter.steps.length).to eq(1)
+ end
+
+ it "accepts empty steps" do
+ adapter = build_adapter([])
+ expect(adapter.steps).to be_empty
+ end
+
+ it "absorbs extra keyword arguments for drop-in compat" do
+ expect do
+ described_class.new(
+ steps_json: "[]",
+ model: "gpt-4",
+ max_tokens: 50_000,
+ min_request_interval: 3.0,
+ rate_limit: nil
+ )
+ end.not_to raise_error
+ end
+
+ it "raises on invalid JSON" do
+ expect do
+ described_class.new(steps_json: "not json")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /Failed to parse/)
+ end
+
+ it "raises when JSON is not an array" do
+ expect do
+ described_class.new(steps_json: '{"step": 1}')
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /must be an array/)
+ end
+
+ it "raises on duplicate step IDs" do
+ steps = [
+ { "step" => 1, "type" => "message", "content" => "a" },
+ { "step" => 1, "type" => "message", "content" => "b" }
+ ]
+ expect do
+ build_adapter(steps)
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /Duplicate step IDs/)
+ end
+ end
+
+ describe "#chat" do
+ context "with a message step" do
+ it "returns a Response with content and end_turn stop_reason" do
+ adapter = build_adapter([message_step])
+ response = adapter.chat([])
+
+ expect(response).to be_a(Dispatch::Adapter::Response)
+ expect(response.content).to eq("Hello from the AI")
+ expect(response.stop_reason).to eq(:end_turn)
+ expect(response.tool_calls).to be_empty
+ end
+
+ it "reports zero token usage" do
+ adapter = build_adapter([message_step])
+ response = adapter.chat([])
+
+ expect(response.usage.input_tokens).to eq(0)
+ expect(response.usage.output_tokens).to eq(0)
+ end
+ end
+
+ context "with a tool_calls step" do
+ it "returns a Response with tool_calls and tool_use stop_reason" do
+ adapter = build_adapter([tool_call_step])
+ response = adapter.chat([])
+
+ expect(response.stop_reason).to eq(:tool_use)
+ expect(response.tool_calls.length).to eq(1)
+
+ tc = response.tool_calls.first
+ expect(tc).to be_a(Dispatch::Adapter::ToolUseBlock)
+ expect(tc.id).to eq("tc_read_001")
+ expect(tc.name).to eq("read_file")
+ expect(tc.arguments).to eq({ "path" => "/some/file.rb" })
+ end
+
+ it "returns nil content when content is nil" do
+ adapter = build_adapter([tool_call_step])
+ response = adapter.chat([])
+
+ expect(response.content).to be_nil
+ end
+ end
+
+ context "with a tool_calls step that has content" do
+ it "returns both content and tool_calls" do
+ adapter = build_adapter([tool_call_with_content_step])
+ response = adapter.chat([])
+
+ expect(response.content).to eq("I will read the file for you.")
+ expect(response.tool_calls.length).to eq(1)
+ expect(response.stop_reason).to eq(:tool_use)
+ end
+ end
+
+ context "with multiple tool calls in a single step" do
+ it "returns all tool calls" do
+ adapter = build_adapter([multi_tool_step])
+ response = adapter.chat([])
+
+ expect(response.tool_calls.length).to eq(2)
+ expect(response.tool_calls.map(&:id)).to eq(%w[tc_write_001 tc_write_002])
+ expect(response.tool_calls.map(&:name)).to eq(%w[write_file write_file])
+ end
+ end
+
+ context "with a multi-step playbook" do
+ it "consumes steps sequentially" do
+ adapter = build_adapter([message_step, tool_call_step])
+
+ r1 = adapter.chat([])
+ expect(r1.content).to eq("Hello from the AI")
+ expect(r1.stop_reason).to eq(:end_turn)
+
+ r2 = adapter.chat([])
+ expect(r2.stop_reason).to eq(:tool_use)
+ expect(r2.tool_calls.first.id).to eq("tc_read_001")
+ end
+
+ it "advances the current_index" do
+ adapter = build_adapter([message_step, tool_call_step])
+ expect(adapter.current_index).to eq(0)
+
+ adapter.chat([])
+ expect(adapter.current_index).to eq(1)
+
+ adapter.chat([])
+ expect(adapter.current_index).to eq(2)
+ end
+ end
+
+ context "when the playbook is exhausted" do
+ it "raises PlaybookExhaustedError with step details" do
+ adapter = build_adapter([message_step])
+ adapter.chat([])
+
+ expect do
+ adapter.chat([])
+ end.to raise_error(Dispatch::Adapter::Tester::PlaybookExhaustedError) do |error|
+ expect(error.total_steps).to eq(1)
+ expect(error.calls_made).to eq(2)
+ expect(error.message).to include("all 1 steps")
+ expect(error.message).to include("2nd time")
+ end
+ end
+ end
+
+ it "records each call in the call_log" do
+ adapter = build_adapter([message_step])
+ messages = [Dispatch::Adapter::Message.new(role: "user", content: "hi")]
+ tools = [{ name: "test", description: "test", parameters: {} }]
+
+ adapter.chat(messages, system: "You are helpful", tools: tools)
+
+ expect(adapter.call_log.length).to eq(1)
+ log_entry = adapter.call_log.first
+ expect(log_entry[:system]).to eq("You are helpful")
+ expect(log_entry[:tools]).to eq(tools)
+ expect(log_entry[:step].step_id).to eq(1)
+ end
+
+ it "reports the configured model name in responses" do
+ adapter = build_adapter([message_step], model: "custom-model")
+ response = adapter.chat([])
+
+ expect(response.model).to eq("custom-model")
+ end
+ end
+
+ describe "#model_name" do
+ it "returns the default model name" do
+ adapter = build_adapter([])
+ expect(adapter.model_name).to eq("tester-playbook")
+ end
+
+ it "returns a custom model name" do
+ adapter = build_adapter([], model: "gpt-4")
+ expect(adapter.model_name).to eq("gpt-4")
+ end
+ end
+
+ describe "#provider_name" do
+ it "returns 'Tester Playbook'" do
+ adapter = build_adapter([])
+ expect(adapter.provider_name).to eq("Tester Playbook")
+ end
+ end
+
+ describe "#max_context_tokens" do
+ it "returns a large context window" do
+ adapter = build_adapter([])
+ expect(adapter.max_context_tokens).to eq(1_000_000)
+ end
+ end
+
+ describe "#list_models" do
+ it "returns a single ModelInfo entry" do
+ adapter = build_adapter([])
+ models = adapter.list_models
+
+ expect(models.length).to eq(1)
+ expect(models.first).to be_a(Dispatch::Adapter::ModelInfo)
+ expect(models.first.id).to eq("tester-playbook")
+ expect(models.first.supports_tool_use).to be true
+ end
+ end
+
+ describe "#finished?" do
+ it "returns false when steps remain" do
+ adapter = build_adapter([message_step])
+ expect(adapter.finished?).to be false
+ end
+
+ it "returns true when all steps consumed" do
+ adapter = build_adapter([message_step])
+ adapter.chat([])
+ expect(adapter.finished?).to be true
+ end
+
+ it "returns true for empty playbook" do
+ adapter = build_adapter([])
+ expect(adapter.finished?).to be true
+ end
+ end
+
+ describe "#remaining_steps" do
+ it "tracks remaining step count" do
+ adapter = build_adapter([message_step, tool_call_step])
+ expect(adapter.remaining_steps).to eq(2)
+
+ adapter.chat([])
+ expect(adapter.remaining_steps).to eq(1)
+
+ adapter.chat([])
+ expect(adapter.remaining_steps).to eq(0)
+ end
+ end
+
+ describe "#verify_all_consumed!" do
+ it "does not raise when all steps consumed" do
+ adapter = build_adapter([message_step])
+ adapter.chat([])
+
+ expect { adapter.verify_all_consumed! }.not_to raise_error
+ end
+
+ it "raises UnconsumedStepsError with remaining step IDs" do
+ adapter = build_adapter([message_step, tool_call_step])
+ adapter.chat([])
+
+ expect do
+ adapter.verify_all_consumed!
+ end.to raise_error(Dispatch::Adapter::Tester::UnconsumedStepsError) do |error|
+ expect(error.total_steps).to eq(2)
+ expect(error.consumed).to eq(1)
+ expect(error.remaining_step_ids).to eq([2])
+ end
+ end
+
+ it "does not raise for empty playbook" do
+ adapter = build_adapter([])
+ expect { adapter.verify_all_consumed! }.not_to raise_error
+ end
+ end
+
+ describe "#reset!" do
+ it "resets the index and call_log" do
+ adapter = build_adapter([message_step])
+ adapter.chat([])
+
+ adapter.reset!
+
+ expect(adapter.current_index).to eq(0)
+ expect(adapter.call_log).to be_empty
+ expect(adapter.finished?).to be false
+ end
+
+ it "allows replaying the playbook" do
+ adapter = build_adapter([message_step])
+ r1 = adapter.chat([])
+ adapter.reset!
+ r2 = adapter.chat([])
+
+ expect(r1.content).to eq(r2.content)
+ end
+ end
+end
diff --git a/spec/dispatch/adapter/tester/step_spec.rb b/spec/dispatch/adapter/tester/step_spec.rb
new file mode 100644
index 0000000..df5cb1e
--- /dev/null
+++ b/spec/dispatch/adapter/tester/step_spec.rb
@@ -0,0 +1,185 @@
+# frozen_string_literal: true
+
+RSpec.describe Dispatch::Adapter::Tester::Step do
+ describe "validation" do
+ it "parses a valid message step" do
+ step = described_class.new("step" => 1, "type" => "message", "content" => "Hello")
+
+ expect(step.step_id).to eq(1)
+ expect(step.type).to eq("message")
+ expect(step.content).to eq("Hello")
+ expect(step).to be_message
+ expect(step).not_to be_tool_calls
+ expect(step.tool_calls).to be_empty
+ end
+
+ it "parses a valid tool_calls step" do
+ step = described_class.new(
+ "step" => 2,
+ "type" => "tool_calls",
+ "tool_calls" => [
+ { "id" => "tc_001", "name" => "read_file", "arguments" => { "path" => "/a.rb" } }
+ ]
+ )
+
+ expect(step.step_id).to eq(2)
+ expect(step).to be_tool_calls
+ expect(step.tool_calls.length).to eq(1)
+ expect(step.tool_calls.first["id"]).to eq("tc_001")
+ end
+
+ it "allows content on tool_calls steps" do
+ step = described_class.new(
+ "step" => 3,
+ "type" => "tool_calls",
+ "content" => "I will do something",
+ "tool_calls" => [
+ { "id" => "tc_001", "name" => "test", "arguments" => {} }
+ ]
+ )
+
+ expect(step.content).to eq("I will do something")
+ end
+
+ it "is frozen after initialization" do
+ step = described_class.new("step" => 1, "type" => "message", "content" => "hi")
+ expect(step).to be_frozen
+ end
+
+ it "has a useful to_s" do
+ step = described_class.new("step" => 42, "type" => "message", "content" => "hi")
+ expect(step.to_s).to eq("Step #42 (message)")
+ end
+
+ context "missing fields" do
+ it "raises when step ID is missing" do
+ expect do
+ described_class.new("type" => "message", "content" => "hi")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /missing required field 'step'/)
+ end
+
+ it "raises when step ID is not an integer" do
+ expect do
+ described_class.new("step" => "one", "type" => "message", "content" => "hi")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /must be an integer/)
+ end
+
+ it "raises when type is missing" do
+ expect do
+ described_class.new("step" => 1, "content" => "hi")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /missing required field 'type'/)
+ end
+
+ it "raises when type is invalid" do
+ expect do
+ described_class.new("step" => 1, "type" => "invalid", "content" => "hi")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /invalid type/)
+ end
+ end
+
+ context "message step validation" do
+ it "raises when content is missing" do
+ expect do
+ described_class.new("step" => 1, "type" => "message")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /requires a non-empty 'content'/)
+ end
+
+ it "raises when content is empty string" do
+ expect do
+ described_class.new("step" => 1, "type" => "message", "content" => "")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /requires a non-empty 'content'/)
+ end
+
+ it "raises when content is not a string" do
+ expect do
+ described_class.new("step" => 1, "type" => "message", "content" => 123)
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /requires a non-empty 'content'/)
+ end
+ end
+
+ context "tool_calls step validation" do
+ it "raises when tool_calls array is missing" do
+ expect do
+ described_class.new("step" => 1, "type" => "tool_calls")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /requires a non-empty 'tool_calls' array/)
+ end
+
+ it "raises when tool_calls array is empty" do
+ expect do
+ described_class.new("step" => 1, "type" => "tool_calls", "tool_calls" => [])
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /requires a non-empty 'tool_calls' array/)
+ end
+
+ it "raises when tool_call is missing 'id'" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "name" => "test", "arguments" => {} }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /missing required field 'id'/)
+ end
+
+ it "raises when tool_call is missing 'name'" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "id" => "tc_001", "arguments" => {} }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /missing required field 'name'/)
+ end
+
+ it "raises when tool_call is missing 'arguments'" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "id" => "tc_001", "name" => "test" }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /missing required field 'arguments'/)
+ end
+
+ it "raises when tool_call 'id' is not a string" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "id" => 1, "name" => "test", "arguments" => {} }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /'id' must be a string/)
+ end
+
+ it "raises when tool_call 'name' is not a string" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "id" => "tc_001", "name" => 123, "arguments" => {} }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /'name' must be a string/)
+ end
+
+ it "raises when tool_call 'arguments' is not a hash" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => [{ "id" => "tc_001", "name" => "test", "arguments" => "bad" }]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /'arguments' must be a JSON object/)
+ end
+
+ it "raises when tool_call entry is not a hash" do
+ expect do
+ described_class.new(
+ "step" => 1, "type" => "tool_calls",
+ "tool_calls" => ["not a hash"]
+ )
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /must be a JSON object/)
+ end
+ end
+
+ context "non-hash input" do
+ it "raises when step data is not a hash" do
+ expect do
+ described_class.new("just a string")
+ end.to raise_error(Dispatch::Adapter::Tester::InvalidPlaybookError, /must be a JSON object/)
+ end
+ end
+ end
+end
diff --git a/spec/dispatch/adapter/tester_spec.rb b/spec/dispatch/adapter/tester_spec.rb
new file mode 100644
index 0000000..9b0bbea
--- /dev/null
+++ b/spec/dispatch/adapter/tester_spec.rb
@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+RSpec.describe Dispatch::Adapter::Tester do
+ it "has a version number" do
+ expect(Dispatch::Adapter::Tester::VERSION).not_to be_nil
+ end
+
+ it "exposes the Playbook class" do
+ expect(Dispatch::Adapter::Tester::Playbook).to be_a(Class)
+ end
+
+ it "exposes the Step class" do
+ expect(Dispatch::Adapter::Tester::Step).to be_a(Class)
+ end
+
+ it "exposes error classes" do
+ expect(Dispatch::Adapter::Tester::Error).to be < StandardError
+ expect(Dispatch::Adapter::Tester::InvalidPlaybookError).to be < Dispatch::Adapter::Tester::Error
+ expect(Dispatch::Adapter::Tester::PlaybookExhaustedError).to be < Dispatch::Adapter::Tester::Error
+ expect(Dispatch::Adapter::Tester::UnconsumedStepsError).to be < Dispatch::Adapter::Tester::Error
+ end
+end
diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb
new file mode 100644
index 0000000..e638f1a
--- /dev/null
+++ b/spec/spec_helper.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+require "dispatch/adapter/interface"
+require "dispatch/adapter/tester"
+
+RSpec.configure do |config|
+ config.example_status_persistence_file_path = ".rspec_status"
+ config.disable_monkey_patching!
+
+ config.expect_with :rspec do |c|
+ c.syntax = :expect
+ end
+end