blumia/gammaray-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

use uv to run test, more fixes

Browse Source
This commit is contained in:
Wang Zichong
2026-07-07 23:43:48 +08:00
parent f7d6a51220
commit 268483577e
16 changed files with 705 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
tests/README.md
View File

@@ -1,31 +1,15 @@
# Test Suite
Integration tests for the gammaray-mcp bridge, written in Python using `pytest`. Tests communicate with the bridge over stdio JSON-RPC (the same protocol MCP clients use), exercising every tool end-to-end against a real GammaRay probe.
Integration tests for the gammaray-mcp bridge, written in Python using `pytest`.
Tests communicate with the bridge over stdio JSON-RPC, exercising every MCP tool
end-to-end against a real GammaRay probe.
## Prerequisites
- The bridge must be built: `cmake --build bridge/build`
- `pytest` (install via `pip3 install pytest` or `pip install pytest` in a venv)
- For probe-dependent tests: a QML test app at `~/Sources/testqml/relayout.qml` (configurable via `TEST_QML` env)
## Running
## Quick start
```bash
# From the project root:
./tests/run_tests.sh --no-probe # unit tests only (no probe needed)
./tests/run_tests.sh -v # full suite, verbose
./tests/run_tests.sh -k "item" # run only item-related tests
./tests/run_tests.sh --no-probe -x # stop on first failure, no probe
```
Or directly with pytest:
```bash
cd tests
LD_LIBRARY_PATH=../install-prefix/lib \
QT_QPA_PLATFORM=offscreen \
QT_PLUGIN_PATH=../bridge/build/lib/x86_64-linux-gnu/qt6/plugins \
pytest -v
./run_tests.sh --no-probe # unit tests only (no probe needed)
./run_tests.sh -v # full suite, verbose
./run_tests.sh -k "item" # item-related tests only
```
## Test files
@@ -37,19 +21,51 @@ pytest -v
| `test_items.py` | `selectQuickItem`, `getItemProperties`, position, visual properties, groups | Yes |
| `test_scenegraph.py` | `selectScenegraphNode`, `getNodeVertices`, `getNodeAdjacency`, `getMaterialShaders`, `getMaterialProperties`, `setRenderMode`, `setSlowMode` | Yes |
## Test QML application
`testapp/main.qml` is a simple Qt Quick Controls application that exercises all
the tools: buttons, switches, sliders, a list view with delegate items, and
visibility/opacity controls.
## Configuration
**`run_tests.sh` is the single configuration point.** All paths default to
project-local locations. When GammaRay is installed elsewhere (e.g. from a
system package), override these by editing the variables at the top of
`run_tests.sh`:
| Variable | Default | Description |
|---|---|---|
| `GAMMARAY_BIN` | `install-prefix/bin/gammaray` | Path to the probe binary |
| `GAMMARAY_LIB_DIR` | `install-prefix/lib` | Runtime library directory for GammaRay |
| `QML_RUNNER` | `/usr/lib/qt6/bin/qml` | QML runtime executable |
| `TEST_QML` | `testapp/main.qml` | Test QML application |
| `BRIDGE_EXE` | `bridge/build/qml-sg-mcp-bridge` | Bridge executable |
| `BRIDGE_RUN_SCRIPT` | `bridge/run.sh` | Bridge wrapper script |
You can also override any of these via environment variables when calling
`run_tests.sh` or `pytest` directly:
```bash
GAMMARAY_BIN=/usr/bin/gammaray GAMMARAY_LIB_DIR=/usr/lib/x86_64-linux-gnu \
./run_tests.sh --no-probe
```
## pytest options
| Flag | Default | Description |
|---|---|---|
| `--no-probe` | `false` | Skip all probe-dependent tests |
| `--probe-timeout` | 15s | Seconds to wait for probe to become ready |
## Markers
- `@pytest.mark.probe` — test requires a running GammaRay probe. Automatically skipped with `--no-probe`.
- `@pytest.mark.probe` — test requires a running GammaRay probe.
Automatically skipped with `--no-probe`.
## Fixtures
- `bridge` — fresh `McpClient` instance per test (initialized, not connected)
- `connected_bridge` — bridge connected to a probe (via `connectProbeDefault`)
- `probe_process` — session-scoped, starts a probe via `systemd-run` (fallback: direct spawn) and waits for `ready()`
## Options
| Flag | Default | Description |
|---|---|---|
| `--no-probe` | `false` | Skip all probe-dependent tests |
| `--probe-timeout` | 15s | Seconds to wait for probe to become ready |
- `probe_process` — session-scoped, starts a probe via `systemd-run` (fallback:
direct spawn) and waits for `ready()`

103
tests/conftest.py
View File

@@ -1,4 +1,16 @@
"""pytest fixtures for gammaray-mcp bridge tests."""
"""pytest fixtures for gammaray-mcp bridge tests.
All paths are configurable via environment variables. run_tests.sh is the
canonical configuration point — it sets sensible defaults and exports them
before calling pytest. If you prefer to run pytest directly, set these:
GAMMARAY_BIN — path to the gammaray probe binary
GAMMARAY_LIB_DIR — directory containing GammaRay shared libraries
QML_RUNNER — path to the QML runtime executable
TEST_QML — path to the test QML application
BRIDGE_EXE — path to the bridge executable
BRIDGE_RUN_SCRIPT — path to bridge/run.sh
"""
import os
import subprocess
@@ -10,11 +22,33 @@ import pytest
from mcp_client import McpClient
PROJECT_ROOT = Path(__file__).resolve().parent.parent
BRIDGE_EXE = PROJECT_ROOT / "bridge" / "build" / "qml-sg-mcp-bridge"
RUN_SCRIPT = PROJECT_ROOT / "bridge" / "run.sh"
PROBE_BIN = PROJECT_ROOT / "install-prefix" / "bin" / "gammaray"
TEST_QML = Path(os.environ.get("TEST_QML", str(Path.home() / "Sources" / "testqml" / "relayout.qml")))
QML_RUNNER = "/usr/lib/qt6/bin/qml"
# All paths read from env vars, with sensible project-local defaults.
# Users only need to override these when GammaRay is installed elsewhere.
GAMMARAY_BIN = Path(os.environ.get(
"GAMMARAY_BIN",
str(PROJECT_ROOT / "install-prefix" / "bin" / "gammaray"),
))
GAMMARAY_LIB_DIR = os.environ.get(
"GAMMARAY_LIB_DIR",
str(PROJECT_ROOT / "install-prefix" / "lib"),
)
QML_RUNNER = Path(os.environ.get(
"QML_RUNNER",
"/usr/lib/qt6/bin/qml",
))
TEST_QML = Path(os.environ.get(
"TEST_QML",
str(PROJECT_ROOT / "tests" / "testapp" / "main.qml"),
))
BRIDGE_EXE = Path(os.environ.get(
"BRIDGE_EXE",
str(PROJECT_ROOT / "bridge" / "build" / "qml-sg-mcp-bridge"),
))
BRIDGE_RUN_SCRIPT = Path(os.environ.get(
"BRIDGE_RUN_SCRIPT",
str(PROJECT_ROOT / "bridge" / "run.sh"),
))
def pytest_addoption(parser):
@@ -42,7 +76,6 @@ def pytest_collection_modifyitems(config, items):
@pytest.fixture(scope="session")
def bridge_path() -> Path:
"""Path to the bridge executable."""
if not BRIDGE_EXE.exists():
pytest.skip(f"Bridge not built at {BRIDGE_EXE}")
return BRIDGE_EXE
@@ -50,20 +83,19 @@ def bridge_path() -> Path:
@pytest.fixture(scope="session")
def run_script_path() -> Path:
"""Path to the run.sh wrapper."""
if not RUN_SCRIPT.exists():
pytest.skip(f"run.sh not found at {RUN_SCRIPT}")
return RUN_SCRIPT
if not BRIDGE_RUN_SCRIPT.exists():
pytest.skip(f"run.sh not found at {BRIDGE_RUN_SCRIPT}")
return BRIDGE_RUN_SCRIPT
@pytest.fixture(scope="session")
@pytest.fixture(scope="module")
def probe_process(request) -> subprocess.Popen | None:
"""Start a GammaRay probe process (session-scoped, one per test run)."""
if request.config.getoption("--no-probe"):
return None
if not PROBE_BIN.exists():
pytest.skip(f"GammaRay probe binary not found at {PROBE_BIN}")
if not GAMMARAY_BIN.exists():
pytest.skip(f"GammaRay probe binary not found at {GAMMARAY_BIN}")
if not TEST_QML.exists():
pytest.skip(f"Test QML file not found at {TEST_QML}")
if not QML_RUNNER.exists():
@@ -71,7 +103,7 @@ def probe_process(request) -> subprocess.Popen | None:
env = os.environ.copy()
env["QT_QPA_PLATFORM"] = "offscreen"
env["LD_LIBRARY_PATH"] = str(PROJECT_ROOT / "install-prefix" / "lib")
env["LD_LIBRARY_PATH"] = GAMMARAY_LIB_DIR
proc = None
used_systemd = False
@@ -94,20 +126,21 @@ def probe_process(request) -> subprocess.Popen | None:
"--working-directory", str(TEST_QML.parent),
"-E", f"QT_QPA_PLATFORM={env['QT_QPA_PLATFORM']}",
"-E", f"LD_LIBRARY_PATH={env['LD_LIBRARY_PATH']}",
str(PROBE_BIN),
str(GAMMARAY_BIN),
"--inject-only", "--listen", "tcp://127.0.0.1:11732",
"--injector", "preload",
QML_RUNNER, str(TEST_QML),
str(QML_RUNNER), str(TEST_QML),
],
capture_output=True, text=True, timeout=10,
)
if result.returncode == 0:
used_systemd = True
proc = subprocess.Popen(["true"]) # sentinel: probe is external
else:
# Fallback: direct spawn
proc = subprocess.Popen(
[str(PROBE_BIN), "--inject-only", "--listen", "tcp://127.0.0.1:11732",
"--injector", "preload", QML_RUNNER, str(TEST_QML)],
[str(GAMMARAY_BIN), "--inject-only", "--listen", "tcp://127.0.0.1:11732",
"--injector", "preload", str(QML_RUNNER), str(TEST_QML)],
env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)
@@ -156,15 +189,35 @@ def bridge(run_script_path) -> McpClient:
c = McpClient(bridge_path=run_script_path)
c.initialize()
yield c
# Gracefully disconnect from the probe before killing the bridge.
# GammaRay's QTcpSocket::disconnectFromHost() is asynchronous — the
# Endpoint::disconnected signal (which clears m_socket) fires on a later
# event loop iteration. If we kill the bridge (SIGTERM) while m_socket is
# still set, the probe does not immediately detect the disconnection, and
# the next test's bridge cannot connect. Sending disconnectProbe() first
# triggers a clean GammaRay protocol-level shutdown that waits for the
# async disconnect to complete, releasing the probe for the next test.
try:
c.call_tool("disconnectProbe", timeout=5)
except Exception:
pass # bridge may have already crashed; best-effort
c.close()
# Allow the probe to detect the disconnection before the next test
# connects.
time.sleep(1)
@pytest.fixture
def connected_bridge(bridge, probe_process) -> McpClient:
"""Bridge connected to a running probe."""
"""Bridge connected to a running probe (retries on transient failures)."""
if probe_process is None:
pytest.skip("No probe available")
result = bridge.call_tool("connectProbeDefault")
assert isinstance(result, dict), f"connectProbeDefault failed: {result}"
assert result.get("connected"), f"Not connected: {result}"
return bridge
pytest.skip("No probe available (try --no-probe to skip probe-dependent tests)")
deadline = time.monotonic() + 30
last_error = None
while time.monotonic() < deadline:
result = bridge.call_tool("connectProbeDefault")
if isinstance(result, dict) and result.get("connected"):
return bridge
last_error = result
time.sleep(3)
raise AssertionError(f"Failed to connect to probe after 30s: {last_error}")

17
tests/mcp_client.py
View File

@@ -30,11 +30,15 @@ class McpClient:
except subprocess.TimeoutExpired:
self._proc.kill()
@property
def returncode(self):
return self._proc.poll()
@property
def stderr(self):
return self._proc.stderr
def send_request(self, method: str, params: dict | None = None) -> dict:
def send_request(self, method: str, params: dict | None = None, timeout: float = 60.0) -> dict:
self._next_id += 1
req = {
"jsonrpc": "2.0",
@@ -45,14 +49,17 @@ class McpClient:
line = json.dumps(req)
self._proc.stdin.write(line + "\n")
self._proc.stdin.flush()
return self._read_response(self._next_id)
return self._read_response(self._next_id, timeout=timeout)
def _read_response(self, expected_id: int, timeout: float = 60.0) -> dict:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = self._proc.stdout.readline()
if not line:
raise RuntimeError("Bridge process terminated")
stderr = self._proc.stderr.read()
rc = self._proc.poll()
raise RuntimeError(
f"Bridge process terminated (rc={rc}, stderr: {stderr[-500:] if stderr else 'none'})")
resp = json.loads(line)
rid = resp.get("id")
if rid == expected_id:
@@ -74,8 +81,8 @@ class McpClient:
self._initialized = True
return resp
def call_tool(self, name: str, arguments: dict | None = None) -> dict:
resp = self.send_request("tools/call", {"name": name, "arguments": arguments or {}})
def call_tool(self, name: str, arguments: dict | None = None, timeout: float = 60.0) -> dict:
resp = self.send_request("tools/call", {"name": name, "arguments": arguments or {}}, timeout=timeout)
content = resp.get("result", {}).get("content", [])
if content and content[0].get("type") == "text":
text = content[0]["text"]

45
tests/run_tests.sh
View File

@@ -1,34 +1,57 @@
#!/usr/bin/env bash
# Run the gammaray-mcp test suite.
# Convenience script to run the gammaray-mcp test suite.
#
# This is the SINGLE configuration point for paths. If you install GammaRay
# from a system package instead of the local install-prefix, change the
# variables below. Everything else (conftest.py, the QML test app, etc.)
# reads from these environment variables.
#
# Usage:
# ./run_tests.sh # probe-required tests (default)
# ./run_tests.sh --no-probe # only unit tests (no probe needed)
# ./run_tests.sh -v # verbose
# ./run_tests.sh -k "item" # run only item-related tests
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
# ---------------------------------------------------------------------------
# Configurable paths — adjust these if GammaRay is installed elsewhere
# ---------------------------------------------------------------------------
: "${GAMMARAY_BIN:=$PROJECT_ROOT/install-prefix/bin/gammaray}"
: "${GAMMARAY_LIB_DIR:=$PROJECT_ROOT/install-prefix/lib}"
: "${QML_RUNNER:=/usr/lib/qt6/bin/qml}"
: "${TEST_QML:=$SCRIPT_DIR/testapp/main.qml}"
: "${BRIDGE_EXE:=$PROJECT_ROOT/bridge/build/qml-sg-mcp-bridge}"
: "${BRIDGE_RUN_SCRIPT:=$PROJECT_ROOT/bridge/run.sh}"
# Export so conftest.py can read them
export GAMMARAY_BIN
export GAMMARAY_LIB_DIR
export QML_RUNNER
export TEST_QML
export BRIDGE_EXE
export BRIDGE_RUN_SCRIPT
# ---------------------------------------------------------------------------
# Ensure the bridge is built
if [ ! -f "$PROJECT_ROOT/bridge/build/qml-sg-mcp-bridge" ]; then
# ---------------------------------------------------------------------------
if [ ! -f "$BRIDGE_EXE" ]; then
echo "Building bridge first..."
cmake -S "$PROJECT_ROOT/bridge" -B "$PROJECT_ROOT/bridge/build" -G Ninja \
-DCMAKE_PREFIX_PATH="$PROJECT_ROOT/install-prefix"
cmake --build "$PROJECT_ROOT/bridge/build"
fi
# Use the same env as run.sh
export LD_LIBRARY_PATH="$PROJECT_ROOT/install-prefix/lib"
# ---------------------------------------------------------------------------
# Runtime environment
# ---------------------------------------------------------------------------
export LD_LIBRARY_PATH="$GAMMARAY_LIB_DIR"
export QT_QPA_PLATFORM=offscreen
export QT_PLUGIN_PATH="$PROJECT_ROOT/bridge/build/lib/x86_64-linux-gnu/qt6/plugins"
cd "$SCRIPT_DIR"
# Install test dependencies if needed
python3 -c "import pytest" 2>/dev/null || {
echo "pytest not found. Installing..."
pip3 install pytest --user
}
exec python3 -m pytest "$@" .
exec uv run --frozen --directory "$PROJECT_ROOT" python "$SCRIPT_DIR/run_with_uv.py" "$@"

8
tests/run_with_uv.py Normal file
View File

@@ -0,0 +1,8 @@
"""Entry point for uv run: adds tests dir to path and runs pytest."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.resolve()))
import pytest
sys.exit(pytest.main(sys.argv[1:]))

37
tests/test_connection.py
View File

@@ -19,26 +19,43 @@ class TestConnection:
assert status.get("state") in ("disconnected", "connecting")
assert status.get("ready") is False
def test_connect_and_disconnect(self, connected_bridge):
# connectProbeDefault is already called by connected_bridge fixture
status = connected_bridge.call_tool("probeStatus")
def test_connect_and_disconnect(self, bridge, probe_process):
if probe_process is None:
pytest.skip("No probe available")
# Connect
result = bridge.call_tool("connectProbeDefault")
assert isinstance(result, dict), f"connectProbeDefault failed: {result}"
assert result.get("connected"), f"Not connected: {result}"
# Verify status
status = bridge.call_tool("probeStatus")
assert isinstance(status, dict)
assert status.get("state") == "ready"
assert status.get("ready") is True
assert "tcp://" in status.get("url", "")
# Now disconnect
result = connected_bridge.call_tool("disconnectProbe")
result = bridge.call_tool("disconnectProbe")
assert isinstance(result, dict)
assert result.get("state") == "disconnected"
def test_reconnect(self, connected_bridge):
def test_reconnect(self, bridge, probe_process):
if probe_process is None:
pytest.skip("No probe available")
bridge.call_tool("connectProbeDefault")
# Disconnect
connected_bridge.call_tool("disconnectProbe")
bridge.call_tool("disconnectProbe")
# Reconnect with explicit args
result = connected_bridge.call_tool("connectProbe",
{"host": "127.0.0.1", "port": 11732})
result = bridge.call_tool("connectProbe",
{"host": "127.0.0.1", "port": 11732})
assert isinstance(result, dict)
assert result.get("connected") is True
assert result.get("state") == "ready"
def test_connect_default_after_disconnect(self, bridge, probe_process):
if probe_process is None:
pytest.skip("No probe available")
bridge.call_tool("connectProbeDefault")
bridge.call_tool("disconnectProbe")
result = bridge.call_tool("connectProbeDefault")
assert isinstance(result, dict)
assert result.get("connected") is True
assert result.get("state") == "ready"

93
tests/testapp/main.qml Normal file
View File

@@ -0,0 +1,93 @@
import QtQuick
import QtQuick.Controls
import QtQuick.Layouts
ApplicationWindow {
visible: true
width: 600
height: 500
title: "GammaRay MCP Test App"
ColumnLayout {
anchors.fill: parent
anchors.margins: 8
spacing: 5
// Row of buttons at the top
RowLayout {
spacing: 6
Button { text: "Add Item"; onClicked: listModel.append({ name: "Item " + (listModel.count + 1) }) }
Button { text: "Remove Item"; onClicked: { if (listModel.count > 0) listModel.remove(listModel.count - 1) } }
Button { text: "Force Layout"; onClicked: listView.forceLayout() }
}
// Switch to toggle visibility
RowLayout {
spacing: 6
Label { text: "Show ListView:" }
Switch {
id: visibilitySwitch
checked: true
onCheckedChanged: listView.visible = checked
}
Label { text: "Opacity:" }
Slider {
id: opacitySlider
from: 0.0; to: 1.0; value: 1.0
onValueChanged: listView.opacity = value
}
}
// ListView with delegate items
Rectangle {
Layout.fillWidth: true
Layout.fillHeight: true
color: "#f0f0f0"
radius: 4
border.color: "#ccc"
border.width: 1
ListView {
id: listView
anchors.fill: parent
anchors.margins: 4
model: ListModel {
id: listModel
ListElement { name: "Alpha" }
ListElement { name: "Beta" }
ListElement { name: "Gamma" }
ListElement { name: "Delta" }
}
delegate: Rectangle {
width: parent.width
height: 32
color: index % 2 === 0 ? "#ffffff" : "#f5f5f5"
border.color: "#ddd"
border.width: 1
radius: 2
Text {
anchors.centerIn: parent
text: model.name
font.pixelSize: 14
}
}
}
}
// Bottom status bar
Rectangle {
Layout.fillWidth: true
Layout.preferredHeight: 24
color: "#e0e0e0"
radius: 2
Text {
anchors.centerIn: parent
text: "Items: " + listModel.count + " | Visible: " + listView.visible
font.pixelSize: 11
color: "#666"
}
}
}
}