613 lines
51 KiB
Markdown
613 lines
51 KiB
Markdown
# GammaRay MCP Bridge — Plan
|
||
|
||
## Goal
|
||
|
||
Build an MCP server that exposes GammaRay probe introspection data (QML SceneGraph, QML items, and Qt Widgets) to LLMs for assisted debugging. The bridge is a **GammaRay client**: it connects to a probe injected into the target Qt app, reads the same models the GammaRay GUI uses, and translates them into MCP JSON-RPC tool calls.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Target Qt/QML App
|
||
│ GammaRay probe injected (preload/attach), loads quickinspector + widgetinspector plugins
|
||
│
|
||
│ GammaRay binary protocol over TCP/local socket
|
||
│ (common/protocol.h — QDataStream-based, NOT JSON-RPC)
|
||
▼
|
||
GammaRay MCP Bridge (new project, GPL-2.0-or-later)
|
||
│ Links gammaray_client + gammaray_common (installed from source)
|
||
│ Uses ClientConnectionManager to connect, ObjectBroker to get models
|
||
│ Connection is lazy: bridge starts without a probe, MCP client calls
|
||
│ connectProbe(host, port) to establish; auto-reconnects on probe restart.
|
||
│
|
||
│ MCP JSON-RPC over stdio (or HTTP+SSE)
|
||
▼
|
||
LLM / AI Agent
|
||
```
|
||
|
||
The bridge is a GammaRay client peer to `gammaray-client` (GUI client) and `gammaray` (launcher+client GUI). It does NOT touch the probe side; it reuses GammaRay's existing probe injection + protocol + model sync.
|
||
|
||
## License
|
||
|
||
- GammaRay libraries are **GPL-2.0-or-later**. Linking `gammaray_client` makes the bridge a derivative work → **bridge must be GPL-compatible**.
|
||
- Accepted by user. No commercial license needed.
|
||
|
||
## GammaRay source location
|
||
|
||
The GammaRay source used for investigation: `/home/blumia/Sources/GammaRay` (version 3.4.0 per `version.txt`).
|
||
|
||
## Step 1: Build and install GammaRay from source [DONE]
|
||
|
||
System packages are too old (system has 2.11.3); protocol version must match between probe and client (`common/protocol.h:141` `Protocol::version()`). Built both probe and client libs from the 3.4.0 source.
|
||
|
||
Verified environment: Qt 6.8.0 (system), `qt6-base-private-dev` / `qt6-declarative-private-dev` installed, CMake 3.31, Ninja 1.12, GCC 12.3.
|
||
|
||
```bash
|
||
# Reused existing build/ dir; only reconfigured install prefix.
|
||
cmake -S /home/blumia/Sources/GammaRay -B /home/blumia/Sources/GammaRay/build \
|
||
-DCMAKE_INSTALL_PREFIX=$(pwd)/install-prefix
|
||
cmake --build /home/blumia/Sources/GammaRay/build # already built
|
||
cmake --install /home/blumia/Sources/GammaRay/build
|
||
```
|
||
|
||
Installed to `install-prefix/` in this project dir (NOT `$HOME/.local` as originally drafted — keeps the toolchain self-contained). Contents:
|
||
- `install-prefix/lib/libgammaray_{client,common,core,ui,...}-qt6_8-x86_64.so.3.4.0`
|
||
- `install-prefix/lib/cmake/GammaRay/` (GammaRayConfig.cmake + GammaRayTarget.cmake + GammaRayMacros.cmake)
|
||
- `install-prefix/bin/gammaray` (launcher+client)
|
||
- `install-prefix/include/gammaray/{client,common,core,launcher,ui}/`
|
||
|
||
Verified at runtime: `Protocol::version()` = 38, `Endpoint::defaultPort()` = 11732. `gammaray_client.so` runtime-links `libgammaray_ui-qt6_8-x86_64.so` (PRIVATE link, as the plan predicted) → `install-prefix/lib` must be on `LD_LIBRARY_PATH` at runtime.
|
||
|
||
**VERIFIED — exported target namespace (gotcha #8)**: `CMakeLists.txt:931-932` has `NAMESPACE GammaRay::` commented out. Confirmed in installed `GammaRayTarget.cmake:22`: targets are **unnamespaced** — `gammaray_client`, `gammaray_common`, `gammaray_core`, `gammaray_ui`, `gammaray_kitemmodels`, `gammaray_launcher`, `gammaray_launcher_ui`, `gammaray_kuserfeedback`, `gammaray_probe`. Use `gammaray_client` / `gammaray_common` directly (no `GammaRay::` prefix).
|
||
|
||
## Step 2: Create the bridge project
|
||
|
||
New working directory, separate from GammaRay source tree.
|
||
|
||
### Dependencies
|
||
|
||
1. **GammaRay** — installed from source (Step 1), provides `gammaray_client`, `gammaray_common`
|
||
2. **qtmcp** (https://github.com/signal-slot/qtmcp) — Qt-based MCP protocol implementation, eliminates hand-writing JSON-RPC/stdio/handshake
|
||
|
||
### Why qtmcp
|
||
|
||
qtmcp provides a complete MCP server framework (`QMcpServer`) with:
|
||
- Built-in `stdio` and `sse` backends — no manual stdin/stdout handling
|
||
- Protocol handshake + version negotiation (supports `2024-11-05` and `2025-03-26`)
|
||
- `registerToolSet()` + `Q_INVOKABLE` auto-exposes QObject methods as MCP tools
|
||
- Supports `QFuture<Result>` for async tool results (matches GammaRay's async APIs like `getShader()`)
|
||
- Supports `QImage` return values (matches GammaRay texture grab / screenshot)
|
||
- License: `LGPL-3.0 OR GPL-2.0 OR GPL-3.0` — choose GPL-2.0 to be compatible with GammaRay's GPL-2.0-or-later
|
||
|
||
### qtmcp — consumed via FetchContent (no separate clone/install needed)
|
||
|
||
**VERIFIED**: qtmcp works with `FetchContent_MakeAvailable()` despite using Qt BuildInternals (`qt_internal_project_setup()` + `qt_build_repo()`). Tested empirically: configure + build + link all succeed. The resulting targets are `Qt6::McpServer`, `Qt6::McpCommon`, `Qt6::McpClient` (with `Qt::` alias). Headers are generated into the consumer's build tree (`${CMAKE_BINARY_DIR}/include/QtMcpServer/`).
|
||
|
||
Skip examples/tests with the **Qt-prefixed** vars (not `BUILD_EXAMPLES`): `-DQT_BUILD_EXAMPLES=OFF -DQT_BUILD_TESTS=OFF` — verified to suppress the examples/ build dir.
|
||
|
||
Caveats of the FetchContent approach:
|
||
- qtmcp builds **shared** libs (`libQt6McpServer.so.6.10.2`) into the consumer's build tree. The bridge exe needs RPATH or `LD_LIBRARY_PATH` pointing at `${CMAKE_BINARY_DIR}/lib/x86_64-linux-gnu/` at runtime. For a distributable bridge, either install qtmcp or switch to static (`-DBUILD_SHARED_LIBS=OFF`, verify).
|
||
- MCP backend plugins (stdio, sse) are built as Qt plugins under `${CMAKE_BINARY_DIR}/lib/x86_64-linux-gnu/qt6/plugins/mcpserverbackend/`. The bridge process must find them via the Qt plugin path / `QT_PLUGIN_PATH` at runtime, otherwise `QMcpServer("stdio")` can't load the stdio backend.
|
||
- qtmcp version at time of verification: `6.10.2` (alpha1 prerelease, from `.cmake.conf`). Pin `GIT_TAG` to a commit/tag for reproducibility rather than tracking `main`.
|
||
|
||
### qtmcp requirements
|
||
|
||
- Qt 6.8+ (check your Qt version; GammaRay needs 6.5+, qtmcp needs 6.8+)
|
||
- C++20 (GammaRay is C++17; bridge project compiled as C++20 can link C++17 libs fine)
|
||
- Uses Qt BuildInternals CMake style (`qt_build_repo()`)
|
||
|
||
### CMakeLists.txt skeleton
|
||
|
||
```cmake
|
||
cmake_minimum_required(VERSION 3.16)
|
||
project(GammaRayMcpBridge LANGUAGES CXX)
|
||
set(CMAKE_CXX_STANDARD 20) # qtmcp requires C++20
|
||
set(CMAKE_AUTOMOC ON)
|
||
|
||
# GammaRay installed from Step 1; pass -DCMAKE_PREFIX_PATH=$(pwd)/install-prefix
|
||
find_package(GammaRay REQUIRED)
|
||
find_package(Qt6 COMPONENTS Core Gui Widgets Network REQUIRED)
|
||
|
||
# qtmcp via FetchContent (no separate install)
|
||
include(FetchContent)
|
||
FetchContent_Declare(
|
||
qtmcp
|
||
GIT_REPOSITORY https://github.com/signal-slot/qtmcp.git
|
||
GIT_TAG main # pin to a tag/commit for reproducibility
|
||
GIT_SHALLOW TRUE
|
||
)
|
||
set(QT_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
|
||
set(QT_BUILD_TESTS OFF CACHE BOOL "" FORCE)
|
||
FetchContent_MakeAvailable(qtmcp)
|
||
|
||
add_executable(gammaray-mcp-bridge
|
||
src/main.cpp
|
||
src/gammaray_session.cpp # wraps ClientConnectionManager + ObjectBroker
|
||
src/gammaray_session.h
|
||
src/scenegraph_tools.cpp # Q_INVOKABLE methods exposed as MCP tools
|
||
src/scenegraph_tools.h
|
||
)
|
||
|
||
target_link_libraries(gammaray-mcp-bridge PRIVATE
|
||
gammaray_client # VERIFIED: no GammaRay:: namespace
|
||
gammaray_common
|
||
Qt6::Core
|
||
Qt6::Gui
|
||
Qt6::Widgets # REQUIRED: see QApplication note below
|
||
Qt6::Network
|
||
Qt6::McpServer # qtmcp, provides Qt::McpServer alias
|
||
Qt6::McpCommon
|
||
)
|
||
|
||
# qtmcp shared libs + stdio plugin live in build tree; make runtime find them:
|
||
set_target_properties(gammaray-mcp-bridge PROPERTIES
|
||
BUILD_RPATH "$<TARGET_FILE_DIR:Qt6::McpServer>;$<TARGET_FILE_DIR:Qt6::McpCommon>"
|
||
)
|
||
```
|
||
|
||
### Build
|
||
|
||
```bash
|
||
cmake -S . -B build -DCMAKE_PREFIX_PATH=$(pwd)/install-prefix -G Ninja
|
||
cmake --build build
|
||
# Run with probe libs + qtmcp runtime libs on the path:
|
||
LD_LIBRARY_PATH=$(pwd)/install-prefix/lib QT_QPA_PLATFORM=offscreen \
|
||
QT_PLUGIN_PATH=$PWD/build/lib/x86_64-linux-gnu/qt6/plugins \
|
||
./build/gammaray-mcp-bridge --connect tcp://127.0.0.1:11732
|
||
```
|
||
|
||
### ❗ MUST use QApplication, not QCoreApplication
|
||
|
||
**VERIFIED the hard way**: `ObjectBroker::model()` on the client side goes through `ClientConnectionManager`'s model factory (`client/clientconnectionmanager.cpp:44`) which constructs a `RemoteModel`. `RemoteModel`'s constructor (`client/remotemodel.cpp:97`) calls `QApplication::style()->sizeFromContents(...)`. With a plain `QCoreApplication` there is no style → **segfault** the moment `ready()` fires and the first model is requested.
|
||
|
||
Fix: use `QApplication` (which pulls in `Qt6::Widgets`). This is consistent with `client/main.cpp:38` (the standalone GammaRay client uses `QApplication`), and `gammaray_client.so` already runtime-links `libQt6Widgets.so.6`. Run the bridge headless with `QT_QPA_PLATFORM=offscreen`.
|
||
|
||
### MCP server skeleton (using qtmcp)
|
||
|
||
```cpp
|
||
#include <QApplication> // NOT QCoreApplication — see note above
|
||
#include <QtMcpServer/QMcpServer>
|
||
#include "scenegraph_tools.h"
|
||
|
||
int main(int argc, char **argv) {
|
||
QApplication app(argc, argv); // RemoteModel needs QApplication::style()
|
||
// ... parse --connect <url> ...
|
||
|
||
QMcpServer server(QStringLiteral("stdio")); // backend name in ctor, no default
|
||
server.setInstructions("QML SceneGraph & Qt Widget introspection via GammaRay");
|
||
|
||
auto *tools = new SceneGraphTools(&server); // holds GammaRay client connection
|
||
server.registerToolSet(tools, {
|
||
{"listScenegraphNodes", "List all QSGNode tree nodes"},
|
||
{"getNodeVertices", "Get vertex data of a SG node"},
|
||
{"getShaderSource", "Get shader source for a material"},
|
||
{"setRenderMode", "Set SceneGraph render visualization mode"},
|
||
// ...
|
||
});
|
||
|
||
QObject::connect(&server, &QMcpServer::finished, &app, &QCoreApplication::quit);
|
||
|
||
// Connect to GammaRay probe, then start MCP server
|
||
tools->connectToProbe(url); // async, uses ClientConnectionManager
|
||
server.start();
|
||
|
||
return app.exec();
|
||
}
|
||
```
|
||
|
||
Tool implementation as Q_INVOKABLE methods (auto-registered via `registerToolSet`):
|
||
|
||
```cpp
|
||
class SceneGraphTools : public QObject {
|
||
Q_OBJECT
|
||
public:
|
||
// NOTE: qtmcp's callTool() only supports void/bool/QString/QStringList/QImage
|
||
// returns (and QFuture<QList<QMcpCallToolResultContent>> for async). A
|
||
// QJsonObject return would hit qFatal() and crash. So structured results
|
||
// are returned as JSON-serialized QString:
|
||
Q_INVOKABLE QString listScenegraphNodes() const; // returns compact JSON
|
||
Q_INVOKABLE QString listQuickWindows() const;
|
||
|
||
// Async tool returns QFuture<QList<QMcpCallToolResultContent>> (the ONLY
|
||
// async return type qtmcp recognizes). NOT QFuture<QString>.
|
||
Q_INVOKABLE QFuture<QList<QMcpCallToolResultContent>> getShaderSource(int row) const;
|
||
|
||
// QImage return supported by qtmcp (for texture/screenshot)
|
||
Q_INVOKABLE QImage grabTexture(const QString &nodeId) const;
|
||
|
||
private:
|
||
GammaRaySession *m_session = nullptr;
|
||
};
|
||
```
|
||
|
||
**VERIFIED** against `src/mcpserver/qmcpserversession.cpp` (`callTool()` switch on `mm.returnMetaType().id()`): only `Void`/`Bool`/`QString`/`QStringList`/`QImage` (sync) + `QFuture<QList<QMcpCallToolResultContent>>` (async) are handled; the `default:` branch calls `qFatal()`. The plan's earlier `QJsonObject` / `QFuture<QString>` sketches were wrong and would have crashed.
|
||
|
||
### qtmcp target names — VERIFIED
|
||
|
||
FetchContent produces targets `Qt6::McpServer`, `Qt6::McpCommon`, `Qt6::McpClient` (with `Qt::` alias). Headers generated at `${CMAKE_BINARY_DIR}/include/QtMcpServer/` etc. The include `#include <QtMcpServer/QMcpServer>` works once `Qt6::McpServer` is linked. Note `QMcpServer` ctor takes the backend name explicitly: `QMcpServer server(QStringLiteral("stdio"));` (no default ctor).
|
||
|
||
## Step 3: Client-side connection flow
|
||
|
||
Reference: `client/clientconnectionmanager.h:43`, `client/client.h:27`, `common/objectbroker.h:28`.
|
||
|
||
```cpp
|
||
#include <QApplication> // QApplication, not QCoreApplication
|
||
#include <client/clientconnectionmanager.h>
|
||
#include <common/objectbroker.h>
|
||
|
||
// One-time init (registers stream operators, factory callbacks)
|
||
ClientConnectionManager::init();
|
||
|
||
QApplication app(argc, argv); // needed for RemoteModel's QApplication::style()
|
||
auto *conMan = new ClientConnectionManager(&app, false /*no splash*/);
|
||
QObject::connect(conMan, &ClientConnectionManager::ready, [&]() {
|
||
// Connection established, object map received, tool model populated.
|
||
// NOW safe to call ObjectBroker::model(...) etc.
|
||
auto *sgModel = ObjectBroker::model(
|
||
QStringLiteral("com.kdab.GammaRay.QuickSceneGraphModel"));
|
||
// ...
|
||
});
|
||
QObject::connect(conMan, &ClientConnectionManager::disconnected,
|
||
&app, &QCoreApplication::quit);
|
||
|
||
conMan->connectToHost(QUrl("tcp://127.0.0.1:11732"));
|
||
```
|
||
|
||
`ClientConnectionManager::ready()` is emitted after protocol handshake + object map sync — models are only available after this. Do NOT access models before `ready()`. **VERIFIED**: against a running 3.4.0 probe (qml target), `ready()` fires and `ObjectBroker::model("com.kdab.GammaRay.QuickSceneGraphModel")` returns a non-null `RemoteModel*` — full handshake works with matching protocol version 38.
|
||
|
||
## Step 4: Available data and model registration names
|
||
|
||
All names verified from `plugins/quickinspector/quickinspector.cpp` and extension `.cpp` files. These are the strings to pass to `ObjectBroker::model(QString)`.
|
||
|
||
### Models (tree/tabular data via RemoteModel)
|
||
|
||
| Model name | Source | Description |
|
||
|---|---|---|
|
||
| `com.kdab.GammaRay.QuickWindowModel` | quickinspector.cpp:347 | List of QQuickWindows |
|
||
| `com.kdab.GammaRay.QuickItemModel` | quickinspector.cpp:352 | QQuickItem tree |
|
||
| `com.kdab.GammaRay.QuickSceneGraphModel` | quickinspector.cpp:371 | **QSGNode tree** (primary target) |
|
||
| `com.kdab.GammaRay.QuickSceneGraph.sgGeometryVertexModel` | sggeometryextension.cpp:30 | Vertex data of selected SG node |
|
||
| `com.kdab.GammaRay.QuickSceneGraph.sgGeometryAdjacencyModel` | sggeometryextension.cpp:31 | Adjacency/draw mode of selected SG node |
|
||
| `com.kdab.GammaRay.QuickSceneGraph.shaderModel` | materialextension.cpp:39 | Shader list for selected material |
|
||
| `com.kdab.GammaRay.QuickSceneGraph.materialPropertyModel` | materialextension.cpp:38 | Material properties |
|
||
| `com.kdab.GammaRay.QuickItem.<...>` | property controller extensions | Item properties (suffix varies) |
|
||
|
||
Model name pattern for property-controller models: `<objectBaseName>.<nameSuffix>` where objectBaseName is `com.kdab.GammaRay.QuickSceneGraph` or `com.kdab.GammaRay.QuickItem` (quickinspector.cpp:327-329). The suffix is registered via `PropertyController::registerModel(model, nameSuffix)` (core/propertycontroller.cpp:62-64).
|
||
|
||
### Remote objects (interfaces via ObjectBroker::object<T>())
|
||
|
||
| Interface | Object name pattern | Key methods |
|
||
|---|---|---|
|
||
| `QuickInspectorInterface` | `com.kdab.GammaRay.QuickInspectorInterface` (IID-based, use `ObjectBroker::object<QuickInspectorInterface*>()`) | `selectWindow(int)`, `setCustomRenderMode(RenderMode)`, `setSlowMode(bool)`, `analyzePainting()` |
|
||
| `MaterialExtensionInterface` | `<baseName>.material` (e.g. `com.kdab.GammaRay.QuickSceneGraph.material`) | `getShader(int row)` → emits `gotShader(QString)` |
|
||
|
||
### Client-side factory registration
|
||
|
||
The GUI client registers factories via `ObjectBroker::registerClientObjectFactoryCallback<T>()` (quickinspectorwidget.cpp:82, 268). The bridge must do the same for interfaces it wants to use:
|
||
|
||
```cpp
|
||
ObjectBroker::registerClientObjectFactoryCallback<QuickInspectorInterface*>(
|
||
[](const QString &, QObject *parent) -> QObject* {
|
||
return new QuickInspectorClient(parent); // or own impl
|
||
});
|
||
```
|
||
|
||
Check whether `QuickInspectorClient` (quickinspectorclient.h) is exported/installed. If not, may need a thin reimplementation calling `Endpoint::invokeObject()` directly. Verify after install by checking installed headers in `$INSTALL_PREFIX/include/gammaray/`.
|
||
|
||
**RESOLVED (open question #1)**: `QuickInspectorClient`, `QuickInspectorInterface`, `MaterialExtensionInterface` and all other `plugins/quickinspector/*` headers are **NOT** in `install-prefix/include/gammaray/`. Only `client/` (3 headers: `clientconnectionmanager.h`, `processtracker.h`, `gammaray_client_export.h`), `common/`, `core/`, `launcher/`, `ui/` are installed. The plugin client-side classes live only in the GammaRay source tree (`plugins/quickinspector/`) and are compiled into the per-tool `gammaray_quickinspector` client plugin `.so` that the *GUI* client loads dynamically. The bridge does NOT load these plugins. **Good news (verified)**: `Endpoint::invokeObject(name, method, args)` is PUBLIC in the installed `endpoint.h`, and it only needs the name→address mapping (synced at handshake) — NOT a local stub object. So the bridge can call probe-side interface methods directly by IID + bare slot name, with zero stub code. See the "Key implementation findings" in the Verification status section for the exact name (`com.kdab.GammaRay.QuickInspectorInterface/1.0`) and method (`"selectWindow"`) conventions. Only for *receiving* signals (e.g. `MaterialExtensionInterface::gotShader`) would a real client stub be needed — handled case-by-case when implementing those tools.
|
||
|
||
## Step 5: SceneGraph model roles and structure
|
||
|
||
`QuickSceneGraphModel` (plugins/quickinspector/quickscenegraphmodel.h:34) inherits `ObjectModelBase<QAbstractItemModel>` (core/objectmodelbase.h:40).
|
||
|
||
### Columns
|
||
- Column 0: node address string (`Util::addressToString(node)`)
|
||
- Column 1: node type string ("Node", "Geometry Node", "Transform Node", "Clip Node", "Opacity Node", "Root Node", "Render Node")
|
||
|
||
### Roles (from ObjectModelBase + QuickSceneGraphModel)
|
||
- `Qt::DisplayRole` — address / type (see columns above)
|
||
- `ObjectModel::ObjectRole` — `QVariant::fromValue(node)` (QSGNode* as void*)
|
||
- `ObjectModel::ObjectIdRole` — `ObjectId` wrapper
|
||
- `ObjectModel::DecorationIdRole` — icon id
|
||
- `ObjectModel::CreationLocationRole` / `DeclarationLocationRole` — source locations
|
||
|
||
`ObjectModel` roles are defined in `common/objectmodel.h` (read it for exact enum values).
|
||
|
||
### Tree structure
|
||
- Root: the topmost QSGNode (walks up from contentItem's itemNode to find true root, quickscenegraphmodel.cpp:79-90)
|
||
- Children: `QSGNode::firstChild()` / `nextSibling()`
|
||
- Updates: refreshed on `QQuickWindow::afterRendering` signal
|
||
- Node deletion: `QuickSceneGraphModel::nodeDeleted(QSGNode*)` signal
|
||
|
||
**Caveat**: QSGNode* values in the model are raw pointers in the probe's process. They cannot be dereferenced from the client. Use them only as opaque IDs (via ObjectId) for selection and model navigation. Actual geometry/material/texture data must be fetched through the corresponding models/extensions.
|
||
|
||
## Step 6: Suggested MCP tools
|
||
|
||
Map GammaRay client capabilities to MCP tool calls.
|
||
|
||
### Connection management tools (NEW — implemented)
|
||
- `connectProbe(host?, port?)` → `GammaRaySession::connectToHost(tcp://host:port)`, block up to 6s for handshake, return `{connected, state, url, error?}`. Defaults `127.0.0.1:11732`. URL remembered for auto-reconnect.
|
||
- `connectProbeDefault()` → convenience wrapper for `connectProbe(127.0.0.1, 11732)` (qtmcp doesn't support arg-overloading, so two tool entries).
|
||
- `probeStatus()` → return `{state, ready, url, error?}`. No side effects.
|
||
- `disconnectProbe()` → drop connection, clear remembered URL (stops auto-reconnect).
|
||
|
||
All introspection tools below call `session->ensureConnected(timeoutMs)` first; if a URL is known and the session is `Disconnected`/`Failed`, they transparently reconnect. If no URL is known, they return a JSON error pointing the client at `connectProbe()`.
|
||
|
||
### Navigation tools
|
||
- `listQuickWindows()` → read `com.kdab.GammaRay.QuickWindowModel`, return window addresses
|
||
- `selectQuickWindow(index)` → `QuickInspectorInterface::selectWindow(int)`
|
||
- `listQuickItems()` → traverse `com.kdab.GammaRay.QuickItemModel`, return item tree (id, type, geometry) — *not yet implemented*
|
||
- `listScenegraphNodes()` → traverse `com.kdab.GammaRay.QuickSceneGraphModel`, return node tree (id, type, address). Uses `primeAndWait()` for async fill.
|
||
- `selectScenegraphNode(address)` → select node via selection model. Returns `{address, type, hasGeometry, hasMaterial, vertexCount, shaderCount, propertyCount}`
|
||
|
||
### Geometry tools
|
||
- `getNodeVertices(address)` → select SG node, read `sgGeometryVertexModel` (vertices: x, y, z, w + render role)
|
||
- `getNodeAdjacency(address)` → read `sgGeometryAdjacencyModel` (drawing mode, vertex indices)
|
||
|
||
### Material/Shader tools
|
||
- `getMaterialShaders(address)` → read `shaderModel` for selected node
|
||
- `getShaderSource(row)` → `MaterialExtensionInterface::getShader(int)` (async: listen for `gotShader` signal)
|
||
- `getMaterialProperties(address)` → read `materialPropertyModel`
|
||
|
||
### Rendering visualization tools
|
||
- `setRenderMode(mode)` → `QuickInspectorInterface::setCustomRenderMode(mode)` where mode ∈ {NormalRendering, VisualizeClipping, VisualizeOverdraw, VisualizeBatches, VisualizeChanges, VisualizeTraces} (quickinspectorinterface.h:52-66). **NOTE**: takes enum NAME string, not int. Values map 1:1 to GammaRay's `RenderMode` enum.
|
||
- `setSlowMode(enabled)` → `QuickInspectorInterface::setSlowMode(bool)`
|
||
|
||
### Texture tools
|
||
- `grabTexture(objectId)` → `TextureExtension` (async image grab; image arrives via `RemoteViewInterface` as `TransferImage`) — *not yet implemented*
|
||
|
||
### Implementation status (updated — see "Known issues / Blocked" section for details on the sub-model population blocker)
|
||
|
||
**Working end-to-end (verified via full test suite — 26/26 tests passing, 2026-07-07):**
|
||
- ✅ `connectProbe(host?, port?)` / `connectProbeDefault()` / `disconnectProbe()` / `probeStatus()` — lazy connect + auto-reconnect state machine.
|
||
- ✅ `listQuickWindows()` / `selectQuickWindow(index)` — window navigation.
|
||
- ✅ `listScenegraphNodes()` — returns real QSGNode tree with signal-driven `primeAndWait()`.
|
||
- ✅ `listQuickItems()` — returns QML item tree with types AND `ItemFlags` (visibility, focus, zero-size, out-of-view).
|
||
- ✅ `selectScenegraphNode(address)` — selects a node via client-side `SelectionModelClient::select()`, which sends `SelectionModelSelect` to the probe. `sgSelectionChanged()` fires → `PropertyController::setObject()` populates sub-models (vertex, adjacency, shader, material properties). Verified: `selHasSelection=true` with correct node path.
|
||
- ✅ `setRenderMode(mode)` / `setSlowMode(enabled)` — rendering visualization controls.
|
||
- ✅ `getNodeVertices(address)` — reads `sgGeometryVertexModel`. Returns "no vertex data" for nodes without geometry (software renderer).
|
||
- ✅ `getNodeAdjacency(address)` — reads `sgGeometryAdjacencyModel`. Same behavior.
|
||
- ✅ `getMaterialShaders(address)` — reads `shaderModel`. Returns "no shaders" for nodes without materials.
|
||
- ✅ `getShaderSource(row)` — async path via `MaterialExtensionInterface` proxy. Works end-to-end when shaderModel has data.
|
||
- ✅ `getMaterialProperties(address)` — reads `materialPropertyModel`. Returns "no material properties" for nodes without materials.
|
||
- ✅ `selectQuickItem(address)` — selects a QML item via `SelectionModelClient` for `QuickItemModel.selection`. Returns item property data.
|
||
- ✅ `getItemProperties(address)` — reads `AggregatedPropertyModel` for a selected QML item. Returns 4-column model (name/value/type/class) with nested property groups.
|
||
- ✅ **Connection tests** (5/5): tools listed, disconnected status, connect+disconnect, reconnect, reconnect after disconnect.
|
||
- ✅ **Navigation tests** (7/7): list windows, select window, list items, list scenegraph nodes, scenegraph has geometry nodes.
|
||
- ✅ **Item property tests** (6/6): tools listed, select item, get properties, has position, has visual properties, has property groups.
|
||
- ✅ **SceneGraph tests** (8/8): select node, get vertices, get adjacency, get shaders, get material properties, set render mode, set slow mode.
|
||
|
||
**Not yet implemented:**
|
||
- ❌ `grabTexture(objectId)` — `TextureExtension` async image grab. Not started.
|
||
|
||
### Known issues / Blocked
|
||
|
||
**All known blockers resolved as of 2026-07-07.**
|
||
|
||
**RESOLVED — Item property model lazy-fetch causes `Loading...` data (2026-07-07).**
|
||
|
||
`RemoteModel::data()` returns `"Loading..."` on the first call and queues an async fetch. The data only arrives after the event loop processes the server reply. The fix adds a polling loop in `getItemProperties()` that repeatedly calls `data()` and runs the event loop until non-`"Loading..."` data arrives (5s timeout). Similarly, grouped properties (background, contentItem, etc.) require explicit `rowCount()` triggering + event loop processing + `hasChildren()` check because `RemoteModel::rowCount(parent)` returns 0 until the child count response arrives asynchronously.
|
||
|
||
The `primeAndWait()` approach was abandoned for the property model because it triggers a GammaRay `RemoteModel` use-after-free crash (`typeinfo name for QSharedMemory` RTTI corruption). The polling approach avoids this by using only single-cell `data()` calls and short event loop iterations instead of deep recursive tree traversal.
|
||
|
||
The fix was NOT a code change but a sequencing fix in `ensureNodeSelected()`:
|
||
|
||
**Root cause**: the `SelectionModelClient` has a 125ms one-shot timer (`requestSelection`) that sends a `SelectionModelStateRequest` to the server. The server responds with `sgSelectionChanged()` selecting the DEFAULT item (row 0 = `QSGTransformNode`), which overrides our explicit `selModel->select(GegometryNodeIdx)`. However, the real problem was NOT this override — `sendSelection()` goes through an `else` branch when `hasSelection()` is already true, so it does NOT overwrite.
|
||
|
||
**The actual fix** (discovered by reading `networkselectionmodel.cpp`):
|
||
1. Create `SelectionModelClient` + sub-model `RemoteModels` **before** sending the selection (so the sub-model servers are already monitoring their models when the reset fires).
|
||
2. Wait 300ms for the default-selection timer to settle.
|
||
3. Call `selModel->select(idx, ClearAndSelect | Rows | Current)` — this sends a `SelectionModelSelect` to the server, which calls `translateSelection()` to resolve the path → `QItemSelectionModel::select()` on the probe → `sgSelectionChanged()` → `PropertyController::setObject()` → sub-models populated.
|
||
4. Wait 2s for sub-model data arrival, then `primeAndWait()` on each.
|
||
5. Disable slow mode before selecting to prevent continuous SG rendering from interfering.
|
||
|
||
**Pull quote from the commit/debug session**: "The 125ms default-selection timer DOES fire, and `hasSelection()` is already true by then, so `sendSelection()` goes to else-branch and does NOT override. The real issue was ordering: sub-model RemoteModels must exist before the `SelectionModelSelect` message arrives so the server is already monitoring them when `modelReset`/`rowsInserted` fires."
|
||
|
||
**GammaRay source modifications reverted**:
|
||
- `selectSGNodeByAddress`/`selectQuickItemByAddress` Q_INVOKABLE methods removed from `quickinspector.h/.cpp` (never needed — client-side `selModel->select()` is sufficient).
|
||
- Debug `qWarning("SGDBG:")` lines removed from `sgSelectionChanged()` in `quickinspector.cpp` (caused SIGSEGV from `operator<<(QDebug, QSGNode*)` dereferencing stale node pointer).
|
||
|
||
**Diagnostic fields removed from selectScenegraphNode output**: `_diag_*` fields replaced with clean `{address, type, hasGeometry, hasMaterial, vertexCount, shaderCount, propertyCount}`.
|
||
|
||
### Half-finished artifacts
|
||
|
||
All items from the original blocker resolved. Remaining minor issues:
|
||
- `listScenegraphNodes` first-call fill stability — first call may show "Loading..." for deep nodes (RemoteModel lazy-fetch behavior); second call returns the full tree. The `primeAndWait()` loop handles this transparently, but single-call would be nicer.
|
||
- `walkItemChildren` `flagsToJson` uses hardcoded flag bit values (1,2,4,8,16,32,64) — fragile if GammaRay's `ItemFlags` enum changes. Should read from `common/quickitemmodelroles.h` if it were installed.
|
||
- Full test suite ~3.5 min (session-scoped probe + module-scoped bridge + 0.5s settle delays). Connection tests still get fresh bridges per test (function-scoped).
|
||
|
||
## Step 7: Launch and run
|
||
|
||
### Connection model (decided)
|
||
|
||
**Phase 1 (now): lazy connect + explicit `connectProbe` tool + auto-reconnect.** The bridge does NOT hard-require a probe URL at startup. A `--connect <url>` (or `GAMMARAY_PROBE_URL` env) is treated as a *default* URL — the bridge tries it best-effort on startup but won't fail if the probe isn't up yet. The MCP client drives connection lifecycle via three tools:
|
||
|
||
- `connectProbe(host?, port?)` — establish a connection to a probe (defaults `127.0.0.1:11732`). Blocks up to ~6s for the handshake. Returns JSON `{connected, state, url, error?}`. The URL is remembered for subsequent auto-reconnect.
|
||
- `probeStatus()` — report `{state, ready, url, error?}` without side effects.
|
||
- `disconnectProbe()` — drop the current connection and forget the URL (auto-reconnect stops until `connectProbe` is called again).
|
||
|
||
Every introspection tool (`listQuickWindows`, `selectQuickWindow`, `listScenegraphNodes`, …) calls `session->ensureConnected(timeoutMs)` first: if a URL is known and the session is `Disconnected`/`Failed`, it transparently re-issues `connectToHost(url)` and waits for `ready()`. If no URL is known, the tool returns a JSON error pointing the user at `connectProbe()`. So in practice a client only needs to call `connectProbe` once after starting the probe; subsequent tool calls keep working across probe restarts.
|
||
|
||
Rationale: opencode (and other MCP hosts) launch the bridge subprocess at host startup, typically BEFORE the developer has started the target app + probe. A hard `--connect` either succeeds by luck or leaves the bridge in a permanently-broken state (the original scaffold's failure mode). The lazy-connect model lets the bridge come up clean and wait for the client to point it at a probe — and recover automatically when the probe is restarted mid-session (common during iterative QML debugging).
|
||
|
||
**Why not auto-discovery (mDNS)?** GammaRay's probe does broadcast via mDNS, which would let the bridge find probes with zero configuration. But (a) multiple probes on a dev machine are common, and silently picking one is wrong; (b) mDNS on Linux is flaky across sandboxes/containers; (c) the explicit `connectProbe` is one cheap MCP call and gives the client full control. mDNS can be added later as an optional `discoverProbes()` tool if needed.
|
||
|
||
**Auto-reconnect details**: `ClientConnectionManager` already retries `transientConnectionError` (host not yet up) every 1s for 60s before emitting `persistentConnectionError` — so the INITIAL handshake has built-in resilience. But a `disconnected` signal (probe killed / target crashed MID-session) is NOT auto-retried by the manager. The bridge adds its own 1s-backoff reconnect on `disconnected` (guarded against re-entrancy and disabled after explicit `disconnectProbe()`). On `persistentConnectionError` the bridge does NOT auto-retry (GammaRay already did for 60s); the user calls `connectProbe()` again once the probe is back. State machine: `Disconnected → Connecting → Ready`, with `→ Failed` on persistent error (recoverable only via `connectProbe()`).
|
||
|
||
**Phase 2 (future, not in scope now): hybrid mode.** Bridge could also expose `attach(pid)` / `launch(path)` MCP tools for LLM-driven dynamic switching using `launcher/core/launcher.h` and `launcher/core/clientlauncher.h`. Out of scope for now.
|
||
|
||
### Phase 1 startup sequence
|
||
|
||
```bash
|
||
# 1. Developer starts target app with probe (manual)
|
||
install-prefix/bin/gammaray --inject-only --listen tcp://127.0.0.1:11732 \
|
||
/usr/lib/qt6/bin/qml /path/to/app.qml -platform offscreen
|
||
|
||
# 2. MCP host launches bridge (stdio). --connect is OPTIONAL: if given, it's
|
||
# used as a default URL and tried best-effort at startup; if omitted, the
|
||
# client must call connectProbe() before any introspection tool returns data.
|
||
# Either way, tools auto-reconnect across probe restarts.
|
||
./run.sh # no --connect required
|
||
# or: ./run.sh --connect tcp://127.0.0.1:11732
|
||
|
||
# 3. (MCP client) call connectProbe(host, port) once the probe is up — or just
|
||
# call listQuickWindows/etc. and the bridge will return a JSON error hinting
|
||
# at connectProbe if no URL is configured.
|
||
```
|
||
|
||
For headless/CI launching of the probe, `systemd-run --user --no-block` (with `StandardOutput=file:...` + `Environment=...` properties) is a reliable way to fully detach the probe so the launching shell returns immediately — plain `&`/`disown` backgrounding tends to hang the parent shell's stdout pipe.
|
||
|
||
Bridge accepts an optional default probe URL via `--connect <url>` (or `GAMMARAY_PROBE_URL` env). At runtime the connection is driven by `connectProbe`/`disconnectProbe` MCP tools; the URL is remembered so tools can auto-reconnect after a probe restart. The bridge's `GammaRaySession` state machine (`Disconnected → Connecting → Ready`, with `→ Failed` on persistent error) is the single source of truth; `probeStatus()` reports it to the client.
|
||
|
||
The bridge acts as an MCP server on stdio. The LLM host (e.g. ZCode, opencode) launches the bridge as a subprocess and communicates via JSON-RPC over stdio.
|
||
|
||
## Key gotchas to remember
|
||
|
||
1. **Protocol version lock**: probe and client must be same GammaRay build. System package too old → build from 3.4.0 source. (`common/protocol.h:141`)
|
||
2. **GPL-2.0-or-later**: bridge linking gammaray_client must be GPL-compatible. Accepted. qtmcp offers LGPL-3.0-only / GPL-2.0-only / GPL-3.0-only — we comply under GPL-2.0-only.
|
||
3. **Qt private headers**: GammaRay build needs `qt6-base-private-dev`, `qt6-declarative-private-dev`. The bridge itself only needs public Qt + installed GammaRay headers. **qtmcp requires Qt 6.8+** (newer than GammaRay's 6.5+ floor); ensure build environment Qt >= 6.8.
|
||
4. **C++20**: qtmcp requires C++20. Bridge project set to C++20; GammaRay C++17 libs link fine.
|
||
5. **Models only available after `ClientConnectionManager::ready()`** — do not call `ObjectBroker::model()` before handshake completes. MCP tool calls arriving before probe connection should return an error or wait.
|
||
6. **QSGNode pointers are opaque IDs on client side** — cannot dereference. Use ObjectId for selection/navigation; fetch real data via property/extension models.
|
||
7. **Async interfaces**: `MaterialExtensionInterface::getShader()` returns via signal `gotShader(QString)`, not synchronously. Texture grabs return via `RemoteViewInterface`. Bridge must handle async and correlate to MCP requests. qtmcp supports `QFuture<Result>` return types for async tool handlers.
|
||
8. **Target namespace**: exported CMake targets may lack `GammaRay::` prefix (NAMESPACE commented out at CMakeLists.txt:931). Verify in installed `GammaRayTarget.cmake`. Same applies to qtmcp installed targets — verify after building qtmcp.
|
||
9. **OpenGL dependency**: SG geometry/material/texture extensions only build when `QT_NO_OPENGL` is not set (quickinspector/CMakeLists.txt:47-66). If headless/no-GL build, these models won't exist. Ensure GammaRay is built WITH OpenGL (default).
|
||
10. **In-process mode exists** (`gammaray --inprocess`): probe + client in same process via `gammaray_inprocessui` module. Not useful for bridge (bridge IS the client). Use TCP/local socket mode.
|
||
11. **Probe ABI**: probes are versioned by Qt ABI. `GAMMARAY_MULTI_BUILD=ON` (default) builds multiple. The launcher picks the right one. Bridge doesn't need to worry about probe ABI, only client ABI matching the installed `gammaray_client`.
|
||
12. **QApplication required (NEW, verified)**: the client-side `RemoteModel` ctor calls `QApplication::style()->sizeFromContents()`. A `QCoreApplication`-only bridge segfaults on the first `ObjectBroker::model()` call after `ready()`. Use `QApplication` + `QT_QPA_PLATFORM=offscreen` for the headless bridge.
|
||
13. **qtmcp shared libs + plugin path (NEW, verified)**: under FetchContent, `libQt6Mcp*.so` and the `mcpserverbackend/libqmcpserverstdio.so` plugin land in the consumer's build tree. Set `LD_LIBRARY_PATH` and `QT_PLUGIN_PATH` (or RPATH) so the bridge finds them at runtime, else `QMcpServer("stdio")` can't load the stdio backend.
|
||
|
||
## Files worth re-reading in GammaRay source
|
||
|
||
When in doubt, these files have the ground truth:
|
||
|
||
- `plugins/quickinspector/quickinspector.cpp:322-402` — model + object registration names
|
||
- `plugins/quickinspector/quickscenegraphmodel.{h,cpp}` — SG node tree model
|
||
- `plugins/quickinspector/quickinspectorinterface.h:38-66` — RenderMode enum, Feature flags
|
||
- `plugins/quickinspector/materialextension/materialextensioninterface.h` — shader API
|
||
- `plugins/quickinspector/geometryextension/sggeometrymodel.h` — vertex/adjacency models
|
||
- `common/objectbroker.h` — model/object retrieval API
|
||
- `common/objectmodel.h` — ObjectRole enum values
|
||
- `client/clientconnectionmanager.h:43` — connection lifecycle
|
||
- `client/client.h:27` — low-level client endpoint
|
||
- `client/remotemodel.h` — client-side model wrapper
|
||
- `core/objectmodelbase.h:40` — base model roles/columns
|
||
- `GammaRayConfig.cmake.in` — installed CMake package contents
|
||
|
||
## Open questions to resolve during implementation
|
||
|
||
1. ~~Are `QuickInspectorClient` and `MaterialExtensionClient` classes exported in installed headers?~~ **RESOLVED**: No. See Step 4 note above — bridge needs own thin client stubs calling `Endpoint::invokeObject()`.
|
||
2. Exact `ObjectModel` role enum values — read `common/objectmodel.h` when implementing. (Installed at `install-prefix/include/gammaray/common/objectmodel.h`.)
|
||
3. How does `RemoteModel` (client/remotemodel.h) expose data? It's a QAbstractItemModel proxying the server model. Standard model API applies. **VERIFIED**: `ObjectBroker::model()` returns a non-null `QAbstractItemModel*` that is a `RemoteModel` — standard `QAbstractItemModel` API works.
|
||
4. Texture image transfer: `TransferImage` (common/transferimage.h) over the wire, arrives via `RemoteViewInterface`. Need to trace the exact signal path for texture grabs. Still open.
|
||
|
||
## Verification status (as of this revision)
|
||
|
||
- ✅ **Step 1 done**: GammaRay 3.4.0 built + installed to `install-prefix/`. Protocol version 38.
|
||
- ✅ **Minimal client built & run**: `minimal-client/` links `gammaray_client` + `gammaray_common`, connects to a live probe, receives `ready()`, fetches `QuickSceneGraphModel` / `QuickWindowModel` via `ObjectBroker` — clean exit. Proves the client ABI + protocol path is viable for the bridge.
|
||
- ✅ **qtmcp FetchContent viable**: `Qt6::McpServer`/`Qt6::McpCommon` build & link from a `FetchContent_MakeAvailable()` call. No separate clone/install.
|
||
- ✅ **Bridge scaffold built & working end-to-end** (`bridge/`): `gammaray-mcp-bridge` links GammaRay client + qtmcp, runs as a stdio MCP server. Smoke test against a live probe confirms:
|
||
- `initialize` → serverInfo + capabilities + protocolVersion `2024-11-05` ✅
|
||
- `tools/list` → 15 tools registered: `connectProbe`, `connectProbeDefault`, `disconnectProbe`, `getMaterialProperties`, `getMaterialShaders`, `getNodeAdjacency`, `getNodeVertices`, `getShaderSource`, `listQuickWindows`, `listScenegraphNodes`, `probeStatus`, `selectQuickWindow`, `selectScenegraphNode`, `setRenderMode`, `setSlowMode` ✅
|
||
- `probeStatus` (before connect) → `{"ready":false,"state":"disconnected","url":""}` ✅
|
||
- `connectProbeDefault()` → `{"connected":true,"state":"ready","url":"tcp://127.0.0.1:11732"}` ✅
|
||
- `probeStatus` (after connect) → `{"ready":true,"state":"ready","url":"tcp://127.0.0.1:11732"}` ✅
|
||
- `listQuickWindows` → `[{"address":"0x... (ApplicationWindow_QMLTYPE_0)","type":""}]` ✅
|
||
- `selectQuickWindow(0)` → `{"index":0,"status":"selected"}` ✅
|
||
- `setSlowMode(true)` → `{"slowMode":true}` ✅
|
||
- `setRenderMode("VisualizeOverdraw")` → `{"renderMode":"VisualizeOverdraw"}` ✅ (takes enum name string, NOT int)
|
||
- `setRenderMode("NormalRendering")` → `{"renderMode":"NormalRendering"}` ✅
|
||
- `disconnectProbe()` → `{"ready":false,"state":"disconnected","url":""}` ✅
|
||
- `listScenegraphNodes` (round 2, after 1s sleep) → full QSGNode tree with 11 GeometryNodes ✅
|
||
- ✅ **Connection model revised**: bridge no longer hard-requires `--connect` at startup. Lazy connect + `connectProbe` MCP tool + auto-reconnect on dropped session. See "Connection model (decided)" in Step 7. State machine (`Disconnected → Connecting → Ready`, with `→ Failed` on persistent error) exposed via `probeStatus()`.
|
||
|
||
### Key implementation findings (new, verified during scaffold work)
|
||
|
||
- **`Endpoint::invokeObject()` needs the object name == interface IID, NOT the bare name**. `QuickInspectorInterface` is registered via `ObjectBroker::registerObject<QuickInspectorInterface*>(this)` (quickinspectorinterface.cpp:59), which derives the name from `qobject_interface_iid` = `"com.kdab.GammaRay.QuickInspectorInterface/1.0"`. The bare `"com.kdab.GammaRay.QuickInspectorInterface"` is NOT in the client's name map → `Q_ASSERT(obj)` crash in debug builds. Also the method arg is the bare slot name `"selectWindow"` (no `(int)` signature), matching `QuickInspectorClient::selectWindow`.
|
||
- **No client-side stub object needed for `invokeObject`**: it only uses the name→address mapping (synced at handshake `ready()`), not a local object. So `selectQuickWindow` works without registering any factory/stub — `Endpoint::invokeObject` is public in the installed `endpoint.h`. (This resolves open question #1's "approach (b)" more cleanly than expected.)
|
||
- **RemoteModel is lazily fetched** (client/remotemodel.cpp): `data()`, `rowCount()`, `hasChildren()` return placeholders ("Loading..." / 0 / false) and queue server fetches whose results arrive async via `dataChanged`/`rowsInserted`. So a tool that reads a model right after `ready()` gets stale/empty data. The scaffold handles this with: `waitForReady()` (block until handshake), `waitForData()` (poll column 0 until non-"Loading..."), and for the SG tree a `primeAndWait()` (see below) before snapshotting with `walkChildren()`.
|
||
- **`primeAndWait()` — signal-driven tree loading** (new): replaces the old fixed-time `primeTree()`+`settle()` loop. Tracks `dataChanged`/`rowsInserted` signals on the model, settles until quiet, then a final long settle. Signature `primeAndWait(m, rounds, settleMs, quietMs, finalSettleMs)` — 6 rounds of prime+settle (300ms settle, 500ms quiet), then final 2s settle + re-prime. Fixes "Loading..." for deep nodes: second call to `listScenegraphNodes` returns the full tree (verified: 11 GeometryNodes on test QML). First call may still show "Loading..." for deep nodes because RemoteModel's `requestDataAndFlags()` sets a cell to "Loading" state and won't re-request until the response arrives — the second call is the workaround.
|
||
- **`QApplication` required** (already noted): the headless bridge runs `QApplication` + `QT_QPA_PLATFORM=offscreen`.
|
||
- **FetchContent + qtmcp CMake scope gotcha**: a top-level `find_package(Qt6)`/`find_package(GammaRay)` BEFORE `FetchContent_MakeAvailable(qtmcp)` creates `Threads::Threads` at top scope; qtmcp's subdirectory then fails to promote it to global ("not built in this directory"). Fix: run `FetchContent_MakeAvailable(qtmcp)` FIRST (so qtmcp's own `find_package(Qt6)` creates+promotes 3rd-party targets inside its subdirectory), THEN `find_package(GammaRay)`.
|
||
- **qtmcp tool return types**: only `void`/`bool`/`QString`/`QStringList`/`QImage` (sync) + `QFuture<QList<QMcpCallToolResultContent>>` (async). `QJsonObject` hits `qFatal`. Structured results → JSON-serialized `QString` (see Step 2 tool section).
|
||
- **Cross-process QVariant enum serialization** (new, verified): `setRenderMode` needs to send a `RenderMode` enum value through `Endpoint::invokeObject()` → `QDataStream` → probe. The bridge defines a minimal `RenderMode` enum in `quickinspector_types.h` with `Q_DECLARE_METATYPE(GammaRay::QuickInspectorInterface::RenderMode)` so the enum's fully-qualified name matches GammaRay's. Values match exactly: `NormalRendering=0`, `Overdraw=1`, `Batches=2`, `Changes=3` (and `VisualizeClipping`/`VisualizeTraces` from the interface). Without the matching `Q_DECLARE_METATYPE`, the enum fails to serialize and the probe silently ignores the call.
|
||
- **`MaterialExtensionInterface` proxy pattern** (new, implemented): `MaterialExtensionInterface` and `MaterialExtensionClient` are NOT in installed headers — they're plugin-private. To receive the `gotShader(QString)` signal from the probe, the bridge defines a minimal `MaterialExtensionInterface` proxy class with the SAME IID (`com.kdab.GammaRay.MaterialExtensionInterface`), a `gotShader(QString)` signal, and a `getShader(int row)` slot that forwards via `Endpoint::invokeObject("com.kdab.GammaRay.QuickSceneGraph.material", "getShader", {row})`. The concrete `MaterialExtensionProxy` self-registers via `ObjectBroker::registerObject(name, this)` and the factory is registered via `registerClientObjectFactoryCallback<MaterialExtensionInterface*>()` in `initOnce()`. The `material_interface.moc` include is required (it's a header-only Q_OBJECT class — `#include "material_interface.moc"` at the bottom of the .cpp).
|
||
- **`ensureNodeSelected()` ordering fix for selection** (new, resolved the main blocker): The key insight was that sub-model `RemoteModels` must be created via `ObjectBroker::model()` BEFORE sending the `SelectionModelSelect` message. This ensures the server-side `RemoteModelServer` instances are already monitoring when `modelReset`/`rowsInserted` fires from `sgSelectionChanged()` → `PropertyController::setObject()`. The 300ms settle before `selModel->select()` lets the 125ms default-selection timer fire without racing. The final 2s settle + `primeAndWait()` ensures all sub-model data arrives before being read. See `scenegraph_tools.cpp:442-515`.
|
||
- **Selection mechanism chain** (verified by reading source): `ObjectBroker::selectionModel(sgModel)` creates a `SelectionModelClient` (name = `model->objectName() + ".selection"`), which syncs to the probe's `SelectionModelServer`. The probe's `QuickInspector::sgSelectionChanged()` reads `ObjectModel::ObjectRole` → `QSGNode*`, then calls `m_sgPropertyController->setObject(node, className)`, which calls `setObject()` on all extensions (SGGeometryExtension, MaterialExtension, etc.) which populate the sub-models. `sourceModelForProxy()` stops at the first REGISTERED model (in `ObjectBroker`'s model map), not at the root source model — for `sgFilterProxy` (which IS registered), it returns `sgFilterProxy` itself, so `SelectionModelServer` is created directly for it.
|
||
|
||
### Running the bridge
|
||
|
||
```bash
|
||
cd bridge
|
||
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=$(pwd)/../install-prefix
|
||
cmake --build build
|
||
# Probe (manual, Phase 1): start a Qt/QML app with the GammaRay probe
|
||
../install-prefix/bin/gammaray --inject-only --listen tcp://127.0.0.1:11732 \
|
||
--injector preload /usr/lib/qt6/bin/qml /path/to/app.qml
|
||
# Bridge (stdio MCP server) — --connect is OPTIONAL now:
|
||
./run.sh
|
||
# or: ./run.sh --connect tcp://127.0.0.1:11732
|
||
# Then (MCP client) call connectProbe(127.0.0.1, 11732) once the probe is up.
|
||
```
|
||
`run.sh` sets `LD_LIBRARY_PATH` (GammaRay + qtmcp libs) and `QT_PLUGIN_PATH` (qtmcp `mcpserverbackend/libqmcpserverstdio.so`) and `QT_QPA_PLATFORM=offscreen`.
|
||
|
||
## Qt Widget support (implemented)
|
||
|
||
GammaRay 的 widget inspector 与 quick inspector 共享同一套底层架构（`PropertyController`、`SelectionModelClient`、`RemoteModel`），大部分 QML 工具的模式已直接复用。
|
||
|
||
### 已实现的 widget 工具
|
||
|
||
**导航（与 QML 类似，模型名不同）**
|
||
- ✅ `listWidgets()` — 遍历 `com.kdab.GammaRay.WidgetTree` 模型，返回 widget 层次树
|
||
- ✅ `selectWidget(address)` — 通过 `SelectionModelClient` 选取 widget，触发 `PropertyController` 填充子模型（属性、attribute 等）
|
||
|
||
**属性/attribute（与 QML 共享读取实现）**
|
||
- ✅ `getWidgetProperties(address)` — 读取同一套 `AggregatedPropertyModel`（通过 `PropertyController`），与 `getItemProperties` 共享实现代码
|
||
- ✅ `getWidgetAttributes(address)` — 读取 `widgetAttributeModel`（`AttributeModel<QWidget, Qt::WidgetAttribute>`），列出 Qt::WidgetAttribute 值
|
||
|
||
**视觉/分析（新功能，需 RemoteViewInterface）**
|
||
- ❌ `grabWidget(address)` — 通过 `WidgetRemoteView`（`com.kdab.GammaRay.WidgetRemoteView`）抓取 widget 截图，异步图像传输
|
||
- ❌ `analyzePainting(address)` — 触发 `PaintAnalyzer` 捕获绘制操作
|
||
- ❌ `exportWidgetAsSvg(address)` — 通过 `WidgetInspectorInterface` 导出 SVG
|
||
- ❌ `exportWidgetAsUiFile(address)` — 导出 Qt Designer .ui 文件
|
||
|
||
**控制**
|
||
- ❌ `setInputRedirection(enabled)` — 允许将鼠标/键盘事件转发到目标 widget
|
||
|
||
### 实现策略
|
||
|
||
| 难度 | 工作项 | 说明 |
|
||
|---|---|---|
|
||
| ✅ 已完成 | `listWidgets()` / `selectWidget()` | 直接复用 `listQuickItems()` / `selectQuickItem()` 的代码模式，模型名改为 `com.kdab.GammaRay.WidgetTree` |
|
||
| ✅ 已完成 | `getWidgetProperties()` | 与 `getItemProperties()` 完全一样，都读 `AggregatedPropertyModel`，已提取共享函数 `readAggregatedPropertyModel()` |
|
||
| ✅ 已完成 | `getWidgetAttributes()` | 读取 `widgetAttributeModel`（`PropertyController` 子模型），模式与 `getMaterialProperties()` 类似 |
|
||
| ✅ 已完成 | `WidgetInspectorInterface` 代理 | IID `com.kdab.GammaRay.WidgetInspector`，用于 `export*`/`setInputRedirection` 等方法，类似 `QuickInspectorProxy` |
|
||
| ❌ 未实现 | `grabWidget()` / `analyzePainting()` | 需要 `RemoteViewInterface` 的 `TransferImage` 异步路径，模式与 `MaterialExtensionInterface` 的 `gotShader` 信号类似 |
|
||
|
||
### 与 QML 工具的关键区别
|
||
|
||
- **选取机制不同**：QML 用 `QuickItemModel.selection`，Widget 用 `WidgetTree.selection` → 不同的 `PropertyController` 实例
|
||
- **模型对象名不同**：property model 对象名为 `com.kdab.GammaRay.QuickItem.<address>.properties` vs `com.kdab.GammaRay.Widget.<address>.properties`
|
||
- **Attribute 模型是 widget 独有的**：QML 项没有 `Qt::WidgetAttribute` 的概念
|
||
- **Widget 没有 SceneGraph**：不需要 `getNodeVertices`/`getMaterialShaders` 等 SG 工具
|
||
- **Widget 有截图和绘制分析**：QML 的 texture grab 类似但不同
|
||
|
||
## Next steps
|
||
|
||
### 已完成的阶段
|
||
|
||
- ✅ **QML SceneGraph 工具**：listScenegraphNodes、getNodeVertices、getNodeAdjacency、getMaterialShaders、getShaderSource、getMaterialProperties、setRenderMode、setSlowMode
|
||
- ✅ **QML Item 属性工具**：listQuickItems、selectQuickItem、getItemProperties
|
||
- ✅ **Qt Widget 工具**：listWidgets、selectWidget、getWidgetProperties、getWidgetAttributes
|
||
- ✅ **跨平台保护**：QuickInspector 和 WidgetInspector 的代理注册均已添加条件守卫，避免在未加载对应插件的 target 进程中崩溃
|
||
- ✅ **测试套件**：27 个 QML 测试 + 7 个 widget 测试，共 34 个集成测试
|
||
|
||
### 剩余工作
|
||
|
||
1. **Test `getShaderSource` async path end-to-end** — needs a QML app with real shaders (current test app uses software renderer where GeometryNodes have `nullptr` geometry).
|
||
2. **Test geometry/material tools against a QML app with real QSGGeometry data** — current test app uses software renderer so `hasGeometry: false` and empty sub-models are correct behavior.
|
||
3. **Implement `grabWidget(objectId)`** — `RemoteViewInterface` async image grab via `TransferImage`. Follows the `MaterialExtensionInterface` proxy pattern.
|
||
4. **Implement `grabTexture(objectId)`** — `TextureExtension` async image grab via `RemoteViewInterface` `TransferImage`.
|
||
5. **Implement `analyzePainting()` / `exportWidgetAsSvg()` / `exportWidgetAsUiFile()`** — via `WidgetInspectorInterface` proxy.
|
||
6. **Fix `listScenegraphNodes` first-call fill stability** — single-call instead of caller needing a second call.
|
||
7. **Build `--connect` startup flag integration** with systemd service for auto-attach during development.
|