Agent Tracing

​

Overview

Siclaw can record what every agent does — the prompt, each LLM call, every tool invocation, token usage, the model that answered, and latency — and stream it to one or more third-party analysis platforms using OpenTelemetry. Each user prompt becomes one trace: a root agent.prompt span with the LLM and tool calls nested underneath, tagged with the model name and per-call token counts. The root span also carries session.id and user.id, identifying the conversation and the user who sent the prompt — both travel with the prompt request, so traces group by session and by user in every deployment mode (per-user pod, in-process, or a shared multi-user runtime). This is the same data you’d use to debug a slow investigation, audit what an agent did, or evaluate prompt/model changes over time. Tracing is off by default, admin-managed from the web UI, and global — one configuration is shared by every agent.

​

Supported platforms

You can configure several platforms at once — every agent run fans out to each enabled platform from a single recording path.

​

Configure

Open the web UI as an admin and go to Metrics → Tracing.

1. Click Add platform, pick the type (Langfuse / Phoenix / Generic OTLP).
2. Enter the OTLP endpoint URL — for Langfuse this must include the full path …/api/public/otel/v1/traces, not just the host.
3. Enter the credentials (Siclaw assembles the auth header for you — e.g. it base64-encodes the Langfuse key pair into a Basic header).
4. Make sure the platform’s toggle is on.

Tracing is active whenever at least one platform is enabled — there is no separate master switch. To pause tracing, disable every platform.

​

Global settings

• service.name — the OpenTelemetry service identifier stamped on every trace. Defaults to siclaw-agentbox. Set distinct names (e.g. siclaw-prod, siclaw-staging) when several Siclaw deployments report to the same backend so you can tell them apart.
• Environment (deployment.environment.name) — buckets traces by environment in Langfuse’s environment filter. Read from SICLAW_TRACING_ENVIRONMENT in the agentbox. Portal injects it per agent (derived from the runtime the agent is bound to), so this needs no per-deployment configuration — that is the authoritative source when present. Setting SICLAW_TRACING_ENVIRONMENT on the runtime deployment is a fallback for deployments where Portal does not supply one. The value is normalized on read (lowercased, illegal characters → -, ≤40 chars, reserved langfuse prefix stripped) so an unnormalized runtime name still lands in a valid bucket rather than silently falling back to Langfuse’s default. Unset everywhere ⇒ default.
• Send content — a privacy gate. When off (default) only metadata and the call tree are exported (model, tokens, tool names, latency). When on, the actual LLM prompts/responses and tool arguments are exported too — far more useful for debugging, but it sends conversation content to the platform.

​

Changes take effect live

Adding, editing, toggling, or deleting a platform — and changing the global settings — hot-reloads every running agent without a restart or redeploy. Behind the scenes the change is broadcast to all active AgentBoxes, which rebuild their exporters in place. New agents pick up the current configuration when they start.

​

Test a platform

Each platform row has a Test button. It fires an empty OTLP request to the configured endpoint with the stored credentials and reports the real HTTP status, so you can confirm the URL and auth are correct before relying on it. The probe is restricted to http/https and blocks cloud-metadata and link-local addresses.

​

Notes

• Credentials are stored server-side and masked in the API and UI (only a prefix is shown); editing a platform without re-entering the secret keeps the stored value.
• Tracing covers agents running in Kubernetes (one pod per user), local server mode, and the standalone CLI.
• This is Plane A observability (online agent-behavior traces). It is separate from the Prometheus/Grafana metrics on the other Metrics tabs.