# MCP Servers
Source: https://docs.siclaw.ai/configuration/mcp

Connect external data sources to Siclaw via Model Context Protocol.

[Model Context Protocol (MCP)](https://modelcontextprotocol.io/) lets you extend Siclaw with external tools and data sources. During investigation, the agent discovers and uses MCP tools automatically — querying Prometheus metrics, searching GitHub issues, reading files, or calling any custom API.

## Supported Transports

| Transport           | Use Case                                       | Config             |
| ------------------- | ---------------------------------------------- | ------------------ |
| **stdio**           | Local processes (npx packages, Python scripts) | `command` + `args` |
| **SSE**             | Remote servers (Server-Sent Events)            | `url`              |
| **streamable-http** | Remote servers (bidirectional HTTP)            | `url`              |

## Configuration

### Via Web UI (Recommended)

In Gateway mode, go to **Settings** > **MCP Servers**:

1. Click **New Server**
2. Select transport type
3. Enter a unique name and optional description
4. Fill in transport-specific fields (command/args or URL)
5. Add environment variables or HTTP headers as needed
6. Save

Changes take effect immediately — all active sessions reload automatically.

<Note>
  Creating and managing MCP servers requires **admin** role. All users can use the tools they provide.
</Note>

### Via settings.json (CLI mode)

For TUI / single-user mode, add MCP servers to `.siclaw/config/settings.json`:

```json theme={null}
{
  "mcpServers": {
    "prometheus": {
      "transport": "stdio",
      "command": "npx",
      "args": ["-y", "@prom-mcp/server"],
      "env": {
        "PROMETHEUS_URL": "http://prometheus:9090"
      }
    }
  }
}
```

## Examples

### Prometheus Metrics

```json theme={null}
{
  "mcpServers": {
    "prometheus": {
      "transport": "stdio",
      "command": "npx",
      "args": ["-y", "@prom-mcp/server"],
      "env": {
        "PROMETHEUS_URL": "http://prometheus.monitoring:9090"
      }
    }
  }
}
```

The agent can then query metrics during investigation:

```
Phase 1: Context Gathering
  [mcp:prometheus] rate(http_request_duration_seconds_sum{service="payment"}[5m])
  [kubectl] get pods -n payments
```

### Filesystem Access

```json theme={null}
{
  "mcpServers": {
    "filesystem": {
      "transport": "stdio",
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
    }
  }
}
```

### GitHub

```json theme={null}
{
  "mcpServers": {
    "github": {
      "transport": "stdio",
      "command": "npx",
      "args": ["-y", "@github/mcp-server"],
      "env": {
        "GITHUB_TOKEN": "your-token"
      }
    }
  }
}
```

### Custom HTTP Service

Any service implementing the MCP protocol can be connected:

```json theme={null}
{
  "mcpServers": {
    "my-service": {
      "transport": "streamable-http",
      "url": "https://mcp.internal.example.com/v1",
      "headers": {
        "Authorization": "Bearer your-token"
      }
    }
  }
}
```

## How It Works

### Tool Discovery

When a session starts, Siclaw connects to all enabled MCP servers and discovers their tools. MCP tools appear alongside built-in tools with a `mcp__` prefix:

```
mcp__prometheus__query        ← from Prometheus MCP server
mcp__github__search_issues    ← from GitHub MCP server
mcp__filesystem__read_file    ← from Filesystem MCP server
```

The agent decides which tools to use based on the investigation context — no manual tool selection needed.

### Config Sync (Gateway Mode)

In multi-user deployments, MCP configuration syncs automatically:

```
Admin creates/edits MCP server in Web UI
  → Saved to database
  → Gateway notifies all active AgentBoxes
  → Each AgentBox fetches merged config
  → Active sessions reload with new tools
```

The merge strategy: DB entries (managed via Web UI) override local seed entries with the same name. Disabling a DB entry removes it from the merged config.

### Kubernetes Considerations

<Warning>
  In Kubernetes mode, `stdio` MCP servers run inside AgentBox pods. Make sure the required binaries or packages are available in that runtime image, or use HTTP-based MCP transports instead.
</Warning>

For HTTP-based transports (`sse`, `streamable-http`), the MCP server runs externally — the AgentBox pod only needs network access to the URL.


# LLM Providers
Source: https://docs.siclaw.ai/configuration/providers

Configure Siclaw to use Anthropic, OpenAI, Ollama, or any compatible LLM.

Siclaw needs an LLM to power its investigation engine.

* **TUI mode** reads `.siclaw/config/settings.json`
* **Local Server / Gateway** is typically configured through the **Models** page in the Web UI

## Anthropic (Recommended)

```json theme={null}
{
  "providers": {
    "default": {
      "baseUrl": "https://api.anthropic.com/v1",
      "apiKey": "sk-ant-...",
      "api": "anthropic",
      "authHeader": true,
      "models": [{
        "id": "claude-sonnet-4-20250514",
        "name": "Claude Sonnet 4",
        "contextWindow": 200000,
        "maxTokens": 16000
      }]
    }
  }
}
```

Recommended models: `claude-sonnet-4-20250514` (best balance) or `claude-opus-4-20250514` (highest quality).

## OpenAI

```json theme={null}
{
  "providers": {
    "default": {
      "baseUrl": "https://api.openai.com/v1",
      "apiKey": "sk-...",
      "api": "openai-completions",
      "authHeader": true,
      "models": [{
        "id": "gpt-4o",
        "name": "GPT-4o",
        "contextWindow": 128000,
        "maxTokens": 16384
      }]
    }
  }
}
```

## OpenAI-Compatible Providers

Any API that implements the OpenAI chat completions format works with Siclaw — Ollama, vLLM, LiteLLM, Azure OpenAI, Moonshot, DeepSeek, and many others.

```json theme={null}
{
  "providers": {
    "default": {
      "baseUrl": "http://localhost:11434/v1",
      "apiKey": "ollama",
      "api": "openai-completions",
      "authHeader": true,
      "models": [{
        "id": "llama3.1:70b",
        "name": "Llama 3.1 70B",
        "contextWindow": 131072,
        "maxTokens": 8192
      }]
    }
  }
}
```

### Common Providers

| Provider             | `baseUrl`                                           | Notes                                   |
| -------------------- | --------------------------------------------------- | --------------------------------------- |
| **Ollama**           | `http://localhost:11434/v1`                         | Local, free. Use 70B+ for best results. |
| **vLLM**             | `http://localhost:8000/v1`                          | Self-hosted GPU inference               |
| **Moonshot (Kimi)**  | `https://api.moonshot.cn/v1`                        | `moonshot-v1-128k`                      |
| **DeepSeek**         | `https://api.deepseek.com/v1`                       | `deepseek-chat`                         |
| **Qwen (DashScope)** | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `qwen-plus`                             |

<Tip>
  See [`settings.example.json`](https://github.com/scitix/siclaw/blob/main/settings.example.json) for a complete example with all fields.
</Tip>

## Configuration Methods

* **TUI mode**: First-run wizard or `/setup` command inside a session
* **Local Server / Gateway**: Configure providers and models in the Web UI

## Embedding Provider

<Warning>
  Without an embedding provider, Investigation Memory semantic search is disabled. All other features work normally.
</Warning>

Embedding is used for memory search — matching current symptoms against past investigation records. Any OpenAI-compatible embedding API works:

```json theme={null}
{
  "embedding": {
    "baseUrl": "https://api.example.com/v1",
    "apiKey": "sk-...",
    "model": "bge-m3",
    "dimensions": 1024
  }
}
```

If `embedding.apiKey` is omitted, Siclaw falls back to the default provider API key.

| Provider                 | Model                    | Dimensions | Notes                                        |
| ------------------------ | ------------------------ | ---------- | -------------------------------------------- |
| **BGE-M3** (recommended) | `bge-m3`                 | 1024       | Multilingual, good for technical content     |
| **OpenAI**               | `text-embedding-3-small` | 1536       | Easy setup if you already have an OpenAI key |
| **Ollama**               | `nomic-embed-text`       | 768        | Local, free                                  |

## Model Recommendations

| Use Case                        | Recommended               | Notes                                |
| ------------------------------- | ------------------------- | ------------------------------------ |
| **Production investigations**   | Claude Sonnet 4 / GPT-4o  | Best quality-to-speed ratio          |
| **Complex root cause analysis** | Claude Opus 4             | Highest reasoning capability         |
| **Cost-sensitive / air-gapped** | Llama 3.1 70B+ via Ollama | Local, no API costs                  |
| **Testing / development**       | Any available model       | Smaller models work for basic checks |


# Channels & Integrations
Source: https://docs.siclaw.ai/features/channels

Access Siclaw via Terminal, Web UI, Slack, Lark, webhooks, and scheduled patrols.

## Supported Channels

All channels share the same investigation engine — the interface differs but the diagnostic capability is identical.

| Channel            | Mode    | Status |
| ------------------ | ------- | ------ |
| **Terminal (TUI)** | CLI     | Stable |
| **Web UI**         | Gateway | Stable |
| **Slack**          | Gateway | Stable |
| **Lark (Feishu)**  | Gateway | Stable |
| **Discord**        | Gateway | Stable |
| **Telegram**       | Gateway | Stable |
| **Webhooks**       | Gateway | Stable |

## Web UI

Available in Local Server mode (`siclaw local`) and Kubernetes deployments:

```bash theme={null}
siclaw local
# Open http://localhost:3000
```

For development (running Runtime and Portal separately):

```bash theme={null}
npm run dev:portal    # Portal + Web UI
npm run dev:runtime   # Runtime control plane
```

The Web UI provides:

* Real-time investigation with streaming updates
* Session history and investigation reports
* Skill management (create, review, publish)
* Built-in metrics dashboard (token usage, cost, latency, sessions)
* MCP server configuration
* Cron job management

## Slack Integration

### Setup

1. Create a Slack App at [api.slack.com/apps](https://api.slack.com/apps)
2. Enable **Socket Mode** and **Event Subscriptions**
3. Subscribe to events: `app_mention`, `message.im`
4. Add bot scopes: `chat:write`, `app_mentions:read`, `im:history`
5. In Siclaw Web UI, open **Channels** and save the bot credentials for Slack:

   * `botToken`
   * `appToken`

### Usage

Mention the bot in any channel:

```
@siclaw Pod payment-service is CrashLoopBackOff in prod
```

Or send a direct message for private investigations.

## Lark (Feishu) Integration

Create a Lark bot app, enable message event subscriptions, then configure it in **Channels** with:

* `appId`
* `appSecret`

## Discord Integration

Configure a Discord bot in **Channels** with:

* `token`

## Telegram Integration

Configure a Telegram bot in **Channels** with:

* `botToken`

## Alert Webhooks

Create a Trigger in the **Triggers** page. Siclaw will generate:

* a unique webhook URL
* a bearer secret

Requests should be sent to the generated endpoint:

```bash theme={null}
curl -X POST https://siclaw.example.com/hooks/v1/<trigger-id> \
  -H "Authorization: Bearer <trigger-secret>" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "High error rate on payment-service",
    "severity": "critical",
    "labels": {"namespace": "prod", "service": "payment"}
  }'
```

Compatible with: **Prometheus Alertmanager**, **Grafana**, **PagerDuty**, and any custom webhook (JSON format).

## Scheduled Patrols (Cron)

Schedule recurring health checks using natural language:

```
"Check GPU utilization across all training nodes every 6 hours"
"Verify pod health in the payments namespace every 30 minutes"
"Run full cluster health check at 9am every Monday"
```

### Creating a Patrol

**Via Web UI**: Go to **Cron** in the sidebar, click **New Patrol**, describe what to check and how often.

Each patrol run produces a status (success / warning / failure), output summary, and timestamp — all visible in the Web UI.

## MCP Servers

Siclaw supports the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) for connecting external data sources — Prometheus metrics, GitHub issues, custom APIs — as investigation tools. The agent discovers and uses them automatically.

See [MCP Servers](/configuration/mcp) for setup and examples.


# Deep Investigation
Source: https://docs.siclaw.ai/features/deep-investigation

How Siclaw's interactive investigation mode works — a focus mode for complex root-cause analysis.

## Overview

Deep Investigation (DP) is a **mode** you toggle on when you want Siclaw to slow
down and reason through a complex problem instead of jumping to a fix. In DP
mode the agent is coached by an enhanced system prompt to:

1. Gather evidence with tools before forming hypotheses.
2. Consider 2–5 candidate causes and weigh them against the evidence.
3. Cross-verify from multiple data sources before drawing conclusions.
4. Present findings with explicit confidence and a causal chain.

DP mode stays on until you explicitly turn it off. It does not activate by
itself.

## Triggering Deep Investigation

### Web UI

Click the magnifying-glass toggle next to the message input. A banner appears
while DP mode is active. Click again (or type `/dp exit`) to turn it off.

### Terminal (TUI)

```
> /dp "Intermittent 5xx errors on API gateway every 30 minutes"
```

Or press `Ctrl+I` to toggle DP mode, then type your question.

### Dig Deeper

Any regular answer can be escalated into DP by clicking the **Dig deeper**
action chip that appears below the agent's reply. This sends a follow-up prompt
asking for a deeper investigation into the same question.

### External API

Automations can drive an agent from outside Siclaw using the `/api/v1/run`
endpoint with an API-key-scoped agent. Issue an API key in the Portal UI
(agent detail page → API keys), then:

```bash theme={null}
curl -X POST https://siclaw.example.com/api/v1/run \
  -H "Authorization: Bearer sk-xxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"text": "Intermittent 5xx errors on API gateway every 30 minutes"}'
```

The agent resolved from the API key runs synchronously and returns the
assistant reply. DP-style reasoning can be requested in the prompt text.

## What DP Changes

DP mode changes how the agent *thinks* and, in supported runtimes, exposes a
same-agent delegation primitive for bounded parallel evidence collection. Normal
chat sessions do not see the delegation tool.

* **Slower triage**: the agent spends more effort cross-verifying evidence
  before proposing a cause.
* **Hypothesis framing**: conclusions are presented with confidence
  and the counter-evidence considered.
* **Parallel evidence collection**: for complex DP investigations, the agent can
  start one to three same-agent sub-investigations and then synthesize their
  returned evidence capsules.
* **Conversation continuity**: DP state persists across messages in the same
  session until explicitly exited.

## Investigation Memory

When [Investigation Memory](/features/memory) is enabled, DP-mode sessions
write their conclusions (root cause, causal chain, evidence refs) to the
memory index. Future investigations with similar symptoms retrieve past
matches automatically to speed up triage.


# Investigation Memory
Source: https://docs.siclaw.ai/features/memory

How Siclaw learns from past investigations to improve future diagnoses.

## Overview

Every deep investigation helps Siclaw get better at future investigations.

Siclaw stores:

* a human-readable investigation report
* a structured summary of the incident
* searchable memory used to improve future hypothesis generation

## How It Works

### Writing Memory

After an investigation finishes, Siclaw saves key facts such as:

* likely root cause
* affected services or resources
* important evidence
* recommended next steps

### Reading Memory

When a new investigation starts, Siclaw can look up similar past incidents and use them to improve hypothesis quality.

This makes repeated incident patterns easier to recognize over time.

<Warning>
  Memory search works best with an [embedding provider](/configuration/providers#embedding). Without one, Siclaw still saves investigation history, but semantic recall is reduced.
</Warning>


# Skill System
Source: https://docs.siclaw.ai/features/skills

Extend Siclaw with reusable diagnostic playbooks and custom scripts.

## What Are Skills?

Skills are reusable diagnostic playbooks. They let your team package recurring investigation steps into a named capability that Siclaw can run on demand.

Typical examples:

* Check why a deployment rollout is stuck
* Review GPU health across training nodes
* Validate a service's DNS and network path
* Collect the same evidence bundle every time an alert fires

## How Teams Use Skills

Siclaw ships with built-in skills, and teams can add their own through the Web UI.

* **Core skills**: built-in and maintained by Siclaw
* **Team skills**: shared across your organization after review
* **Personal skills**: private to one user unless promoted later

## Creating a Skill

```markdown theme={null}
# GPU Health Check

Check NVIDIA GPU health status across cluster nodes.

## Parameters

- `namespace` (optional): Target namespace. Default: all namespaces.
- `node` (optional): Specific node to check.

## Usage

Check all GPUs: `local_script gpu-health-check`
Check specific node: `local_script gpu-health-check --node gpu-worker-01`
```

You can create and edit skills in the Web UI, then describe:

* what the skill is for
* which parameters it accepts
* how Siclaw should use it during an investigation
* optional scripts when a structured workflow needs custom execution

## Review and Approval

Skills with scripts go through a mandatory review workflow before they become available:

```
draft → request review → pending → AI + static analysis → approved/rejected
```

The review process checks for risky behavior such as destructive shell commands, unsafe script patterns, or actions that would violate Siclaw's read-only model.

Only approved skills can be used in shared environments.

<Note>
  Siclaw is designed for read-only diagnostics. Team skills should follow the same model: investigate, summarize, and recommend actions rather than change production systems directly.
</Note>

## Running Skills

Skills can be used in several ways:

* directly during a conversation
* as part of a deeper investigation
* from scheduled patrols
* from webhook-triggered workflows

This makes them a good fit for turning repeated incident response habits into reusable, reviewable runbooks.


# Agent Tracing
Source: https://docs.siclaw.ai/features/tracing

Export agent behavior to Langfuse, Phoenix, or any OTLP backend for observability and evaluation.

## Overview

Siclaw can record what every agent does — the prompt, each LLM call, every tool
invocation, token usage, the model that answered, and latency — and stream it to
one or more third-party analysis platforms using **OpenTelemetry**.

Each user prompt becomes one trace: a root `agent.prompt` span with the LLM and
tool calls nested underneath, tagged with the model name and per-call token
counts. The root span also carries `session.id` and `user.id`, identifying the
conversation and the user who sent the prompt — both travel with the prompt
request, so traces group by session and by user in every deployment mode
(per-user pod, in-process, or a shared multi-user runtime). This is the same data
you'd use to debug a slow investigation, audit what an agent did, or evaluate
prompt/model changes over time.

Tracing is **off by default**, **admin-managed from the web UI**, and **global** —
one configuration is shared by every agent.

## Supported platforms

| Platform         | OTLP endpoint                              | Credentials                                  |
| ---------------- | ------------------------------------------ | -------------------------------------------- |
| **Langfuse**     | `https://<host>/api/public/otel/v1/traces` | Public key + Secret key (sent as HTTP Basic) |
| **Phoenix**      | `http://<host>:6006/v1/traces`             | API key (Bearer) + project name              |
| **Generic OTLP** | any OTLP/HTTP traces endpoint              | Arbitrary headers (JSON)                     |

You can configure **several platforms at once** — every agent run fans out to
each enabled platform from a single recording path.

## Configure

Open the web UI as an admin and go to **Metrics → Tracing**.

1. Click **Add platform**, pick the type (Langfuse / Phoenix / Generic OTLP).
2. Enter the **OTLP endpoint URL** — for Langfuse this must include the full path
   `…/api/public/otel/v1/traces`, not just the host.
3. Enter the credentials (Siclaw assembles the auth header for you — e.g. it
   base64-encodes the Langfuse key pair into a Basic header).
4. Make sure the platform's toggle is **on**.

Tracing is active whenever **at least one platform is enabled** — there is no
separate master switch. To pause tracing, disable every platform.

### Global settings

* **`service.name`** — the OpenTelemetry service identifier stamped on every
  trace. Defaults to `siclaw-agentbox`. Set distinct names (e.g. `siclaw-prod`,
  `siclaw-staging`) when several Siclaw deployments report to the same backend so
  you can tell them apart.
* **Environment** (`deployment.environment.name`) — buckets traces by environment
  in Langfuse's environment filter. Read from `SICLAW_TRACING_ENVIRONMENT` in the
  agentbox. **Portal injects it per agent** (derived from the runtime the agent is
  bound to), so this needs no per-deployment configuration — that is the authoritative
  source when present. Setting `SICLAW_TRACING_ENVIRONMENT` on the runtime deployment
  is a fallback for deployments where Portal does not supply one. The value is
  normalized on read (lowercased, illegal characters → `-`, ≤40 chars, reserved
  `langfuse` prefix stripped) so an unnormalized runtime name still lands in a valid
  bucket rather than silently falling back to Langfuse's `default`. Unset everywhere ⇒
  `default`.
* **Send content** — a privacy gate. When **off** (default) only metadata and the
  call tree are exported (model, tokens, tool names, latency). When **on**, the
  actual LLM prompts/responses and tool arguments are exported too — far more
  useful for debugging, but it sends conversation content to the platform.

<Warning>
  Turn **Send content** on only for a backend inside your trust domain (e.g. a
  self-hosted Langfuse on your internal network). Tool **output** is always
  sanitized; Send content additionally releases LLM input/output and tool
  arguments.
</Warning>

## Changes take effect live

Adding, editing, toggling, or deleting a platform — and changing the global
settings — **hot-reloads every running agent** without a restart or redeploy.
Behind the scenes the change is broadcast to all active AgentBoxes, which rebuild
their exporters in place. New agents pick up the current configuration when they
start.

## Test a platform

Each platform row has a **Test** button. It fires an empty OTLP request to the
configured endpoint with the stored credentials and reports the real HTTP status,
so you can confirm the URL and auth are correct before relying on it. The probe
is restricted to `http`/`https` and blocks cloud-metadata and link-local
addresses.

## Notes

* Credentials are stored server-side and **masked** in the API and UI (only a
  prefix is shown); editing a platform without re-entering the secret keeps the
  stored value.
* Tracing covers agents running in Kubernetes (one pod per user), local server
  mode, and the standalone CLI.
* This is **Plane A** observability (online agent-behavior traces). It is separate
  from the Prometheus/Grafana metrics on the other Metrics tabs.


# Siclaw
Source: https://docs.siclaw.ai/index

AI-powered SRE copilot — hypothesis-driven deep investigation that learns from every incident.

# Siclaw

An open-source AI agent that diagnoses Kubernetes infrastructure issues the way your best engineer does: gather context, form hypotheses, validate them in parallel, and produce evidence-backed conclusions.

**Read-only by default.** Siclaw never modifies your cluster.

## What Makes Siclaw Different

Siclaw runs a **complete investigation workflow**, not just a single command followed by another prompt:

```
You: "Pod payment-service is CrashLoopBackOff after deploying v2.3"

Siclaw:
  Phase 1  Gathered 12 signals (pod status, events, logs, recent deploys)
  Phase 2  Generated 3 hypotheses, ranked by evidence
  Phase 3  3 sub-agents validated hypotheses in parallel (47s)
  Phase 4  Root cause: OOMKilled — memory limit 256Mi insufficient for v2.3
           Confidence: 92% | Evidence chain: 4 signals
           Remediation: increase memory limit to 512Mi
```

## Key Capabilities

<CardGroup>
  <Card title="Deep Investigation" icon="magnifying-glass" href="/features/deep-investigation">
    4-phase workflow for evidence gathering, hypothesis testing, and root-cause analysis.
  </Card>

  <Card title="Investigation Memory" icon="brain" href="/features/memory">
    Learns from every investigation. Past diagnoses improve future hypotheses automatically.
  </Card>

  <Card title="Extensible Skills" icon="puzzle-piece" href="/features/skills">
    Custom diagnostic playbooks with mandatory security review. Your team's runbooks, executable by AI.
  </Card>

  <Card title="Multi-Channel" icon="comments" href="/features/channels">
    Terminal, Web UI, Slack, Lark, Discord, Telegram. Same investigation engine, any interface.
  </Card>
</CardGroup>

## Quick Start

```bash theme={null}
mkdir -p ~/siclaw-work && cd ~/siclaw-work
npm install -g siclaw
siclaw local        # Web UI at http://localhost:3000
```

Or `siclaw` for personal CLI mode.

<CardGroup>
  <Card title="Getting Started" icon="rocket" href="/start/getting-started">
    Install and configure in under 5 minutes.
  </Card>

  <Card title="Your First Investigation" icon="play" href="/start/first-investigation">
    Walk through a complete diagnosis step by step.
  </Card>
</CardGroup>

## When to Use Siclaw

**Good fit:**

* Kubernetes pod crashes, restarts, scheduling failures
* Node issues (NotReady, resource pressure, kernel problems)
* Network connectivity and DNS resolution problems
* Deployment rollout failures and config drift
* Recurring incidents where institutional knowledge matters

**Not designed for:**

* Real-time alerting (use Prometheus/Grafana for that)
* Automated remediation (Siclaw diagnoses, humans fix)
* Non-Kubernetes infrastructure (cloud VMs, bare metal)


# CLI & Local Server
Source: https://docs.siclaw.ai/install/cli

Install Siclaw globally and run as CLI or local server.

## Recommended Working Directory

Siclaw stores most runtime data in a `.siclaw/` folder **relative to the directory where you start it**. Create a dedicated working directory and reuse it:

```bash theme={null}
mkdir -p ~/siclaw-work
cd ~/siclaw-work
```

## Install

```bash theme={null}
npm install -g siclaw
```

## Local Server (recommended)

```bash theme={null}
siclaw local
```

Launches the Web UI at `http://localhost:3000`.

On first launch, open the Web UI and **register the first user** — that account becomes the admin. Subsequent registrations require admin authentication.

## CLI (TUI)

```bash theme={null}
siclaw
```

Personal terminal mode. Same investigation engine, no server.

### Pairing with a local server

If `siclaw local` is already running in the **same working directory**, the TUI detects it automatically and uses the Portal Web UI as the single source of truth for skills, knowledge, credentials, agents, MCP servers, and LLM providers. In that mode the TUI is strictly an observer:

* `/ls [skills|knowledge|mcp|credentials|agents]` — inspect what the current session sees
* `/agent` — show the active Portal agent and all available ones; create / edit happens in the Web UI
* `/setup` — read-only view with "Open in Portal →" links (standalone mode still writes locally)

Pass `--agent <name>` to scope the session to one Portal-configured agent:

```bash theme={null}
siclaw --agent sre-oncall
siclaw agents            # list available agents non-interactively
```

When `siclaw local` is running in the cwd and you start `siclaw` for the first time (no `settings.json` yet), the setup wizard prints Portal Web UI instructions, offers to open `http://localhost:3000/settings/models` in your browser, and exits — provider setup belongs in Portal, not in a per-workstation `settings.json`.

## From Source

```bash theme={null}
git clone https://github.com/scitix/siclaw.git
cd siclaw
npm ci
make build-portal-web
npm run build

# TUI
node siclaw.mjs

# Local server
node siclaw.mjs local
```

## Configuration

By default, Siclaw reads and writes these paths relative to your current working directory:

```
.siclaw/
├── config/
│   └── settings.json     ← LLM provider, model, API key
├── credentials/           ← Imported kubeconfigs, SSH keys, API tokens
├── skills/
└── user-data/
    ├── memory/
    │   └── .memory.db
    └── ...
```

### LLM Configuration

Standalone TUI (no local Portal running in the cwd) reads `.siclaw/config/settings.json`. The easiest path is to let the first-run wizard create it for you. If a `siclaw local` server is running in the same cwd, the wizard redirects to Portal Web UI instead (see [Pairing with a local server](#pairing-with-a-local-server)) — provider setup for Portal-paired TUI happens in the Web UI's **Models** page and applies to every TUI that pairs with that Portal.

Minimal example:

```json theme={null}
{
  "providers": {
    "openai": {
      "baseUrl": "https://api.openai.com/v1",
      "apiKey": "sk-...",
      "api": "openai-completions",
      "models": [{ "id": "gpt-4o", "name": "GPT-4o" }]
    },
    "anthropic": {
      "baseUrl": "https://api.anthropic.com/v1",
      "apiKey": "sk-ant-...",
      "api": "anthropic",
      "models": [{ "id": "claude-sonnet-4-20250514", "name": "Claude Sonnet 4" }]
    }
  }
}
```

Once inside the TUI session, use `/setup` to manage providers, models, and credentials.

In Local Server mode, configure providers and models in the **Models** page of the Web UI.

See [LLM Providers](/configuration/providers) for provider examples.

### Kubernetes Credentials

Siclaw tools resolve kubeconfig from its own credential store, not from your shell's `KUBECONFIG`.

* In TUI mode, use `/setup` to import a kubeconfig
* In Local Server mode, add it in **Credentials**

After import, Siclaw can route `kubectl` calls through that stored credential.

## Embedding (Optional)

To enable Investigation Memory with semantic search, add an embedding config:

```json theme={null}
{
  "embedding": {
    "baseUrl": "https://api.example.com/v1",
    "apiKey": "sk-...",
    "model": "bge-m3",
    "dimensions": 1024
  }
}
```

If `embedding.apiKey` is omitted, Siclaw falls back to the default LLM provider key.

## Traces

Deep investigation traces are written to:

```text theme={null}
.siclaw/traces/
```

The path is resolved relative to the working directory where Siclaw was launched.


# Container Images
Source: https://docs.siclaw.ai/install/docker

Build and publish Siclaw container images for Kubernetes deployment.

## What Docker Supports Today

Siclaw ships three Dockerfiles, one per deployment image:

* `portal` — Web UI + REST API + DB (the user-facing front door)
* `runtime` — Control plane: channels, cron, AgentBox spawner
* `agentbox` — Per-user execution runtime (spawned by Runtime)

For a single-machine setup, use [CLI & Local Server](/install/cli). The repository does **not** include a supported `docker compose` deployment.

## Build Images

From the repo root:

```bash theme={null}
make docker REGISTRY=registry.example.com/myteam TAG=latest
```

That builds:

* `registry.example.com/myteam/siclaw-runtime:latest`
* `registry.example.com/myteam/siclaw-portal:latest`
* `registry.example.com/myteam/siclaw-agentbox:latest`

To build one image only:

```bash theme={null}
make docker-runtime  REGISTRY=registry.example.com/myteam TAG=latest
make docker-portal   REGISTRY=registry.example.com/myteam TAG=latest
make docker-agentbox REGISTRY=registry.example.com/myteam TAG=latest
```

## Push Images

```bash theme={null}
make push REGISTRY=registry.example.com/myteam TAG=latest
```

Or push individually:

```bash theme={null}
make push-runtime  REGISTRY=registry.example.com/myteam TAG=latest
make push-portal   REGISTRY=registry.example.com/myteam TAG=latest
make push-agentbox REGISTRY=registry.example.com/myteam TAG=latest
```

## Runtime Notes

* The Portal image serves the Web UI on container port `3003`.
* Runtime and AgentBox communicate over mTLS inside the cluster.
* The AgentBox image is designed to be spawned by the Runtime inside Kubernetes.
* Core skills are baked into the images; dynamic skills and credentials are synced at runtime.

## Next Step

After publishing images, continue with [Kubernetes Deployment](/install/kubernetes).


# Kubernetes Deployment
Source: https://docs.siclaw.ai/install/kubernetes

Deploy Siclaw on Kubernetes with full tenant isolation using Helm.

## Architecture

Kubernetes is the production deployment model for Siclaw:

```
Portal Deployment
  ├── Web UI (React) + REST API
  ├── Auth + user management
  └── MySQL connection

Runtime Deployment
  ├── Channels (Slack / Lark / Discord / Telegram)
  ├── Cron / scheduled tasks
  └── K8s AgentBox spawner

AgentBox Pod  (spawned per user / workspace)
  ├── Isolated agent runtime
  ├── Synced skills and credentials
  └── Internal mTLS back to Runtime
```

## Prerequisites

* A Kubernetes cluster
* A MySQL database reachable from the cluster
* Published Siclaw images for `runtime`, `portal`, and `agentbox`

Build and push images first if you are using your own registry:

```bash theme={null}
make docker REGISTRY=registry.example.com/myteam TAG=latest
make push REGISTRY=registry.example.com/myteam TAG=latest
```

## Quick Start

Install from the chart in this repository:

```bash theme={null}
helm upgrade --install siclaw ./helm/siclaw \
  --namespace siclaw \
  --create-namespace \
  --set image.registry=registry.example.com/myteam \
  --set image.tag=latest \
  --set database.url="mysql://user:pass@mysql.svc.cluster.local:3306/siclaw"
```

All three images share `image.registry` and `image.tag` by default.

## Important Values

Current top-level values look like this:

```yaml theme={null}
image:
  registry: registry.example.com/myteam
  tag: latest

runtime:
  replicas: 1

portal:
  enabled: true
  replicas: 1
  service:
    type: NodePort
    port: 3003
    nodePort: 31003

database:
  url: mysql://user:pass@mysql.svc.cluster.local:3306/siclaw
```

Use `database.existingSecret.name` if you do not want to pass the connection string on the command line.

## Accessing the UI

The Portal Service is the front door. With default values it listens on port `3003` inside the cluster and on NodePort `31003` on any node.

* Keep `portal.service.type: NodePort` for quick access via `http://<any-node-ip>:31003`
* Or point an Ingress at the Portal Service on service port `3003`

Example Ingress backend:

```yaml theme={null}
backend:
  service:
    name: siclaw-portal
    port:
      number: 3003
```

<Note>
  WebSocket support is required for live investigation updates. Keep proxy read/send timeouts high enough for multi-minute investigations.
</Note>

## Authentication

On first launch, open the Portal UI and register the first user — that account becomes the admin. Registration is open only for the very first account; every subsequent registration requires admin authentication.

## Metrics

Runtime and Portal both expose Prometheus metrics at `/metrics`. The chart can create ServiceMonitor, PodMonitor, Grafana dashboard, and PrometheusRule resources under the `metrics.*` values block.

The Runtime endpoint also exposes bounded KB capability lifecycle series such as
`siclaw_gateway_capability_starts_total`,
`siclaw_gateway_capability_start_duration_ms`,
`siclaw_gateway_capability_active_runs`,
`siclaw_gateway_capability_materialization_failures_total`, and
`siclaw_gateway_capability_relay_failures_total`. They intentionally carry no
run, repository, user, or operation identifiers; use logs/traces for drill-down.

Common settings:

```yaml theme={null}
metrics:
  enabled: true
  serviceMonitor:
    enabled: true
  podMonitor:
    enabled: true
  grafanaDashboard:
    enabled: true
  prometheusRule:
    enabled: false
```

If you want bearer-token protection for `/metrics`, add:

```yaml theme={null}
runtime:
  env:
    SICLAW_METRICS_TOKEN: "your-secret-token"
```

To remove the `user_id` label from token and cost metrics:

```yaml theme={null}
runtime:
  env:
    SICLAW_METRICS_USER_ID: "false"
```

## Operational Notes

* Kubernetes mode requires MySQL. SQLite is only for single-process local use.
* AgentBox pods are created on demand by the Runtime.
* Runtime ↔ AgentBox traffic is secured with mTLS automatically.
* KB compile/test boxes additionally receive a default-on ingress NetworkPolicy;
  the chart's Runtime pod is the only default caller on port 3000. Extend
  `agentbox.networkPolicy.ingressFrom` only for an intentional extra caller.
* A Runtime rolling restart preserves K8s AgentBox pods so the replacement can
  re-adopt live capability runs. Cluster-wide box deletion remains an explicit
  cleanup operation, not a process-shutdown side effect.
* If you deploy monitoring resources from the chart, do not also apply duplicate monitor manifests manually.


# Core Concepts
Source: https://docs.siclaw.ai/start/core-concepts

Key ideas behind Siclaw's investigation approach.

## How Siclaw Investigates

Siclaw uses a **4-phase hypothesis-driven investigation engine**: gather context → generate hypotheses → validate in parallel → produce a structured report with root cause and remediation.

Every investigation is **read-only** — Siclaw never modifies your cluster. See [Deep Investigation](/features/deep-investigation) for the full workflow.

## Runtime Modes

Siclaw runs in three modes sharing one agent core:

| Mode             | Use Case                                      | Start Command                                     |
| ---------------- | --------------------------------------------- | ------------------------------------------------- |
| **CLI (TUI)**    | Personal terminal diagnostics                 | `siclaw`                                          |
| **Local Server** | Team use with Web UI and shared configuration | `siclaw local`                                    |
| **Kubernetes**   | Production multi-tenant deployment            | `helm upgrade --install siclaw ./helm/siclaw ...` |

Use `siclaw` for personal terminal workflows, `siclaw local` for a browser-based local setup, and Kubernetes for team deployments.

## Skills

Skills are reusable diagnostic playbooks your team can create, review, and share. See [Skills](/features/skills) for details.

## Investigation Memory

Siclaw can reuse findings from previous incidents so recurring problems are recognized faster. See [Memory](/features/memory) for details.

## Security Model

Siclaw is designed for read-only diagnostics. It investigates, explains, and recommends next steps without changing your environment directly.

## MCP Integration

Siclaw supports the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) for connecting external tools and data sources. See [MCP Servers](/configuration/mcp) for setup and examples.


# Your First Investigation
Source: https://docs.siclaw.ai/start/first-investigation

Walk through a complete deep investigation from start to finish.

## Scenario

A pod in your production cluster is stuck in `CrashLoopBackOff` after a deployment. Let's use Siclaw to diagnose it.

## Start Siclaw

```bash theme={null}
siclaw
```

If you want Siclaw to inspect a Kubernetes cluster, import a kubeconfig first with `/setup`.

## Describe the Problem

```
? What would you like to investigate?
> Pod payment-service is CrashLoopBackOff in namespace prod after deploying v2.3
```

<Tip>
  Be specific. Include the pod name, namespace, and what changed (e.g., "after deploying v2.3"). More context = better hypotheses.
</Tip>

## Phase 1: Context Gathering

Siclaw automatically runs diagnostic commands to understand the situation:

```
── Phase 1: Context Gathering ─────────────────────────────
  kubectl get pods -n prod | grep payment-service
  kubectl describe pod payment-service-xxx -n prod
  kubectl logs payment-service-xxx -n prod --previous
  kubectl get events -n prod --sort-by='.lastTimestamp'
```

All commands are **read-only** — nothing is modified.

## Phase 2: Hypothesis Generation

Based on the evidence, Siclaw generates ranked hypotheses:

```
── Phase 2: Hypothesis Generation ──────────────────────────
  H1  OOMKilled — memory limit too low for v2.3      confidence: 78%
  H2  Config mount failure — missing configmap key    confidence: 45%
  H3  Liveness probe mismatch — endpoint changed      confidence: 32%
```

<Note>
  If Investigation Memory has data, Siclaw will also check past incidents for similar patterns and adjust hypothesis confidence accordingly.
</Note>

## Phase 3: Parallel Validation

Up to 3 sub-agents independently validate each hypothesis:

```
── Phase 3: Parallel Validation (3 sub-agents) ─────────────
  Agent-1 validating H1 · Agent-2 validating H2 · Agent-3 validating H3
```

Each sub-agent runs targeted commands to confirm or refute its hypothesis. They don't share information — this prevents confirmation bias.

## Phase 4: Conclusion

Siclaw synthesizes all evidence into a structured report:

```
── Phase 4: Conclusion ──────────────────────────────────────
  Root Cause: OOMKilled — memory limit 256Mi insufficient for v2.3
  Confidence: 92% · Evidence: 4 signals · Duration: 47s

  Causal chain:
    1. v2.3 deployment added new caching layer
    2. Memory usage increased from ~180Mi to ~310Mi
    3. Pod exceeded 256Mi memory limit
    4. Kernel OOMKilled the process → container restart → CrashLoopBackOff

  Recommended next steps:
    - Increase the deployment memory limit to 512Mi
    - Add memory requests to match expected usage
```

The full trace is saved to `.siclaw/traces/deep-search-{timestamp}.md` (relative to where Siclaw was launched).

## Deep Investigation Mode

For complex issues, you can explicitly trigger a deep investigation:

```
> /dp "Intermittent 5xx errors on the API gateway, happening every 30 minutes"
```

This activates the structured 4-phase workflow (triage → hypotheses → parallel validation → conclusion) for thorough investigation.

## More Examples

Here are other common scenarios Siclaw handles well:

### OOMKilled Pods

```
> Pods in namespace ml-training keep getting OOMKilled, happening more since yesterday
```

Siclaw will check memory limits vs actual usage, recent deployment changes, and correlate with node memory pressure.

### Node NotReady

```
> Node worker-07 went NotReady 20 minutes ago, pods are being evicted
```

Siclaw will inspect node conditions, kubelet logs, kernel messages (dmesg), and network connectivity to the API server.

### Intermittent Network Issues

```
> /dp "Service mesh intermittent 503 errors between order-service and inventory-service"
```

Using `/dp` triggers the full structured investigation — useful for complex cross-service issues.

## What's Next?

* [Deep Investigation](/features/deep-investigation) — how the investigation workflow works
* [Skills](/features/skills) — create reusable diagnostic playbooks
* [Memory](/features/memory) — how investigation history improves future diagnoses


# Getting Started
Source: https://docs.siclaw.ai/start/getting-started

Install Siclaw and run your first investigation in under 5 minutes.

## Prerequisites

* **Node.js** >= 22.19.0
* An **LLM API key** (Anthropic, OpenAI, or any OpenAI-compatible provider)

## Install

```bash theme={null}
mkdir -p ~/siclaw-work
cd ~/siclaw-work
npm install -g siclaw
```

### Local Server (recommended for teams)

```bash theme={null}
siclaw local
```

Launches the Web UI at `http://localhost:3000`.

On first launch, open the page and **register the first user** — that account becomes the admin. After that, new registrations require admin authentication.

### CLI

```bash theme={null}
siclaw
```

TUI mode for personal terminal diagnostics.

### From Source

```bash theme={null}
git clone https://github.com/scitix/siclaw.git
cd siclaw
npm ci
make build-portal-web
npm run build
node siclaw.mjs
```

## Configure Your LLM

On first run, Siclaw launches a setup wizard to configure your LLM provider.

<Note>
  If you started `siclaw local` first (recommended for teams), the wizard detects the running Portal and redirects provider setup to the Portal Web UI's **Models** page. One configuration in Portal serves every paired TUI — no per-workstation `settings.json` to drift out of sync.
</Note>

If you prefer the standalone TUI path, you can also edit `.siclaw/config/settings.json` manually:

```json theme={null}
{
  "providers": {
    "default": {
      "baseUrl": "https://api.anthropic.com/v1",
      "apiKey": "sk-ant-...",
      "api": "anthropic",
      "models": [{ "id": "claude-sonnet-4-20250514", "name": "Claude Sonnet 4" }]
    }
  }
}
```

<Note>
  Siclaw supports any OpenAI-compatible API. See [LLM Providers](/configuration/providers) for Ollama, vLLM, Azure, and other setups.
</Note>

## Run Your First Investigation

Describe an issue:

```
? What would you like to investigate?
> Pod CrashLoopBackOff in production cluster after deployment
```

For complex issues, activate [Deep Investigation](/features/deep-investigation) mode (`/dp` or Ctrl+I) to run a structured investigation — the agent triages, proposes hypotheses for your review, validates in parallel after your confirmation, and produces a structured report with root cause, confidence score, and remediation steps.

Investigation traces are saved to `.siclaw/traces/` (relative to where Siclaw was launched).

## Add Cluster Access

To investigate Kubernetes issues, import a kubeconfig into Siclaw:

* **Standalone TUI** (no local Portal): `/setup` inside the session
* **Local Server, or TUI paired with one**: **Clusters** / **Hosts** in the Web UI — paired TUIs pick up the imports on the next launch

Siclaw uses its stored credentials when running diagnostic tools.

## What's Next?

* [Core Concepts](/start/core-concepts) — understand the investigation engine
* [Your First Investigation](/start/first-investigation) — walk through a complete diagnosis
* [LLM Providers](/configuration/providers) — detailed provider configuration
* [Deploy for your team](/install/kubernetes) — production multi-user deployment


# Troubleshooting
Source: https://docs.siclaw.ai/start/troubleshooting

Solutions for common setup and runtime issues.

## Node.js Version

Siclaw requires Node.js 22.19.0 or later.

```bash theme={null}
node --version
# Must be v22.19.0+
```

If you're on an older version, use [nvm](https://github.com/nvm-sh/nvm):

```bash theme={null}
nvm install 22
nvm use 22
```

## LLM Connection Failures

**Symptom**: "Failed to connect to LLM provider" or empty responses.

**Check**:

1. Open the active config file:
   ```bash theme={null}
   ls .siclaw/config/settings.json
   ```
2. Verify `baseUrl`, `api`, `apiKey`, and model ID are correct.
3. Test the provider directly:
   ```bash theme={null}
   curl https://api.anthropic.com/v1/messages \
     -H "x-api-key: YOUR_API_KEY" \
     -H "content-type: application/json" \
     -d '{"model":"claude-sonnet-4-20250514","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}'
   ```
4. For OpenAI-compatible providers, confirm `baseUrl` is correct and reachable from the machine running Siclaw.

## kubectl Permission Denied

**Symptom**: "Error from server (Forbidden)" during investigation.

Siclaw uses a kubeconfig imported into its credential store. Verify:

```bash theme={null}
# See whether Siclaw has imported credentials
ls .siclaw/credentials/manifest.json
```

If no kubeconfig has been imported yet:

* TUI: run `/setup`
* Local Server: open **Credentials** in the Web UI

Then verify the same kubeconfig can read your cluster:

```bash theme={null}
kubectl --kubeconfig /path/to/imported.kubeconfig get pods --all-namespaces
```

Siclaw needs read-oriented access. A typical minimum RBAC baseline is:

```yaml theme={null}
rules:
  - apiGroups: ["", "apps", "batch", "events.k8s.io"]
    resources: ["*"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods/log", "pods/exec"]
    verbs: ["get", "create"]
```

## Memory Search Not Working

**Symptom**: `memory_search` tool not available, or "embedding provider not configured".

Investigation Memory semantic search requires an embedding provider. Add to `.siclaw/config/settings.json`:

```json theme={null}
{
  "embedding": {
    "baseUrl": "https://api.example.com/v1",
    "apiKey": "sk-...",
    "model": "bge-m3",
    "dimensions": 1024
  }
}
```

Without this, all other features work normally — only semantic memory search is disabled.

## Port Conflicts

**Symptom**: "EADDRINUSE" when starting Gateway.

```bash theme={null}
# Check what's using the port
lsof -i :3000
```

Current Gateway builds listen on `3000` by default. Stop the conflicting process, then start Siclaw again.

## SQLite Lock Error

**Symptom**: "Database is locked" or "Another instance is already running".

Only one Siclaw process can use the same SQLite database at a time. Check for existing processes:

```bash theme={null}
ps aux | grep siclaw
# Kill stale processes if needed
```

The default lockfile is:

```text theme={null}
.siclaw/data.sqlite.lock
```

This path is relative to the directory where you started Siclaw.

## Skill Script Rejected

**Symptom**: Script stuck in "pending" or "rejected" status.

Skill scripts go through a 3-step review:

1. **Static analysis** — 27 danger patterns checked (e.g., `rm -rf`, `chmod 777`, `curl | sh`)
2. **AI review** — LLM checks for destructive operations
3. **Human approval** — a `skill_reviewer` must approve

If rejected, check the rejection reason in the Web UI and modify the script to be read-only.

## Getting Help

* [GitHub Issues](https://github.com/scitix/siclaw/issues) — bug reports and feature requests
* [Slack Community](https://join.slack.com/t/siclaw-scitix/shared_invite/zt-3rrsoc2ic-JIfbfvT1_04sqgQorSRfmw) — questions and discussion