> ## Documentation Index
> Fetch the complete documentation index at: https://docs.toktra.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Linux Agent

> Install and configure the Toktra Linux agent for enterprise LLM usage monitoring.

The Linux agent captures LLM usage metadata using eBPF socket tracing. It attaches BPF probes to the kernel's network stack and OpenSSL to extract SNI hostnames from outbound TLS connections — it **never blocks traffic and never inspects content**.

## Requirements

* Linux kernel 5.10 or later with BTF (BPF Type Format) enabled
* systemd
* Root privileges for installation (required to load BPF programs)

<Info>
  BTF is required for CO-RE (Compile Once, Run Everywhere) compatibility. To check whether your kernel has BTF enabled, run `ls /sys/kernel/btf/vmlinux`. If the file exists, your kernel is supported.
</Info>

## How it works

The agent consists of two components:

* **eBPF programs** — Seven BPF probes (tracepoints, kprobes, and OpenSSL uprobes) compiled with CO-RE for kernel 5.10+ portability. The probes track outbound TCP connections on port 443, measure bytes sent and received, and extract the TLS SNI hostname via OpenSSL uprobes attached to `libssl.so`. The BPF programs never block connections — they only observe and record.
* **Rust daemon (`toktra-agent`)** — An async daemon that polls the BPF ring buffer, classifies events against known LLM provider hostnames, batches events (up to 50 events or 5 seconds), and transmits telemetry securely to Toktra over mTLS. The daemon runs as a systemd service.

<Note>
  The eBPF programs operate in observe-only mode. No BPF action ever drops or modifies a packet. All connections proceed normally.
</Note>

## Installation

<Tabs>
  <Tab title="Debian / Ubuntu (.deb)">
    <Steps>
      <Step title="Download the package">
        Download the `.deb` package from the Toktra admin dashboard under **Devices → Download Agent → Linux**.
      </Step>

      <Step title="Install the package">
        ```bash theme={null}
        sudo dpkg -i toktra-agent_1.0.0_amd64.deb
        ```

        The postinst script runs `systemctl daemon-reload && systemctl enable --now toktra-agent` automatically.
      </Step>

      <Step title="Verify the service is running">
        ```bash theme={null}
        systemctl status toktra-agent
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="RHEL / CentOS / Fedora (.rpm)">
    <Steps>
      <Step title="Download the package">
        Download the `.rpm` package from the Toktra admin dashboard under **Devices → Download Agent → Linux**.
      </Step>

      <Step title="Install the package">
        ```bash theme={null}
        sudo rpm -i toktra-agent-1.0.0.x86_64.rpm
        ```

        Or with `dnf`:

        ```bash theme={null}
        sudo dnf localinstall toktra-agent-1.0.0.x86_64.rpm
        ```
      </Step>

      <Step title="Enable and start the service">
        ```bash theme={null}
        sudo systemctl enable --now toktra-agent
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="From source">
    Build the agent from source if you need to target a specific architecture or kernel configuration.

    ```bash theme={null}
    # Prerequisites: clang 14+, llvm-strip, Rust 1.75+
    cd agent-linux

    # Build eBPF programs + Rust daemon
    make all

    # Build and install
    sudo make install

    sudo systemctl daemon-reload
    sudo systemctl enable --now toktra-agent
    ```
  </Tab>
</Tabs>

### Installed paths

| Path                                           | Contents                  |
| ---------------------------------------------- | ------------------------- |
| `/usr/bin/toktra-agent`                        | Rust daemon binary        |
| `/usr/lib/toktra/toktra_socket.bpf.o`          | Compiled eBPF object file |
| `/usr/lib/systemd/system/toktra-agent.service` | systemd unit file         |
| `/etc/toktra/`                                 | Configuration directory   |

## Configuration

The agent loads configuration by layering: defaults ← `/etc/toktra/agent.conf` ← environment variables. Environment variables always take precedence.

### Configuration file

Create or edit `/etc/toktra/agent.conf` (TOML format):

```toml theme={null}
# Toktra ingest URL
ingest_url = "https://ingest.toktra.io"

# Device certificate paths (written by enrollment)
device_cert_path = "/etc/toktra/device.crt"
device_key_path  = "/etc/toktra/device.key"
ca_cert_path     = "/etc/toktra/ca.crt"

# Telemetry batching
batch_size          = 50   # events per batch
flush_interval_secs = 5    # seconds between flushes

# Logging
log_level = "info"  # trace, debug, info, warn, error

# Prometheus metrics
metrics_port = 9090

# eBPF configuration
bpf_object_path = "/usr/lib/toktra/toktra_socket.bpf.o"
libssl_path     = "/usr/lib/x86_64-linux-gnu/libssl.so.3"
```

### Environment variables

All configuration keys can be overridden with environment variables. Add overrides to `/etc/toktra/agent.env` (loaded by the systemd unit via `EnvironmentFile`):

```bash theme={null}
TOKTRA_INGEST_URL=https://ingest.toktra.io
TOKTRA_CERT_PATH=/etc/toktra/device.crt
TOKTRA_KEY_PATH=/etc/toktra/device.key
TOKTRA_CA_CERT_PATH=/etc/toktra/ca.crt
TOKTRA_LOG_LEVEL=info
TOKTRA_METRICS_PORT=9090
TOKTRA_BATCH_SIZE=50
TOKTRA_FLUSH_INTERVAL=5
```

After editing the configuration, restart the service:

```bash theme={null}
sudo systemctl restart toktra-agent
```

## Device enrollment

On first run, the daemon automatically enrolls the device:

1. Generates an Ed25519 key pair and writes the private key to `/etc/toktra/device.key`.
2. Sends a CSR to the Toktra enrollment endpoint.
3. Writes the signed device certificate to `/etc/toktra/device.crt`. The certificate is valid for 90 days.
4. Uses the device certificate for mTLS on all subsequent transmissions.

Certificates renew automatically before expiry.

## systemd service

The agent runs as a `Type=notify` systemd service and signals readiness with `sd_notify`. The unit applies security hardening directives:

```ini theme={null}
[Service]
Type=notify
ExecStart=/usr/bin/toktra-agent
Restart=on-failure
RestartSec=5
WatchdogSec=60
User=root

# Minimum capabilities for eBPF
CapabilityBoundingSet=CAP_BPF CAP_PERFMON CAP_NET_ADMIN CAP_SYS_PTRACE
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
```

Common service management commands:

```bash theme={null}
# Check status and recent logs
systemctl status toktra-agent
journalctl -u toktra-agent -f

# Restart after configuration change
sudo systemctl restart toktra-agent

# Disable and stop
sudo systemctl disable --now toktra-agent
```

## Prometheus metrics

The daemon exposes Prometheus-format metrics on `127.0.0.1:9090/metrics` (configurable with `metrics_port`). Scrape this endpoint with Prometheus or a compatible collector.

Available metrics:

| Metric                               | Type      | Description                                      |
| ------------------------------------ | --------- | ------------------------------------------------ |
| `toktra_events_received_total`       | Counter   | Events received from BPF ring buffer             |
| `toktra_events_sent_total`           | Counter   | Events sent to Toktra                            |
| `toktra_events_dropped_total`        | Counter   | Events dropped due to overflow or send failure   |
| `toktra_bpf_errors_total`            | Counter   | BPF subsystem errors (load failures, map errors) |
| `toktra_send_errors_total`           | Counter   | HTTP send errors                                 |
| `toktra_batch_send_duration_seconds` | Histogram | Batch send duration                              |
| `toktra_active_connections`          | Gauge     | Connections currently tracked in BPF map         |
| `toktra_buffered_events`             | Gauge     | Events buffered awaiting send                    |
| `toktra_circuit_breaker_open`        | Gauge     | Circuit breaker state (`1`=open, `0`=closed)     |

Example Prometheus scrape config:

```yaml theme={null}
scrape_configs:
  - job_name: toktra-agent
    static_configs:
      - targets: ['localhost:9090']
```

## osquery extension (fleet-wide queries)

The agent ships a Go osquery table extension (`toktra_extension`) that exposes a `toktra_connections` virtual table. Use this with Fleet or Kolide to run fleet-wide LLM usage queries.

```bash theme={null}
# Run the extension with osquery
osqueryi --extension /usr/lib/toktra/toktra_extension
```

The extension connects to the daemon's Unix socket at `/run/toktra/connections.sock` to read real-time connection data.

Example queries:

```sql theme={null}
-- All active LLM connections
SELECT pid, comm, dst_hostname, provider, bytes_sent
FROM toktra_connections;

-- Connections to OpenAI only
SELECT pid, comm, dst_hostname, bytes_sent, bytes_received
FROM toktra_connections
WHERE provider = 'openai';

-- Processes making LLM calls
SELECT DISTINCT comm, provider
FROM toktra_connections
ORDER BY comm;
```

The `toktra_connections` table schema:

| Column           | Type    | Description                  |
| ---------------- | ------- | ---------------------------- |
| `pid`            | integer | Process ID                   |
| `comm`           | text    | Process name                 |
| `dst_hostname`   | text    | Destination SNI hostname     |
| `dst_ip`         | text    | Destination IP address       |
| `dst_port`       | integer | Destination port             |
| `bytes_sent`     | bigint  | Bytes sent                   |
| `bytes_received` | bigint  | Bytes received               |
| `timestamp`      | bigint  | Unix timestamp (nanoseconds) |
| `provider`       | text    | Identified LLM provider      |
| `category`       | text    | Usage category               |

<Tip>
  Set `TOKTRA_SOCKET_PATH` to override the default Unix socket path if you run the daemon in a non-standard location.
</Tip>