takt

Observability

日本語

TAKT observability is opt-in. When disabled, workflow execution, session logs, provider events, and the existing logging.usage_events output keep their current behavior.

Visualize Locally with OTLP

Start the local observability stack:

docker compose -f docker-compose.observability.yml up -d

Enable TAKT observability in ~/.takt/config.yaml or .takt/config.yaml:

observability:
  enabled: true
  monitor: true
  session_log_exporter: true
  usage_events_phase: true

monitor: true writes a per-run metrics snapshot to .takt/runs/<run>/monitor.json, and session_log_exporter: true writes an OTel-derived shadow session log to .takt/runs/<run>/logs/<session>-otel-session-shadow.jsonl. Both are local files and do not require an OTLP endpoint.

Point the OpenTelemetry HTTP exporters at the local collector and run TAKT:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
takt run

When observability.enabled: true and OTEL_EXPORTER_OTLP_ENDPOINT is set, TAKT sends spans and metrics through OTLP while keeping the local exporters enabled by config. Without OTEL_EXPORTER_OTLP_ENDPOINT, TAKT keeps using only the local exporters and does not send telemetry over the network. When observability.enabled: false, the OpenTelemetry SDK is not initialized even if OTLP environment variables are set.

Open Grafana at http://127.0.0.1:3000 and inspect the takt service. Traces use the existing workflow span tree (workflow.<name> with step.<name> and phase or judge spans — named phase.<step>.<phaseName> and judge_stage.<step>.<stage>.<method> — below it), and metrics are exported alongside the local monitor.json stream.

Exported Metrics

TAKT emits these counters only when observability.enabled: true.

Metric	Main attributes	Emitted when
takt.token.input_tokens	takt.run.id, takt.provider.name, takt.model.name, takt.step.name	A phase or judge_stage.* span ends with provider input and output usage.
takt.token.output_tokens	takt.run.id, takt.provider.name, takt.model.name, takt.step.name	A phase or judge_stage.* span ends with provider input and output usage.
takt.token.cached_input_tokens	takt.run.id, takt.provider.name, takt.model.name, takt.step.name	A phase or judge_stage.* span includes cached input tokens.
takt.token.estimated_cost_usd	takt.run.id, takt.provider.name, takt.model.name, takt.step.name	Provider, model, usage, and a known pricing entry are available. Unknown models and missing model names do not emit this metric.
takt.provider.errors	takt.run.id, takt.provider.name, takt.model.name, takt.provider.error_type	A provider failure is recorded only when providerErrorType(response) classifies it as provider-caused. Unclassified exceptions and error responses that the function does not classify are not included. When response.retryCount is present, retries are counted separately with error_type = retry.
takt.quality_gate.results	takt.run.id, takt.workflow.name, takt.step.name, takt.quality_gate.name, takt.quality_gate.result	A command quality gate passes or fails. Manual/text quality gates are not counted. takt.quality_gate.name uses the sanitized gate.name when set, otherwise (unnamed).
takt.workflow.loops_detected	takt.run.id, takt.workflow.name, takt.step.name	The loop detector warns for the current step.
takt.workflow.cycles_detected	takt.run.id, takt.workflow.name, takt.step.name	A configured loop monitor cycle threshold is reached.

Token counters require input and output token usage. OpenAI long-context cost tiers are selected when input usage is greater than or equal to 270,000 tokens. Token and provider error counters use takt.model.name = "(default)" when the provider default model is used and no model name has been resolved. Cost estimates are best-effort counters and are skipped rather than reported as zero when the model is unknown, the model name is missing, or cache token data is inconsistent.

While a workflow is still running, OpenTelemetry exporters can deliver completed child spans before the long-lived root workflow.<name> span has ended. To make those active traces discoverable in Tempo, TAKT also emits a short-lived workflow_start.<workflowName> span under the root workflow span. This helper span carries the workflow and run attributes, including takt.workflow.status = running, but it does not replace or rename the root, step, phase, or judge spans. It is used only for trace discovery and is not converted into a canonical shadow session log record.

Useful Tempo TraceQL filters for active workflows include:

{ resource.service.name = "takt" && span."takt.workflow.name" = "takt-default" }
{ resource.service.name = "takt" && span."takt.run.id" = "<run-id>" }
{ resource.service.name = "takt" && span."takt.task.pr_number" = 826 }
{ resource.service.name = "takt" && span."takt.task.issue_number" = 792 }
{ resource.service.name = "takt" && span."takt.git.branch" = "takt/816/implement-review-flow" }
{ resource.service.name = "takt" && span."takt.task.summary" =~ ".*review flow.*" }
{ resource.service.name = "takt" && name =~ "workflow_start\\..*" }

After a workflow completes or aborts, TAKT prints a TraceQL discovery: block when observability is enabled. The same discovery data is saved in .takt/runs/<run>/meta.json under observability.traceDiscovery so the run can be found later. The generated queries always include takt.run.id and add filters for available task or git metadata, such as takt.task.pr_number, takt.task.issue_number, and takt.git.branch.

Example CLI output:

TraceQL discovery:
  { resource.service.name = "takt" && span."takt.run.id" = "<run-id>" }
  { resource.service.name = "takt" && span."takt.task.pr_number" = 826 }
  { resource.service.name = "takt" && span."takt.task.issue_number" = 792 }
  { resource.service.name = "takt" && span."takt.git.branch" = "takt/816/implement-review-flow" }

When a workflow aborts or errors, the root workflow.<name> span also records step-level failure attributes:

Attribute	Meaning
takt.failure.kind	Abort category, such as step_error, runtime_error, or iteration_limit.
takt.failure.step	Deepest failing step preserved across nested workflows.
takt.failure.reason	Sanitized abort reason.

The base endpoint is required for OTLP export:

Environment variable	Purpose
OTEL_EXPORTER_OTLP_ENDPOINT	Required opt-in endpoint. TAKT derives /v1/traces and /v1/metrics from it.
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT	Optional absolute HTTP(S) trace endpoint override. Used only when OTEL_EXPORTER_OTLP_ENDPOINT is also set.
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT	Optional absolute HTTP(S) metric endpoint override. Used only when OTEL_EXPORTER_OTLP_ENDPOINT is also set.

TAKT explicitly resolves and validates only these three endpoint variables. The other standard OTEL_EXPORTER_OTLP_* variables — such as _HEADERS, _TIMEOUT, and _COMPRESSION — are not interpreted by TAKT and reach the OpenTelemetry SDK unchanged. Child processes are handled more strictly: credential-bearing variables (_HEADERS, client certificate, and client key) are excluded from the environment TAKT passes to them, while non-sensitive variables pass through.

Endpoint values used for OTLP export must be absolute http or https URLs. A trace or metric endpoint without OTEL_EXPORTER_OTLP_ENDPOINT does not opt in to OTLP export; TAKT keeps the local-only exporter set. When the base endpoint is set, any configured trace or metric override is validated before the run starts. Export delivery failures after startup, such as a stopped local collector, do not block the workflow run.

Override Observability via Environment Variables

Each observability config flag can be overridden per process without editing config files:

Environment variable	Overrides
TAKT_OBSERVABILITY_ENABLED	observability.enabled
TAKT_OBSERVABILITY_MONITOR	observability.monitor
TAKT_OBSERVABILITY_SESSION_LOG_EXPORTER	observability.session_log_exporter
TAKT_OBSERVABILITY_USAGE_EVENTS_PHASE	observability.usage_events_phase

Each variable accepts true or false and takes priority over the values in ~/.takt/config.yaml and .takt/config.yaml.

Nested takt runs launched from a command gate inherit the observability settings and OTLP endpoints through these environment variables automatically; credential-bearing exporter variables (_HEADERS, client certificate, and client key) are excluded from that propagation.

Enable Phase Usage Events

Add this to ~/.takt/config.yaml or .takt/config.yaml:

observability:
  enabled: true
  usage_events_phase: true

This writes phase-level usage events to:

.takt/runs/<run>/logs/<session>-usage-events.phase.jsonl

The phase usage stream is separate from the existing logging.usage_events file. It does not replace logs/<session>-usage-events.jsonl.

Event Granularity

Records are grouped by workflow phase:

Phase	Meaning
phase1_execute	Main step execution
phase2_report	Output contract/report generation
phase3_structured	Structured status judgment
phase3_tag	Tag fallback status judgment
phase3_fallback	AI judge fallback status judgment

Missing usage is recorded with usage_missing: true and a reason. Missing usage is not treated as zero tokens by the analysis command.

Each record also includes the step’s persona and tags when defined. persona is a string, and tags is an array of strings. Missing values are omitted.

Analyze Usage

Build the project first:

npm run build

Then aggregate one or more files or run directories:

npm run analyze:usage -- .takt/runs/<run>/logs/*-usage-events.phase.jsonl
npm run analyze:usage -- .takt/runs/<run>

The default output is a Markdown table grouped by step x phase x provider x model.

Use CSV output for spreadsheets or downstream scripts:

npm run analyze:usage -- --format csv .takt/runs/<run> > usage.csv

The output columns are:

Column	Meaning
step / phase / provider / model	Aggregation key
runs	Unique run_id count
calls	Number of phase usage records
missing	Records with unavailable usage
input_tokens / output_tokens / total_tokens	Token totals for records with usage
cached_input_tokens / cache_creation_input_tokens / cache_read_input_tokens	Cache-related token totals
avg_total_tokens / median_total_tokens / stddev_total_tokens	Per-call total token statistics, excluding missing usage

For before/after comparisons, run the command separately for each set of run directories and compare the resulting tables or CSV files.

Summarize token usage across runs

tools/token-usage.sh aggregates token usage across all worktree and local runs, broken down by task and step.

./tools/token-usage.sh              # latest 10 runs (excludes mock)
./tools/token-usage.sh --top 20     # latest 20 runs
./tools/token-usage.sh --csv        # CSV output
./tools/token-usage.sh --all        # include mock/zero-token runs
./tools/token-usage.sh /path/to/dir # scan a specific directory

By default it scans both ../takt-worktrees/ and .takt/runs/. Requires observability.usage_events_phase: true in config.

CSV output includes persona and tags columns after step. Tags are joined with |, and records with different persona or tags are kept in separate rows.

Dependencies: node, jq

This site is open source. Improve this page.