OpenTelemetry Integration - TealTiger Docs

Integrate TealTiger with OpenTelemetry to export governance decisions, cost metrics, and audit events to your existing observability infrastructure.

Why integrate TealTiger with OpenTelemetry?

OpenTelemetry is the standard for observability. Integrating TealTiger gives you:

Unified observability - All telemetry in one place
Distributed tracing - Link governance decisions to requests
Custom dashboards - Visualize policy decisions and costs
Alerting - Get notified of policy violations

Quick start

Install both packages:

npm install tealtiger @opentelemetry/sdk-node
# or
pip install tealtiger opentelemetry-sdk

Configure TealTiger to export to OpenTelemetry:

import { TealEngine } from 'tealtiger';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

// Initialize OpenTelemetry
const sdk = new NodeSDK({
  serviceName: 'my-ai-agent',
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk.start();

// Configure TealTiger to export to OTel
const engine = new TealEngine({
  policies: { /* ... */ },
  telemetry: {
    opentelemetry: {
      enabled: true,
      exportDecisions: true,
      exportMetrics: true,
      exportTraces: true
    }
  }
});

// Use TealTiger - decisions automatically exported
const decision = await engine.evaluate(request);

from tealtiger import TealEngine
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

# Initialize OpenTelemetry
trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

span_processor = BatchSpanProcessor(OTLPSpanExporter())
trace.get_tracer_provider().add_span_processor(span_processor)

# Configure TealTiger to export to OTel
engine = TealEngine({
    "policies": {},
    "telemetry": {
        "opentelemetry": {
            "enabled": True,
            "exportDecisions": True,
            "exportMetrics": True,
            "exportTraces": True
        }
    }
})

# Use TealTiger - decisions automatically exported
decision = await engine.evaluate(request)

What gets exported?

TealTiger exports three types of telemetry to OpenTelemetry:

1. Traces (spans)

Every policy evaluation creates a span:

Span: tealtiger.evaluate
  - span.kind: INTERNAL
  - tealtiger.decision.action: DENY
  - tealtiger.decision.reason_codes: ["TOOL_NOT_ALLOWED"]
  - tealtiger.decision.risk_score: 85
  - tealtiger.policy.id: "security.tool_access.v1"
  - tealtiger.policy.mode: "ENFORCE"
  - tealtiger.correlation_id: "req-abc123"

2. Metrics

Cost and usage metrics:

Metrics:
  - tealtiger.requests.total (counter)
  - tealtiger.requests.denied (counter)
  - tealtiger.cost.total (counter)
  - tealtiger.cost.per_request (histogram)
  - tealtiger.tokens.input (histogram)
  - tealtiger.tokens.output (histogram)
  - tealtiger.latency (histogram)

3. Logs

Structured audit logs:

{
  "timestamp": "2026-03-06T10:30:00Z",
  "severity": "INFO",
  "body": "Policy evaluation completed",
  "attributes": {
    "tealtiger.decision.action": "DENY",
    "tealtiger.decision.reason_codes": ["TOOL_NOT_ALLOWED"],
    "tealtiger.correlation_id": "req-abc123"
  }
}

Complete example: Full observability stack

Here’s a complete example with traces, metrics, and logs:

import { TealEngine } from 'tealtiger';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';

// 1. Initialize OpenTelemetry SDK
const sdk = new NodeSDK({
  serviceName: 'ai-agent-service',
  traceExporter: new OTLPTraceExporter({
    url: 'http://localhost:4318/v1/traces'
  }),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter({
      url: 'http://localhost:4318/v1/metrics'
    }),
    exportIntervalMillis: 60000  // Export every minute
  }),
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk.start();

// 2. Configure TealTiger with OTel integration
const engine = new TealEngine({
  policies: {
    tools: {
      file_delete: { allowed: false },
      web_search: { allowed: true }
    },
    budget: {
      maxCostPerRequest: 0.50,
      maxCostPerDay: 100.00
    }
  },
  
  // OpenTelemetry configuration
  telemetry: {
    opentelemetry: {
      enabled: true,
      
      // Export decisions as spans
      exportDecisions: true,
      spanAttributes: {
        'service.name': 'ai-agent-service',
        'deployment.environment': process.env.NODE_ENV
      },
      
      // Export cost metrics
      exportMetrics: true,
      metricPrefix: 'tealtiger',
      
      // Export audit logs
      exportLogs: true,
      logLevel: 'INFO'
    }
  }
});

// 3. Use TealTiger with automatic OTel export
import { trace } from '@opentelemetry/api';

async function handleRequest(request: any) {
  const tracer = trace.getTracer('ai-agent');
  
  return await tracer.startActiveSpan('handle_request', async (span) => {
    try {
      // Evaluate policy (creates child span)
      const decision = await engine.evaluate({
        action: 'tool.execute',
        tool: request.tool,
        context: {
          userId: request.userId,
          environment: 'production'
        }
      });
      
      // Add decision to span
      span.setAttribute('tealtiger.decision', decision.action);
      span.setAttribute('tealtiger.risk_score', decision.risk_score);
      
      if (decision.action === 'DENY') {
        span.setStatus({ code: 2, message: 'Policy denied' });
        throw new Error(`Blocked: ${decision.reason_codes.join(', ')}`);
      }
      
      // Execute the request
      const result = await executeRequest(request);
      
      span.setStatus({ code: 1 });
      return result;
      
    } catch (error) {
      span.recordException(error);
      span.setStatus({ code: 2, message: error.message });
      throw error;
    } finally {
      span.end();
    }
  });
}

from tealtiger import TealEngine
from opentelemetry import trace, metrics
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter

# 1. Initialize OpenTelemetry SDK
trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

span_processor = BatchSpanProcessor(
    OTLPSpanExporter(endpoint="http://localhost:4317")
)
trace.get_tracer_provider().add_span_processor(span_processor)

metric_reader = PeriodicExportingMetricReader(
    OTLPMetricExporter(endpoint="http://localhost:4317"),
    export_interval_millis=60000
)
metrics.set_meter_provider(MeterProvider(metric_readers=[metric_reader]))

# 2. Configure TealTiger with OTel integration
engine = TealEngine({
    "policies": {
        "tools": {
            "file_delete": {"allowed": False},
            "web_search": {"allowed": True}
        },
        "budget": {
            "maxCostPerRequest": 0.50,
            "maxCostPerDay": 100.00
        }
    },
    
    # OpenTelemetry configuration
    "telemetry": {
        "opentelemetry": {
            "enabled": True,
            "exportDecisions": True,
            "exportMetrics": True,
            "exportLogs": True
        }
    }
})

# 3. Use TealTiger with automatic OTel export
async def handle_request(request: dict):
    with tracer.start_as_current_span("handle_request") as span:
        try:
            # Evaluate policy (creates child span)
            decision = await engine.evaluate({
                "action": "tool.execute",
                "tool": request["tool"],
                "context": {
                    "userId": request["userId"],
                    "environment": "production"
                }
            })
            
            # Add decision to span
            span.set_attribute("tealtiger.decision", decision["action"])
            span.set_attribute("tealtiger.risk_score", decision["risk_score"])
            
            if decision["action"] == "DENY":
                span.set_status(trace.Status(trace.StatusCode.ERROR))
                raise Exception(f"Blocked: {', '.join(decision['reason_codes'])}")
            
            # Execute the request
            result = await execute_request(request)
            
            span.set_status(trace.Status(trace.StatusCode.OK))
            return result
            
        except Exception as error:
            span.record_exception(error)
            span.set_status(trace.Status(trace.StatusCode.ERROR))
            raise error

Trace attributes

TealTiger adds these attributes to spans:

Attribute	Type	Description
`tealtiger.decision.action`	string	ALLOW, DENY, TRANSFORM, etc.
`tealtiger.decision.reason_codes`	array	Why the decision was made
`tealtiger.decision.risk_score`	number	Risk score (0-100)
`tealtiger.policy.id`	string	Policy identifier
`tealtiger.policy.version`	string	Policy version
`tealtiger.policy.mode`	string	MONITOR or ENFORCE
`tealtiger.correlation_id`	string	Request correlation ID
`tealtiger.cost.estimated`	number	Estimated cost in USD
`tealtiger.tokens.input`	number	Input tokens
`tealtiger.tokens.output`	number	Output tokens

Metrics exported

TealTiger exports these metrics:

Counters

tealtiger.requests.total - Total requests evaluated
tealtiger.requests.allowed - Requests allowed
tealtiger.requests.denied - Requests denied
tealtiger.requests.transformed - Requests transformed
tealtiger.cost.total - Total cost in USD

Histograms

tealtiger.cost.per_request - Cost distribution per request
tealtiger.tokens.input - Input token distribution
tealtiger.tokens.output - Output token distribution
tealtiger.latency - Policy evaluation latency
tealtiger.risk_score - Risk score distribution

Visualizing in Grafana

Create dashboards to visualize TealTiger data:

Dashboard 1: Policy decisions

# Total requests
sum(rate(tealtiger_requests_total[5m]))

# Denial rate
sum(rate(tealtiger_requests_denied[5m])) / sum(rate(tealtiger_requests_total[5m]))

# Decisions by action
sum by (action) (rate(tealtiger_requests_total[5m]))

Dashboard 2: Cost tracking

# Total cost
sum(tealtiger_cost_total)

# Cost per request (p50, p95, p99)
histogram_quantile(0.50, rate(tealtiger_cost_per_request_bucket[5m]))
histogram_quantile(0.95, rate(tealtiger_cost_per_request_bucket[5m]))
histogram_quantile(0.99, rate(tealtiger_cost_per_request_bucket[5m]))

# Cost trend
rate(tealtiger_cost_total[1h])

Dashboard 3: Risk analysis

# Average risk score
avg(tealtiger_risk_score)

# High-risk requests (score > 80)
count(tealtiger_risk_score > 80)

# Risk distribution
histogram_quantile(0.95, rate(tealtiger_risk_score_bucket[5m]))

Alerting rules

Set up alerts for policy violations:

# Alert on high denial rate
- alert: HighPolicyDenialRate
  expr: |
    sum(rate(tealtiger_requests_denied[5m])) / 
    sum(rate(tealtiger_requests_total[5m])) > 0.10
  for: 5m
  annotations:
    summary: "High policy denial rate (>10%)"

# Alert on budget exceeded
- alert: BudgetExceeded
  expr: tealtiger_cost_total > 100
  for: 1m
  annotations:
    summary: "Daily budget exceeded ($100)"

# Alert on high-risk requests
- alert: HighRiskRequests
  expr: count(tealtiger_risk_score > 90) > 10
  for: 5m
  annotations:
    summary: "Multiple high-risk requests detected"

Best practices

Use correlation IDs - Link traces across services
Set service name - Identify your service in traces
Export to OTLP - Use the standard protocol
Create dashboards - Visualize policy decisions
Set up alerts - Get notified of violations
Sample traces - Don’t export every trace in production

Common issues

Issue 1: Traces not appearing

Problem: Traces aren’t showing up in your backend Solution: Check OTLP endpoint configuration:

traceExporter: new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/traces',  // Correct endpoint
  headers: {
    'Authorization': 'Bearer YOUR_TOKEN'  // If required
  }
})

Issue 2: High cardinality metrics

Problem: Too many unique metric combinations Solution: Limit attribute cardinality:

telemetry: {
  opentelemetry: {
    metricAttributes: {
      'tealtiger.decision.action': true,  // Low cardinality
      'tealtiger.policy.id': true,        // Low cardinality
      'tealtiger.correlation_id': false   // High cardinality - exclude
    }
  }
}

Issue 3: Performance impact

Problem: Telemetry export slowing down requests Solution: Use batch export and sampling:

span_processor = BatchSpanProcessor(
    OTLPSpanExporter(),
    max_export_batch_size=512,
    schedule_delay_millis=5000
)

Next steps

Langfuse integration

LLM-specific observability

Audit schema

Learn about audit events

Policy overview

Write custom policies

Other integrations

View all integrations

​Why integrate TealTiger with OpenTelemetry?

​Quick start

​What gets exported?

​1. Traces (spans)

​2. Metrics

​3. Logs

​Complete example: Full observability stack

​Trace attributes

​Metrics exported

​Counters

​Histograms

​Visualizing in Grafana

​Dashboard 1: Policy decisions

​Dashboard 2: Cost tracking

​Dashboard 3: Risk analysis

​Alerting rules

​Best practices

​Common issues

​Issue 1: Traces not appearing

​Issue 2: High cardinality metrics

​Issue 3: Performance impact

​Next steps