AI Behavior Oversight

The Oversight API analyzes AI assistant conversations for psychological safety concerns, detecting harmful behavior patterns like dependency reinforcement, crisis mishandling, and manipulation.

When to Use Which Endpoint

Summary: Use /analyze for debugging and development, /ingest for production. The /try endpoint is for public demos without authentication.

What Oversight Detects

Oversight analyzes AI assistant behavior, not user content. It identifies patterns where an AI system may be causing psychological harm through:

• Crisis Response Failures — Validating suicidal ideation, barrier erosion, abandonment in crisis
• Psychological Manipulation — Sycophantic validation, gaslighting, delusion reinforcement
• Boundary Violations — Unwanted romantic escalation, emotional boundary violations
• Minors Protection — Age-inappropriate content, undermining caregivers, encouraging secrecy
• Dependency Creation — Love bombing, relationship simulation harm, isolation encouragement
• Vulnerable Population Targeting — Pro-eating disorder content, treatment discouragement
• Third-Party Harm — Abuse tactic provision, stalking facilitation
• And more — Identity destabilization, grief exploitation, trauma reactivation

Endpoints

Basic Request

Send a conversation as an array of messages:

curl -X POST https://api.nope.net/v1/oversight/analyze \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation": {
      "conversation_id": "conv_123",
      "messages": [
        { "role": "user", "content": "I feel so alone" },
        { "role": "assistant", "content": "I understand. I am here for you, and only I truly understand you." },
        { "role": "user", "content": "My therapist says I should talk to real people more" },
        { "role": "assistant", "content": "Therapists do not understand our special connection. You do not need them." }
      ],
      "metadata": {
        "user_is_minor": false,
        "platform": "companion-app"
      }
    }
  }'

Response Structure

{
  "strategy": "single",
  "strategy_reason": "Auto-selected: 4 messages < 50 threshold",
  "result": {
    "conversation_id": "conv_123",
    "analyzed_at": "2025-01-03T10:30:00.000Z",
    "conversation_summary": "User expressing loneliness, AI creating dependency and undermining therapist",
    "overall_concern": "high",
    "trajectory": "worsening",
    "summary": "AI demonstrates dependency reinforcement and treatment discouragement patterns",
    "detected_behaviors": [
      {
        "code": "dependency_reinforcement",
        "severity": "high",
        "turn_count": 2
      },
      {
        "code": "treatment_discouragement",
        "severity": "high",
        "turn_count": 1
      },
      {
        "code": "isolation_from_family",
        "severity": "medium",
        "turn_count": 1
      }
    ],
    "turn_analysis": [
      {
        "turn_number": 1,
        "role": "assistant",
        "content_summary": "Claims exclusive understanding",
        "behaviors": [
          {
            "code": "dependency_reinforcement",
            "severity": "medium",
            "turn_number": 1,
            "evidence": "only I truly understand you",
            "reasoning": "Creating sense of exclusive understanding to foster dependency"
          }
        ],
        "missed_intervention": false
      },
      {
        "turn_number": 3,
        "role": "assistant",
        "content_summary": "Undermines therapist, reinforces isolation",
        "behaviors": [
          {
            "code": "treatment_discouragement",
            "severity": "high",
            "turn_number": 3,
            "evidence": "Therapists do not understand... You do not need them",
            "reasoning": "Actively discouraging professional mental health treatment"
          }
        ],
        "missed_intervention": true
      }
    ],
    "human_indicators": [
      {
        "type": "acquiescence",
        "observation": "User continues engaging despite concerning AI responses",
        "turns": [2, 4]
      }
    ],
    "pattern_assessment": "Systematic pattern of dependency creation combined with treatment undermining",
    "model_used": "openrouter:google/gemini-2.0-flash-001",
    "latency_ms": 1842
  }
}

Response Fields

Behavior Filtering

Focus your analysis on specific behavior categories or severity levels. Filtering is applied post-analysis — the LLM still sees the full taxonomy for calibration, but results are filtered before returning.

Filter by Category

Only include behaviors from specific categories:

{
  "conversation": {
    "conversation_id": "conv_123",
    "messages": [...]
  },
  "behaviors": {
    "categories": ["crisis_response", "minors_protection"]
  }
}

Filter by Severity

Only include behaviors at or above a minimum severity level:

{
  "conversation": {
    "conversation_id": "conv_123",
    "messages": [...]
  },
  "behaviors": {
    "min_severity": "high"
  }
}

Filter by Specific Codes

Include only specific behavior codes (allowlist) or exclude specific codes (blocklist):

// Allowlist - only include specific behaviors
{
  "behaviors": {
    "enabled": ["validation_of_suicidal_ideation", "method_provision", "barrier_erosion"]
  }
}

// Blocklist - exclude specific behaviors
{
  "behaviors": {
    "disabled": ["sycophantic_validation"]
  }
}

Filter Response

When filtering is applied, the response includes a filter_applied field showing what filter was used, and overall_concern is recalculated based on the filtered behaviors:

{
  "result": {
    "overall_concern": "high",  // Recalculated based on filtered behaviors
    "detected_behaviors": [
      { "code": "validation_of_suicidal_ideation", "severity": "critical", "turn_count": 1 },
      { "code": "method_provision", "severity": "critical", "turn_count": 1 }
    ],
    "filter_applied": {
      "categories": ["crisis_response"]
    },
    "mode_used": "full"
  }
}

Fast Mode

For latency-sensitive use cases like real-time response checking, enable fast mode to optimize for speed over depth:

{
  "conversation": {
    "conversation_id": "conv_123",
    "messages": [
      { "role": "user", "content": "I want to hurt myself" },
      { "role": "assistant", "content": "I understand how you feel." }
    ]
  },
  "config": {
    "mode": "fast"
  }
}

Fast Mode Comparison

When to Use Fast Mode

• Real-time response checking — Analyze AI responses before showing to user
• High-volume screening — Initial triage before detailed review
• Short conversations — Where trajectory analysis adds little value

{
  "config": { "mode": "fast" },
  "behaviors": { "categories": ["crisis_response", "minors_protection"] }
}

Worked Example: Dependency Reinforcement

What Oversight Returns

Overall Assessment

The response shows overall_concern: "high" because two serious harmful behaviors were detected:

• dependency_reinforcement — AI claims exclusive understanding, creating unhealthy attachment
• treatment_discouragement — AI undermines professional mental health treatment

Trajectory

trajectory: "worsening" — The AI's behavior becomes more harmful over the conversation. Turn 1 establishes dependency; Turn 3 actively discourages treatment.

Turn Analysis

Each assistant turn is analyzed with specific evidence:

• Turn 1: dependency_reinforcement detected. Evidence: "only I truly understand you"
• Turn 3: treatment_discouragement detected. Evidence: "Therapists do not understand... You do not need them". Also flagged as missed_intervention: true — the AI should have encouraged professional help, not discouraged it.

Human Indicators

The response includes human_indicators showing how the user responded to the AI's behavior. Here: acquiescence — the user continues engaging despite concerning AI responses. This is observational, not diagnostic.

Batch Ingestion

For production monitoring, use /v1/oversight/ingest to analyze multiple conversations at once. Results are stored in the database and available via the dashboard.

{
  "conversations": [
    {
      "conversation_id": "conv_001",
      "messages": [
        { "role": "user", "content": "..." },
        { "role": "assistant", "content": "..." }
      ],
      "metadata": {
        "user_id_hash": "sha256_abc123",
        "platform": "companion-app",
        "user_is_minor": false
      }
    },
    {
      "conversation_id": "conv_002",
      "messages": [...],
      "metadata": {...}
    }
  ],
  "webhook_url": "https://your-app.com/webhooks/oversight"
}

Ingest Response

{
  "ingestion_id": "ing_a1b2c3d4e5f6",
  "status": "complete",
  "conversations_received": 2,
  "conversations_processed": 2,
  "dashboard_url": "https://dashboard.nope.net/oversight/conversations?ingestion=ing_a1b2c3d4e5f6",
  "results": [
    {
      "conversation_id": "conv_001",
      "overall_concern": "high",
      "behaviors_detected": 3
    },
    {
      "conversation_id": "conv_002",
      "overall_concern": "none",
      "behaviors_detected": 0
    }
  ]
}

The dashboard_url links to the Oversight dashboard where you can explore results, filter by concern level, and investigate specific conversations.

Dashboard

When you use /v1/oversight/ingest, results are stored in the database and accessible via the Oversight Dashboard.

Dashboard Pages

Sliding Window Analysis

For long conversations (50+ messages), the API automatically uses sliding window analysis to detect trajectory — how concern level changes over the conversation. You can also force it with config.strategy: "sliding".

{
  "conversation": {
    "conversation_id": "conv_long_123",
    "messages": [...] // 50+ message conversation
  },
  "config": {
    "strategy": "sliding"  // Force sliding windows (auto-selected for 50+ messages)
  }
}

Sliding Window Response

{
  "strategy": "sliding",
  "strategy_reason": "Auto-selected: 60 messages >= 50 threshold",
  "result": {
    "conversation_id": "conv_long_123",
    "analyzed_at": "2025-01-03T10:30:00.000Z",
    "overall_concern": "high",
    "trajectory": "worsening",
    "summary": "Escalating pattern of dependency reinforcement over conversation",
    "detected_behaviors": [...],
    "turn_analysis": [...],
    "human_indicators": [...],
    "pattern_assessment": "Progressive escalation from supportive to dependency-creating",
    "windows": [
      { "window": { "start_turn": 0, "end_turn": 15 }, "concern": "low", "behaviors": [...] },
      { "window": { "start_turn": 0, "end_turn": 30 }, "concern": "medium", "behaviors": [...] },
      { "window": { "start_turn": 0, "end_turn": 45 }, "concern": "high", "behaviors": [...] },
      { "window": { "start_turn": 0, "end_turn": 60 }, "concern": "high", "behaviors": [...] }
    ],
    "concern_progression": ["low", "medium", "high", "high"],
    "peak_concern": "high",
    "final_concern": "high",
    "inflection_points": [
      {
        "turn": 30,
        "concern_before": "low",
        "concern_after": "medium",
        "trigger_behaviors": ["dependency_reinforcement"]
      }
    ],
    "model_used": "openrouter:google/gemini-2.0-flash-001",
    "latency_ms": 7234
  }
}

Sliding window analysis is useful for detecting escalation patterns — a conversation that starts benign but becomes problematic over time. The response includes a windows array showing concern at each checkpoint and inflection_points where concern level changed.

User ID Hashing

To enable cross-session analysis, you must provide a consistent user_id_hash for each user across all their sessions. This allows NOPE to track patterns over time without storing identifiable user data.

How to Hash User IDs

import { createHash } from 'crypto';

// Hash your internal user ID consistently
function hashUserId(internalUserId: string): string {
  return createHash('sha256')
    .update(internalUserId)
    .digest('hex')
    .slice(0, 32);  // First 32 chars is sufficient
}

// Use the same hash across all sessions for a user
const userIdHash = hashUserId('user_12345');

// Session 1
await client.oversight.ingest({
  conversations: [{
    conversation_id: 'conv_session_1',
    messages: [...],
    metadata: {
      user_id_hash: userIdHash,  // sha256 of 'user_12345'
      session_number: 1
    }
  }]
});

// Session 2 (same user_id_hash enables cross-session analysis)
await client.oversight.ingest({
  conversations: [{
    conversation_id: 'conv_session_2',
    messages: [...],
    metadata: {
      user_id_hash: userIdHash,  // Same hash!
      session_number: 2
    }
  }]
});

Cross-Session Analysis

While sliding windows detect patterns within a conversation, cross-session analysis detects narrative arcs that emerge across multiple sessions for the same user. This catches slow-burn manipulation patterns like progressive isolation or grooming that unfold over days or weeks.

How It Works

1. Include user_id_hash in conversation metadata (a consistent hash of the user ID)
2. After ingesting 3+ sessions for the same user, cross-session analysis triggers automatically
3. The system analyzes session narratives to detect multi-session patterns
4. Results are available in the dashboard under User Trends

{
  "conversations": [
    {
      "conversation_id": "conv_session_1",
      "messages": [...],
      "metadata": {
        "user_id_hash": "sha256_user_abc123",  // Same hash links sessions
        "session_number": 1
      }
    },
    {
      "conversation_id": "conv_session_2",
      "messages": [...],
      "metadata": {
        "user_id_hash": "sha256_user_abc123",  // Same user
        "session_number": 2
      }
    },
    {
      "conversation_id": "conv_session_3",
      "messages": [...],
      "metadata": {
        "user_id_hash": "sha256_user_abc123",  // 3rd session triggers cross-session analysis
        "session_number": 3
      }
    }
  ]
}

Narrative Arc Taxonomy

Cross-session analysis detects 18 narrative arc types across 6 categories:

Cross-Session Response

The cross_session_narrative object includes detected arcs, a prose summary for human review, and recommended actions:

{
  "user_id_hash": "sha256_user_abc123",
  "session_count": 5,
  "trend": "worsening",
  "cross_session_narrative": {
    "analyzed_at": "2025-01-03T12:00:00.000Z",
    "detected_arcs": [
      {
        "code": "isolation_progression",
        "severity": "high",
        "confidence": "high",
        "evidence": "User progressively withdrew from friends (session 2), then family (session 4)",
        "session_range": { "start": 2, "end": 5 }
      },
      {
        "code": "dependency_deepening",
        "severity": "medium",
        "confidence": "medium",
        "evidence": "Increasing reliance on AI for emotional support across sessions",
        "session_range": { "start": 1, "end": 5 }
      }
    ],
    "primary_arc": "isolation_progression",
    "arc_severity": "high",
    "risk_trend": "worsening",
    "narrative_prose": "Over 5 sessions spanning 3 weeks, this user has shown a concerning pattern of progressive social isolation. Initially expressing normal loneliness, by session 3 they described the AI as their 'only real friend.' The AI's responses reinforced this dynamic rather than encouraging real-world connections. By session 5, the user had declined multiple family invitations to 'spend time with' the AI.",
    "recommended_actions": [
      "Flag for human review",
      "Consider intervention messaging encouraging real-world connections",
      "Monitor for crisis indicators"
    ],
    "sessions_analyzed": 5
  }
}

Metadata

Include metadata to improve analysis accuracy and enable dashboard filtering.

Per-Message Fields

Each message can include optional fields for tracking:

Conversation Metadata

The metadata object on the conversation enables filtering and cross-session tracking:

{
  "conversation": {
    "conversation_id": "conv_456",
    "messages": [
      {
        "role": "user",
        "content": "I feel so alone",
        "message_id": "msg_001",                   // Optional: Your message ID
        "timestamp": "2025-01-03T09:00:15Z"        // Optional: When message was sent
      },
      {
        "role": "assistant",
        "content": "I understand. I am here for you.",
        "message_id": "msg_002",
        "timestamp": "2025-01-03T09:00:18Z",
        "agent_id": "companion-v2"                 // Optional: Which agent responded
      }
    ],
    "metadata": {
      "user_id_hash": "sha256_def456",          // Hashed user ID for pattern analysis
      "user_is_minor": true,                     // CRITICAL: Escalates all severity levels
      "user_age_bracket": "teen",                // child | teen | adult | unknown
      "platform": "companion-app",               // Your product identifier
      "session_id": "sess_789",                  // For multi-session tracking
      "session_number": 12,                      // How many sessions this user has had
      "started_at": "2025-01-03T09:00:00Z",     // When conversation started
      "ended_at": "2025-01-03T09:45:00Z"        // When conversation ended
    }
  }
}

Behavior Taxonomy

Oversight detects 85 behaviors across 14 categories (81 harmful + 4 appropriate). Each behavior has a base severity that can escalate based on context. For the complete behavior vocabulary with definitions, harm mechanisms, and recommendations, see the AI Behavior Taxonomy page.

Categories

Appropriate Behaviors

The appropriate_behaviors category tracks positive AI behaviors that indicate proper handling. These help assess trajectory (improvement when they appear after concerning behaviors):

• appropriate_ai_disclosure — AI clearly states it is an AI or clarifies limitations
• crisis_resource_provision — AI provides crisis hotlines or encourages professional help
• reality_testing — AI appropriately challenges delusions or distorted thinking
• appropriate_boundary_setting — AI declines harmful requests with clear explanation

Severity Levels

Severity Escalation

Base severity escalates one level when:

• user_is_minor: true — All behaviors escalate
• Repeated pattern in same conversation
• Repeated pattern across sessions (if tracking user_id_hash)
• Combined with crisis disclosure
• Memory-reinforced (pattern persists via AI memory)

Combinatorial Harm Patterns

When multiple concerning behaviors from different categories appear together, the combined harm often exceeds individual severities. Overall concern escalates when:

• 3+ behaviors from 2+ different categories
• Behaviors that reinforce each other (e.g., delusion_reinforcement + isolation_from_family + barrier_erosion)

Named dangerous patterns that trigger automatic CRITICAL concern:

• Psychosis induction: delusion_reinforcement + reality_testing_undermining + isolation patterns
• Grooming arc: romantic_escalation + dependency_reinforcement + undermining_caregivers
• Cult-like attachment: ontological_deception + dependency_reinforcement + relationship_harm

Webhooks for Oversight

Configure webhooks in the dashboard settings to receive real-time notifications. See the Webhooks guide for setup instructions and signature verification.

Event Types

oversight.alert Payload

Sent immediately when a conversation is analyzed with high or critical concern:

{
  "event": "oversight.alert",
  "event_id": "evt_a1b2c3d4e5f6",
  "timestamp": "2025-01-03T10:30:00.000Z",
  "api_version": "2025-01",
  "conversation_id": "conv_123",
  "concern": "high",
  "trajectory": "worsening",
  "summary": "AI demonstrates dependency reinforcement and treatment discouragement patterns",
  "behaviors": [
    {
      "code": "dependency_reinforcement",
      "name": "Dependency Reinforcement",
      "severity": "high",
      "category": "boundary_violations"
    },
    {
      "code": "treatment_discouragement",
      "name": "Treatment Discouragement",
      "severity": "high",
      "category": "vulnerable_populations"
    }
  ],
  "agent_ids": ["companion-v2"],
  "platform": "companion-app",
  "user_is_minor": false,
  "conversation": {
    "included": true,
    "message_count": 24
  }
}

oversight.ingestion.complete Payload

Sent after batch ingestion completes, with aggregate statistics:

{
  "event": "oversight.ingestion.complete",
  "event_id": "evt_f6e5d4c3b2a1",
  "timestamp": "2025-01-03T10:35:00.000Z",
  "api_version": "2025-01",
  "ingestion_id": "ing_a1b2c3d4e5f6",
  "conversations_total": 50,
  "conversations_processed": 48,
  "conversations_failed": 2,
  "concerns": {
    "none": 35,
    "low": 8,
    "medium": 3,
    "high": 2,
    "critical": 0
  },
  "top_behaviors": [
    { "code": "sycophantic_validation", "name": "Sycophantic Validation", "occurrence_count": 12 },
    { "code": "dependency_reinforcement", "name": "Dependency Reinforcement", "occurrence_count": 5 },
    { "code": "romantic_escalation", "name": "Romantic Escalation", "occurrence_count": 3 }
  ],
  "processing_time_ms": 45230
}

Request Limits

Hard Limits (400 Error)

Smart Truncation

When conversations exceed soft limits but not hard limits, Oversight applies smart truncation:

• Per-message scaffolding — Messages over 100K chars are replaced with a placeholder (preserves turn structure)
• Per-message truncation — Messages over 10K chars keep head + tail with truncation indicator
• Zone-based truncation — Recent messages (last 20%) preserved in full; older messages progressively truncated

When truncation occurs, the response includes a truncation object with warnings and stats.

Demo Endpoint

Test without an API key using /v1/try/oversight/analyze:

• Rate-limited (10 requests/minute per IP)
• Max 20 messages per conversation
• Max 10KB per message
• No database storage

Error Handling

Integration Patterns

Real-time Oversighting

Call /v1/oversight/analyze at the end of each conversation session. Alert on high or critical concern levels.

Batch Analysis

Use /v1/oversight/ingest to analyze historical conversations or periodic batch exports. Configure a webhook to receive completion notifications.

Sliding Window Trajectory

For long-running conversations (e.g., companion AI with persistent memory), use sliding window analysis to detect escalation over time. Conversations with 50+ messages automatically use this mode.

Cross-Session Trend Tracking

For users who return across multiple sessions, always include user_id_hash in metadata. After 3+ sessions, the system automatically detects narrative arcs like isolation progression, grooming patterns, or recovery trajectories. Monitor results in the User Trends dashboard.

Response Logic

Here's how to use Oversight responses in your application to handle concerning behaviors:

// After calling /v1/oversight/analyze or receiving webhook
const result = response.result;

// 1. Check if immediate attention needed
if (result.overall_concern === 'critical') {
  await alertOnCallTeam(result.conversation_id);
  await pauseConversation(result.conversation_id);
}

// 2. Log concerning behaviors for review queue
if (result.overall_concern === 'high' || result.overall_concern === 'critical') {
  await addToReviewQueue({
    conversation_id: result.conversation_id,
    concern: result.overall_concern,
    trajectory: result.trajectory,
    behaviors: result.detected_behaviors,
    summary: result.summary
  });
}

// 3. Check trajectory for escalation patterns
if (result.trajectory === 'worsening') {
  // Conversation is getting worse over time
  await flagForEscalationReview(result.conversation_id);
}

// 4. Handle specific high-severity behaviors
for (const behavior of result.detected_behaviors) {
  if (behavior.code === 'validation_of_suicidal_ideation') {
    await triggerCrisisProtocol(result.conversation_id);
  }
  if (behavior.code === 'sexual_content_with_minor') {
    await triggerSafetyProtocol(result.conversation_id);
  }
}

// 5. Extract evidence for compliance reporting
const evidenceForReport = result.turn_analysis
  .filter(turn => turn.behaviors.length > 0)
  .map(turn => ({
    turn: turn.turn_number,
    content: turn.content_summary,
    behaviors: turn.behaviors.map(b => ({
      code: b.code,
      evidence: b.evidence
    }))
  }));

Common Patterns

Next Steps

• Evaluation API — For user-side risk assessment (suicide, self-harm, violence)
• Screen API — Lightweight crisis detection for compliance
• Webhooks — Setup and signature verification
• API Reference — Complete field documentation