LangChain Integration

The @prosody/langchain package provides LangChain tools and utilities for building emotion-aware AI agents.

ProsodyAI + LangChain enables agents that understand not just what users say, but how they feel.

Installation

npm install @prosody/langchain langchain @langchain/core

pnpm add @prosody/langchain langchain @langchain/core

pip install prosody-langchain langchain langchain-core

Quick Start

import { ProsodyEmotionTool } from '@prosody/langchain';
import { ChatOpenAI } from '@langchain/openai';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';

// Create the emotion analysis tool
const emotionTool = new ProsodyEmotionTool({
  apiKey: process.env.PROSODY_API_KEY,
  vertical: 'contact_center',
});

// Create an agent with emotion awareness
const agent = createToolCallingAgent({
  llm: new ChatOpenAI({ model: 'gpt-4' }),
  tools: [emotionTool],
  prompt: `You are a helpful customer service agent. 
Use the emotion tool to understand how customers are feeling 
and adapt your responses accordingly.`,
});

const executor = new AgentExecutor({ agent, tools: [emotionTool] });

// The agent can now analyze audio and respond empathetically
const result = await executor.invoke({
  input: 'Help this customer',
  audio: customerAudioBuffer,
});

Use Cases

Emotion-Aware Chatbots

Build chatbots that adapt their tone based on user sentiment:

const emotionTool = new ProsodyEmotionTool({ apiKey, vertical: 'contact_center' });

// Agent can detect frustration and respond empathetically
// "I can hear you're frustrated. Let me prioritize getting this resolved for you."

Voice AI Assistants

Create voice assistants that understand emotional context:

const tools = [
  new ProsodyEmotionTool({ apiKey, vertical: 'healthcare' }),
  new ProsodyPredictionTool({ apiKey }),
];

// Agent can detect distress and escalate appropriately

Sales Copilots

Build AI that helps sales reps respond to buyer signals:

const emotionTool = new ProsodyEmotionTool({ apiKey, vertical: 'sales' });

// Agent detects skepticism and suggests providing evidence
// "The prospect seems skeptical. Consider sharing the case study."

Interview Analysis

Analyze candidate responses in real-time:

const emotionTool = new ProsodyEmotionTool({ apiKey, vertical: 'hr_interviews' });

// Agent flags evasive responses for follow-up

Available Tools

Tool	Description
`ProsodyEmotionTool`	Analyze audio for emotion
`ProsodyPredictionTool`	Get conversation predictions
`ProsodySessionTool`	Manage conversation sessions
`ProsodyVerticalTool`	Get vertical-specific metrics

Features

Automatic audio handling - Tools accept file paths, URLs, or buffers
Streaming support - Real-time analysis during conversations
Session management - Track multi-turn conversation state
Type safety - Full TypeScript support
Async/await - Native async support for all operations

Architecture

┌─────────────────────────────────────────────────────────┐
│                    LangChain Agent                       │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐     │
│  │   LLM       │  │   Memory    │  │   Tools     │     │
│  │  (GPT-4)    │  │ (Session)   │  │             │     │
│  └─────────────┘  └─────────────┘  └──────┬──────┘     │
│                                           │             │
│                            ┌──────────────┴───────────┐ │
│                            │    ProsodyEmotionTool    │ │
│                            └──────────────┬───────────┘ │
└───────────────────────────────────────────┼─────────────┘
                                            │
                                            ▼
                               ┌─────────────────────────┐
                               │     ProsodyAI API       │
                               │  - Emotion Analysis     │
                               │  - Conversation Predict │
                               │  - Vertical Metrics     │
                               └─────────────────────────┘

Next Steps

Tools Reference - Detailed tool documentation
Agent Examples - Building emotion-aware agents
Code Examples - Real-world implementations

LangChain Integration

On this page