AssemblyAI Transcription & Audio Intelligence

AssemblyAI provides audio transcription plus AI-powered analysis layers: sentiment, entities, speakers, chapters, and LeMUR for LLM-over-audio tasks.

Auth & Setup

npm install assemblyai

API key: D:\Ecosystem\secrets\MASTER_API_KEYS.env → ASSEMBLYAI_API_KEY

Base URL: https://api.assemblyai.com/v2/

Recommended model: universal-2 (not nano — deprecated)

Basic Transcription (Node.js SDK)

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

const transcript = await client.transcripts.transcribe({
  audio_url: 'https://example.com/recording.mp3',
  speech_model: 'universal-2'
});

console.log(transcript.text);

Transcription with Intelligence Features

const transcript = await client.transcripts.transcribe({
  audio_url: 'https://example.com/call.mp3',
  sentiment_analysis: true,
  entity_detection: true,
  speaker_labels: true,       // diarization — who said what
  auto_chapters: true,        // automatic chapter markers
  summarization: true,
  summary_model: 'informative',
  summary_type: 'bullets'
});

// Speaker labels
transcript.utterances.forEach(u => {
  console.log(`Speaker ${u.speaker}: ${u.text}`);
});

// Sentiment per sentence
transcript.sentiment_analysis_results.forEach(r => {
  console.log(`${r.sentiment}: ${r.text}`);
});

LeMUR — LLM Over Audio

Run any Claude prompt against the audio without transcribing first.

const response = await client.lemur.task({
  transcript_ids: [transcript.id],
  prompt: 'Summarize the key objections raised and how they were handled.',
  final_model: 'anthropic/claude-3-5-sonnet'
});

console.log(response.response);

Async Pattern (REST API)

# Step 1: Submit
curl -X POST https://api.assemblyai.com/v2/transcript \
  -H "Authorization: $ASSEMBLYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"audio_url": "https://example.com/audio.mp3", "speech_model": "universal-2"}'

# Returns: {"id": "abc123", "status": "processing"}

# Step 2: Poll (10s interval)
curl https://api.assemblyai.com/v2/transcript/abc123 \
  -H "Authorization: $ASSEMBLYAI_API_KEY"

Intelligence Feature Cost Matrix

| Feature | Extra Cost | Use Case | |---------|-----------|----------| | sentiment_analysis | Low | Call coaching, review mining | | entity_detection | Low | CRM extraction, lead qualification | | speaker_labels | Low | Multi-party call diarization | | auto_chapters | Free | Video chapter markers | | summarization | Low | Executive call summaries | | LeMUR | Per token | Any LLM analysis over audio |

Use Cases for Video Production

Transcribe client testimonial recordings for captions
Extract chapter markers from long-form video recordings
Analyze call recordings for script research (real objection language)
Generate SRT files for video captioning pipeline

AssemblyAI Transcription & Audio Intelligence

AssemblyAI Transcription & Audio Intelligence

Auth & Setup

Basic Transcription (Node.js SDK)

Transcription with Intelligence Features

LeMUR — LLM Over Audio

Async Pattern (REST API)

Intelligence Feature Cost Matrix

Use Cases for Video Production

Related Topics