LiveKit integration

livekit-plugins-shunyalabsai v1.0.0

Official Shunyalabs plugin for LiveKit Agents. Plug shunyalabs.STT and shunyalabs.TTS directly into a LiveKit AgentSession - supports real-time streaming transcription, batch recognition, and high-fidelity multilingual voice synthesis.

Requirements

Python 3.9+, LiveKit Agents framework, and a valid Shunyalabs API key.

Installation

Terminal

pip install livekit-plugins-shunyalabsai

Authentication

You need a Shunya Labs API key. Create one in the console - sign in at console.shunyalabs.ai, open API Keys, and click Create New Key (it is shown only once). Provide it as an environment variable or pass it directly to the plugin classes:

Environment variable

export SHUNYALABS_API_KEY="your-api-key"

Python — inline

stt = shunyalabs.STT(api_key="your-api-key")
tts = shunyalabs.TTS(api_key="your-api-key")

Your API key mints the token - the SDK does it for you

The ASR and TTS gateways never accept your API key directly. The plugin automatically exchanges your key for a short-lived access token - a JWT minted at app.shunyalabs.ai/api/auth/token - attaches it to every request, and refreshes it in the background before it expires. You only ever handle the API key; the token is generated, used, and rotated for you, and your raw key is never sent to the STT or TTS service.

Quick start

The minimal wiring - pass shunyalabs.STT and shunyalabs.TTS into an AgentSession:

Python

from livekit.agents import AgentSession
from livekit.plugins import shunyalabs, silero

session = AgentSession(
    stt=shunyalabs.STT(language="en"),
    tts=shunyalabs.TTS(voice="Rajesh", style="<Neutral>"),
    vad=silero.VAD.load(),
)

STT - `shunyalabs.STT`

Streaming and batch speech-to-text backed by the Shunyalabs ASR service. Audio frames from LiveKit are streamed over WebSocket; transcription events are pushed back as SpeechEvents.

Parameters

Parameter	Type	Default	Description
`api_key`	`str`	`None`	API key. Falls back to `SHUNYALABS_API_KEY` env var.
`language`	`str`	`"auto"`	BCP-47 language code or `"auto"` for detection.
`api_url`	`str`	`https://asrv2prod.shunyalabs.ai`	REST batch endpoint base URL.
`ws_url`	`str`	`wss://asrv2prod.shunyalabs.ai/v1/realtime`	WebSocket streaming endpoint URL.

Capabilities

Capability	Supported
Streaming (real-time)	Yes
Interim results	Yes
Offline / batch recognition	Yes

Streaming STT

Real-time transcription over WebSocket with event mapping to LiveKit's SpeechEventType:

Shunyalabs event	LiveKit SpeechEventType
`PARTIAL`	`INTERIM_TRANSCRIPT`
`FINAL_SEGMENT`	`FINAL_TRANSCRIPT` + `END_OF_SPEECH`
`FINAL`	`FINAL_TRANSCRIPT` + `RECOGNITION_USAGE`

Python — streaming STT

from livekit.agents import AgentSession
from livekit.plugins import shunyalabs, silero

session = AgentSession(
    stt=shunyalabs.STT(language="en"),
    vad=silero.VAD.load(),
)

@session.on("user_speech_committed")
def on_speech(ev):
    print(f"User said: {ev.transcript}")

Batch STT

Single-shot transcription of an audio buffer via POST /v1/audio/transcriptions:

Python — batch STT

from livekit.plugins import shunyalabs

stt = shunyalabs.STT(language="en")

# Inside an agent context:
event = await stt.recognize(audio_buffer)
print(event.alternatives[0].text)

TTS - `shunyalabs.TTS`

Streaming and chunked text-to-speech. Token-by-token streaming collects text then synthesises on flush via WebSocket; the batch API handles single-shot synthesis over HTTP.

Parameters

Parameter	Type	Default	Description
`api_key`	`str`	`None`	API key. Falls back to `SHUNYALABS_API_KEY` env var.
`api_url`	`str`	`https://ttsv2.shunyalabs.ai`	HTTP batch endpoint base URL.
`ws_url`	`str`	`wss://ttsv2.shunyalabs.ai/v1/realtime`	WebSocket streaming endpoint URL.
`model`	`str`	`"zero-indic"`	TTS model name.
`voice`	`str`	`"Rajesh"`	Voice name for the API.
`style`	`str`	`None`	Emotion style tag.
`language`	`str`	`"en"`	Language code for transliteration.
`sample_rate`	`int`	`24000`	Output audio sample rate in Hz. The gateway emits 24 kHz PCM on both the streaming and batch paths.
`output_format`	`str`	`"pcm"`	Audio format for the batch (`synthesize`) path: `pcm`, `wav`, `mp3`, `ogg_opus`, `flac`. The real-time stream is always PCM.
`speed`	`float`	`1.0`	Speaking speed multiplier (0.25 – 4.0).

Style tags

Tag	Description
`<Neutral>`	Neutral tone by default
`<Happy>`	Happy / cheerful
`<Sad>`	Sad / melancholic
`<Angry>`	Angry / intense
`<Fearful>`	Fearful / anxious
`<Surprised>`	Surprised / excited
`<Disgust>`	Disgusted
`<News>`	News anchor style
`<Conversational>`	Casual conversational - recommended for voice agents
`<Narrative>`	Storytelling / narration
`<Enthusiastic>`	Enthusiastic / energetic

Text formatting

The plugin automatically prepends the style tag before sending text to the API:

Python

tts = shunyalabs.TTS(voice="Rajesh", style="<Happy>")
# Input:  "Welcome to our platform"
# Sent:   "<Happy> Welcome to our platform"

Streaming TTS example

Python

from livekit.agents import AgentSession
from livekit.plugins import shunyalabs

session = AgentSession(
    tts=shunyalabs.TTS(
        style="<Conversational>",
        model="zero-indic",
        voice="Nisha",
    ),
)

Chunked (batch) TTS example

Python

from livekit.plugins import shunyalabs

tts = shunyalabs.TTS(voice="Varun")
stream = tts.synthesize("Hello, how can I help you today?")

Custom endpoints

Both services can be repointed without changing code or upgrading the package. Resolution precedence: explicit argument → endpoint returned by the token service → environment variable → built-in default.

Environment variables (batch = HTTP, streaming = WSS)

export SHUNYALABS_ASR_URL="https://<host>"                # batch
export SHUNYALABS_ASR_WS_URL="wss://<host>/v1/realtime"     # streaming
export SHUNYALABS_TTS_URL="https://<host>"
export SHUNYALABS_TTS_WS_URL="wss://<host>/v1/realtime"

Python — per instance

stt = shunyalabs.STT(api_url="https://<host>", ws_url="wss://<host>/v1/realtime")
tts = shunyalabs.TTS(api_url="https://<host>", ws_url="wss://<host>/v1/realtime")

Zero-downtime repointing

If the token service returns an endpoints object, the SDK uses it automatically - so Shunya Labs can move an endpoint centrally, with no code change or package release on your side.

Full agent example

Python

import asyncio
from livekit.agents import AgentSession, Agent, RoomInputOptions
from livekit.plugins import shunyalabs, silero

class MyAgent(Agent):
    def __init__(self):
        super().__init__(instructions="You are a helpful voice assistant.")

async def entrypoint(ctx):
    session = AgentSession(
        stt=shunyalabs.STT(language="auto"),
        tts=shunyalabs.TTS(
            model="zero-indic",
            voice="Rajesh",
            style="<Conversational>",
        ),
        vad=silero.VAD.load(),
    )
    await session.start(
        agent=MyAgent(),
        room=ctx.room,
        room_input_options=RoomInputOptions(),
    )

Multilingual example

Python

# Hindi speaker
tts_hindi = shunyalabs.TTS(
    voice="Rajesh",
    language="hi", style="<Neutral>",
)

# English speaker
tts_english = shunyalabs.TTS(
    voice="Varun",
    language="en", style="<Conversational>",
)

LiveKit integration

Installation

Authentication

Quick start

STT - shunyalabs.STT

Parameters

Capabilities

Streaming STT

Batch STT

TTS - shunyalabs.TTS

Parameters

Style tags

Text formatting

Streaming TTS example

Chunked (batch) TTS example

Custom endpoints

Full agent example

Multilingual example

STT - `shunyalabs.STT`

TTS - `shunyalabs.TTS`