SmartVoice Chat 🗣️

Intelligent Voice Conversation Skill for Clawdbot

Offline voice-to-voice interaction powered by Sherpa-ONNX. Automatically detects voice/text input and replies in the same format.

✨ Features

• 🎯 Auto Detection - Automatically detects voice vs text input
• 🗣️ Voice-to-Voice - Replies in same format (voice→voice, text→text)
• 🌏 Chinese-English - Native mixed language support
• 🔒 Fully Offline - No cloud, privacy-preserving
• 📱 Telegram Ready - Outputs OGG format for voice messages

📋 System Requirements

Operating System

• macOS: 11.0+ (Big Sur or later)
• Linux: Ubuntu 20.04+, Debian 11+, or equivalent
• Arch: x86_64 or ARM64 (Apple Silicon supported)

Dependencies

Hardware Requirements

Network Requirements

• Required: For downloading models and dependencies (initial setup only)
• Runtime: Fully offline after installation

🔧 Sherpa-ONNX Models

STT (Speech-to-Text) Options

Recommended: sherpa-onnx-paraformer-zh-2024-03-09 for best accuracy.

TTS (Text-to-Speech) Options

Chinese-English (Recommended)

English Voices

Japanese

Spanish

French

German

Kokoro Multi-Language (103+ Speakers)

Recommended: vits-melo-tts-zh_en for mixed Chinese-English.

🎧 Try Before You Download

Listen to samples at the Sherpa-ONNX Text-to-Speech Space on HuggingFace.

📚 More Languages and Models

Sherpa-ONNX supports 40+ languages and 100+ pre-trained models:

• Full Model List: https://k2-fsa.github.io/sherpa/onnx/tts/all/
• VITS Models: https://k2-fsa.github.io/sherpa/onnx/tts/pretrained_models/vits.html
• GitHub Releases: https://github.com/k2-fsa/sherpa-onnx/releases/tag/tts-models

Supported languages include: Arabic, Bulgarian, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hindi, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Swedish, Thai, Turkish, Ukrainian, Vietnamese, and more.

📦 Installation

Quick Install (Recommended)

1. Clone this repository

git clone https://github.com/Johnny-xuan/smart-voice-chat.git ~/smart-voice-chat

1. Download Sherpa-ONNX runtime

# macOS
curl -L https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.12.23/sherpa-onnx-v1.12.23-osx-universal2-shared.tar.bz2 | tar xjf -
mkdir -p ~/.clawdbot/sherpa-asr/runtime
mv sherpa-onnx*/* ~/.clawdbot/sherpa-asr/runtime/

1. Download models

# STT Model (Chinese)
mkdir -p ~/.clawdbot/sherpa-asr/models
cd ~/.clawdbot/sherpa-asr/models
curl -L https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-paraformer-zh-2024-03-09.tar.bz2 | tar xjf -

# TTS Model (Chinese-English)
mkdir -p ~/.clawdbot/tools/sherpa-onnx-tts/models
cd ~/.clawdbot/tools/sherpa-onnx-tts/models
curl -L https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-melo-tts-zh_en.tar.bz2 | tar xjf -

1. Install to Clawdbot skills

# Find Clawdbot skills directory
CLAWDBOT_SKILLS=$(npm root -g)/clawdbot@*/node_modules/clawdbot/skills

# Copy skill
cp -r ~/smart-voice-chat "$CLAWDBOT_SKILLS/"

# Verify
clawdbot skills list | grep smart-voice

Expected output: │ ✓ ready │ 🗣️ smart-voice- │ ...

1. Restart Clawdbot

pkill -9 clawdbot
clawdbot-gateway &

⚙️ Configuration

Environment Variables (Optional)

Add to ~/.clawdbot/clawdbot.json:

{
  "skills": {
    "entries": {
      "smart-voice-chat": {
        "env": {
          "SMART_VOICE_CHAT_STT_MODEL": "~/.clawdbot/sherpa-asr/models/sherpa-onnx-paraformer-zh-2024-03-09",
          "SMART_VOICE_CHAT_TTS_MODEL": "~/.clawdbot/tools/sherpa-onnx-tts/models/vits-melo-tts-zh_en",
          "SMART_VOICE_CHAT_OUTPUT_DIR": "/tmp/smart-voice-chat"
        }
      }
    }
  }
}

SKILL.md Format

The SKILL.md frontmatter must use valid JSON:

---
name: smart-voice-chat
description: "Voice conversation with auto-detection (voice-to-voice, text-to-text)"
metadata: {"clawdbot":{"emoji":"🗣️","os":["darwin","linux"],"requires":{"anyBins":["ffmpeg"]}}}
---

Important:
- Always wrap description in quotes
- Avoid special characters (use to instead of →)
- requires only supports: bins, anyBins, env, config
- Does NOT support: python field (ignored by Clawdbot)

💡 Usage

Default Mirror Mode

You: [Voice message] "What's the weather like today?"
AI:  [Voice + Text] "It's sunny today, 25°C"

You: "What's the weather like today?"
AI:  "It's sunny today, 25°C" [Text only]

Override with Keywords

You: "Reply with voice: Will it rain tomorrow?"
AI:  [Voice only] "It might rain lightly tomorrow"

You: "Reply with text: What time is it now?"
AI:  [Text only] "It's 4 PM now"

🔧 Tech Stack

🐛 Troubleshooting

Skill not showing in clawdbot skills list

1. Check SKILL.md syntax:

head -5 ~/.clawdbot/skills/smart-voice-chat/SKILL.md

1. Verify FFmpeg is installed:

which ffmpeg

1. Check logs:

tail -50 ~/.clawdbot/logs/gateway.err.log

OGG conversion fails

Install FFmpeg:

brew install ffmpeg  # macOS
sudo apt install ffmpeg  # Ubuntu/Debian

📄 License

MIT License - see LICENSE

🙏 Acknowledgments

• Clawdbot - AI Agent Framework
• Sherpa-ONNX - Offline speech processing
• Paraformer - Alibaba's ASR model
• VITS-Melo - MyShell's TTS model

Author: Johnny
GitHub: smart-voice-chat