Flutter Skill

Give your AI Agent eyes and hands inside your Flutter app.

Flutter Skill is a bridge that connects AI Agents (like Claude Code, Cursor, Windsurf) directly to a running Flutter application. It creates a bi-directional control channel allowing agents to:

• Inspect the UI structure (Widget Tree, Text Content, Element Properties)
• Act on widgets (Tap, Double Tap, Long Press, Swipe, Drag, Scroll, Enter Text)
• Validate state (Text Values, Checkbox State, Wait for Elements)
• Capture screenshots (Full App or Specific Elements)
• Navigate (Get Routes, Go Back, View Navigation Stack)
• Debug (View Logs, Errors, Performance Metrics)
• Zero Config setup (Auto-injects dependencies)

Features

UI Inspection

Interactions

State & Validation

Screenshot

Navigation

Debug & Logs

Development Tools

Architecture

graph TD
    Agent[AI Agent]
    subgraph "Flutter Skill Layer"
        CLI[flutter_skill CLI]
        MCP[flutter_skill MCP Server]
    end
    subgraph "Target App"
        App[Flutter App]
        VM[Dart VM Service]
    end

    Agent <-->|MCP / JSON-RPC| MCP
    Agent <-->|Shell| CLI
    CLI -->|Auto-Patch| App
    CLI <-->|Capture URI| VM
    MCP <-->|WebSocket| VM
    VM <-->|Extensions| App

Installation

Choose your preferred method:

Option 1: pub.dev (Recommended)

dart pub global activate flutter_skill

Option 2: Homebrew (macOS)

brew tap ai-dashboad/flutter-skill
brew install flutter-skill

Option 3: From Source

git clone https://github.com/ai-dashboad/flutter-skill.git
cd flutter-skill
dart pub global activate --source path .

Quick Start

You don't need to manually edit your code. The skill handles it for you.

1. Launch with Auto-Setup

# Launch your Flutter project
flutter_skill launch /path/to/your/flutter_project

What happens automatically:
1. Checks for flutter_skill dependency. If missing, adds it.
2. Checks main.dart for initialization. If missing, injects FlutterSkillBinding.ensureInitialized().
3. Runs flutter run.
4. Captures the VM Service URI and saves it to .flutter_skill_uri.

3. Let the Agent Take Over

CLI Mode (Claude Code):

# Inspect the screen
flutter_skill inspect

# Tap a button
flutter_skill act tap "login_button"

# Enter text
flutter_skill act enter_text "email_field" "[email protected]"

# Take screenshot
flutter_skill screenshot ./screenshot.png

MCP Mode (Cursor/Windsurf/Claude Desktop):

Add to your MCP configuration:

{
  "flutter-skill": {
    "command": "flutter-skill",
    "args": ["server"]
  }
}

Alternative configurations:

// If installed via Homebrew
{
  "flutter-skill": {
    "command": "flutter-skill-mcp"
  }
}

// If running from source
{
  "flutter-skill": {
    "command": "dart",
    "args": ["run", "/path/to/flutter-skill/bin/server.dart"]
  }
}

Then tools like connect_app, inspect, tap, screenshot become available to the Agent.

CLI Commands

# Launch app with auto-setup
flutter_skill launch <project_path> [-d <device>]

# Connect to running app
flutter_skill connect <vm_service_uri>

# Inspect UI elements
flutter_skill inspect

# Perform actions
flutter_skill act tap <key_or_text>
flutter_skill act enter_text <key> "text value"
flutter_skill act scroll_to <key_or_text>

# Take screenshot
flutter_skill screenshot [output_path]

# Start MCP server
flutter_skill server

Target App Setup (Manual)

Usually not needed - the launch command handles this automatically.

pubspec.yaml:

dependencies:
  flutter_skill:
    path: /path/to/flutter-skill

main.dart:

import 'package:flutter_skill/flutter_skill.dart';
import 'package:flutter/foundation.dart';

void main() {
  if (kDebugMode) {
    FlutterSkillBinding.ensureInitialized();
  }
  runApp(const MyApp());
}

Widget Keys

For reliable element identification, add ValueKey to your widgets:

ElevatedButton(
  key: const ValueKey('login_button'),
  onPressed: () {},
  child: const Text('Login'),
)

The skill can find elements by:
1. Key (most reliable): tap "login_button"
2. Text content: tap "Login"
3. Widget type: find_by_type "ElevatedButton"

IDE Extensions

VSCode Extension

Download from GitHub Releases and install manually:
- Download flutter-skill-0.2.0.vsix
- Run: code --install-extension flutter-skill-0.2.0.vsix

IntelliJ/Android Studio Plugin

Download from GitHub Releases and install manually:
- Download flutter-skill-intellij-0.2.0.zip
- In IDE: Settings → Plugins → ⚙ → Install Plugin from Disk

Development & Testing

# Run integration tests against mock app
dart run test/integration_test.dart

# Run the test app
cd test_dummy && flutter run -d macos

License

MIT