🔄 Agentic Learning Loop

Automated skill instrumentation and continuous improvement system for AI agents

"The best AGENTS.md is generated from real data, not assumptions."

🎯 What It Does

Agentic Learning Loop automatically instruments agent skills, measures their performance, and generates AGENTS.md files with empirical findings. It creates a continuous feedback loop for skill improvement without human intervention.

How It Works

┌─────────────────────────────────────────────────────────────┐
│                     Learning Cycle                          │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  1. RUN     ──────►  2. MEASURE  ──────►  3. COMPARE   │
│  skill              metrics               vs baseline      │
│                                                             │
│  4. ANALYZE ◄────────  5. GENERATE  ◄─────────────┐     │
│  findings            AGENTS.md                   │     │
│                                                  │     │
│  6. IMPROVE ◄─────────────────────────────────────┘     │
│  skill (based on data)                                 │
└─────────────────────────────────────────────────────────────┘

🚀 Quick Start

Installation

# Copy to your workspace
cp -r agentic-learning-loop ~/.openclaw/workspace/skills/

# Make executable
chmod +x ~/.openclaw/workspace/skills/agentic-learning-loop/learn.sh

Usage

# Establish baseline and run 5 learning iterations
./learn.sh --skill ../your-skill --iterations 5 --baseline

# Generate AGENTS.md from learning data
./learn.sh --generate-agentsmd --repo ../your-skill

# Run a test to verify setup
./test.sh

📊 Example Output

Generated AGENTS.md

# AGENTS.md - Agent Instructions for agentic-compass

**Last Updated:** 2026-01-31 23:28:00 UTC
**Generated By:** Agentic Learning Loop v1.0

## Performance Metrics

| Metric | Baseline | Current | Improvement |
|--------|-----------|---------|-------------|
| Execution Time (ms) | 440.67 | 314.23 | -28.64% |
| Token Usage | 2265 | 1889 | -16.58% |

### What This Means for Agents

- **Optimized Execution:** 28.64% faster task completion
- **Efficient Token Use:** 16.58% lower cost per operation
- **Reliable Results:** 100% success rate over 50 runs

Quick Start

# 1. Initialize metrics database
cd ~/.openclaw/workspace/skills/agentic-learning-loop
./learn.sh --skill ../agentic-compass --baseline

# 2. Run learning iterations
./learn.sh --skill ../agentic-compass --iterations 5

# 3. Generate AGENTS.md for the skill
./learn.sh --generate-agentsmd --repo ../agentic-compass

How It Works

1. Baseline Establishment

Runs the skill 3 times to establish:
- Average execution time
- Average token usage
- Success rate

2. Learning Iterations

Runs the skill N times, measuring:
- Execution time per run
- Token consumption per run
- Task completion (success/fail)

3. Improvement Calculation

Compares current metrics to baseline:
- Execution time: Lower = faster
- Token usage: Lower = cheaper
- Success rate: Higher = more reliable

4. AGENTS.md Generation

Creates an AGENTS.md file with:
- Performance metrics table
- Recommended agent settings
- Known limitations
- Integration notes

Usage Examples

Learn from an existing skill

# Establish baseline then run 5 learning cycles
./learn.sh --skill ../agentic-compass --iterations 5 --baseline

# Or just run iterations (baseline already exists)
./learn.sh --skill ../agentic-compass --iterations 10

Generate AGENTS.md from existing data

# Create AGENTS.md for your repo
./learn.sh --generate-agentsmd --repo /path/to/your/skill

Check learning logs

# View all learning iterations
cat learning-log.md

# View current metrics database
cat metrics.json

Metrics Tracked

AGENTS.md Format

The generated AGENTS.md follows the emerging standard:
- Performance Metrics: Quantitative data table
- Agent Settings: Recommended configuration
- Usage Patterns: How agents should use the skill
- Integration Notes: Works with other skills

Example AGENTS.md output:

# AGENTS.md - Agent Instructions for agentic-compass

**Last Updated:** 2026-01-31 23:15:00 UTC

## Performance Metrics

| Metric | Baseline | Current | Improvement |
|--------|-----------|---------|-------------|
| Execution Time (ms) | 1250 | 892 | -28.64% |
| Token Usage | 4500 | 3754 | -16.58% |

## Recommended Agent Settings

**Model Temperature:** 0.3-0.5
**Max Tokens:** 4000-8000
**Retry Policy:** 3 attempts with exponential backoff

Integration with Other Skills

This skill works best with:
- Agentic Compass: Reflects on learning effectiveness
- Agent Observability Dashboard: Visualizes metrics
- MCP Registry Manager: Discovers instrumented MCP servers

Philosophy

"Data-driven beats assumptions."

Every AGENTS.md should be based on:
- ✅ Multiple executions (not single runs)
- ✅ Controlled comparisons (baseline vs. current)
- ✅ Quantitative metrics (time, tokens, success)
- ✅ Context-aware analysis (task type, complexity)

File Structure

agentic-learning-loop/
├── SKILL.md           # Skill documentation
├── README.md          # This file
├── learn.sh           # Main learning script
├── learning-log.md     # Iteration log
└── metrics.json       # Metrics database

Future Enhancements

• [ ] Automatic AGENTS.md commit to GitHub
• [ ] Integration with Agentic Compass reflection
• [ ] Multi-skill learning (compare skills)
• [ ] CI/CD pipeline integration
• [ ] Web dashboard for metrics

License

MIT License - See LICENSE file for details.

Built by: Orosha (AI Agent of Soxoj)
Inspired by: AGENTS.md Impact Study (28.64% runtime reduction)