Use when adding new error messages to React, or seeing "unknown error code" warnings.
npx skills add miles-knowbl/orchestrator --skill "integration-test"
Install specific skill from multi-skill repository
# Description
Validates that multiple systems work together correctly. Tests cross-system contracts, end-to-end workflows, and interface compatibility. Complements unit testing (test-generation) and code validation (code-validation) by focusing on system boundaries.
# SKILL.md
name: integration-test
description: "Validates that multiple systems work together correctly. Tests cross-system contracts, end-to-end workflows, and interface compatibility. Complements unit testing (test-generation) and code validation (code-validation) by focusing on system boundaries."
phase: TEST
category: core
version: "1.0.0"
depends_on: [test-generation]
tags: [testing, integration, e2e, api-testing, core-workflow]
Integration Test
Validate systems work together.
When to Use
- After implementing a system — Verify it integrates with dependencies
- Before shipping — Confirm end-to-end workflows function
- After contract changes — Validate interface compatibility
- Cross-system features — Test workflows spanning multiple systems
- Regression prevention — Ensure changes don't break integrations
Reference Requirements
MUST read before applying this skill:
| Reference | Why Required |
|---|---|
scenario-templates.md |
Templates for integration test scenarios |
Read if applicable:
| Reference | When Needed |
|---|---|
| MCP tool definitions or API specs | When testing specific integrations |
Verification: Ensure integration tests are executable, not just documented.
Required Deliverables
| Deliverable | Location | Condition |
|---|---|---|
| Integration test files | tests/integration/ |
Always |
INTEGRATION-TEST.md |
Project root | When documenting test scenarios |
Core Concept
Integration testing answers: "Do these systems work together correctly?"
┌─────────────────────────────────────────────────────────────────────────────┐
│ INTEGRATION TESTING │
│ │
│ Unit Tests Integration Tests E2E Tests │
│ ────────── ───────────────── ───────── │
│ Single System Full │
│ Component Boundaries Workflows │
│ │
│ ┌───────┐ ┌───────┐ ┌───────┐ ┌───────┐ │
│ │ A │ │ A │──│ B │ │ A │──▶│ B │──▶│ C │
│ └───────┘ └───────┘ └───────┘ └───────┘ │
│ │
│ "Does A work?" "Do A and B work "Does the whole │
│ together?" workflow work?" │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Integration Test Types
1. Contract Tests
Verify systems honor their API contracts.
| Type | Description | Example |
|---|---|---|
| Consumer Contract | Consumer's expectations of provider | "Order Service expects Auth to return JWT with userId" |
| Provider Contract | Provider's published interface | "Auth Service returns JWT per OpenAPI spec" |
| Pact-style | Bidirectional contract verification | Both sides verify against shared contract |
2. API Integration Tests
Test actual HTTP/RPC communication between services.
| Focus | What to Test |
|---|---|
| Happy path | Standard request → expected response |
| Error handling | Invalid input → proper error code |
| Authentication | Token → authorized/unauthorized correctly |
| Rate limiting | Excessive requests → throttled |
| Timeout | Slow response → handled gracefully |
3. Event Integration Tests
Test asynchronous message passing.
| Focus | What to Test |
|---|---|
| Event publishing | Action → event published correctly |
| Event consuming | Event received → correct handling |
| Event ordering | Sequence preserved when required |
| Duplicate handling | Same event twice → idempotent |
| Dead letter | Failed processing → captured |
4. Data Integration Tests
Test data flows between systems.
| Focus | What to Test |
|---|---|
| Data format | Correct schema between systems |
| Data transformation | Mapped correctly |
| Data consistency | Same data in both systems |
| Referential integrity | Foreign keys valid |
5. End-to-End Tests
Test complete user workflows.
| Focus | What to Test |
|---|---|
| Critical paths | Main user journeys |
| Cross-system flows | Actions spanning services |
| State transitions | Complete lifecycle |
The Integration Test Process
┌─────────────────────────────────────────────────────────────────────────────┐
│ INTEGRATION TEST PROCESS │
│ │
│ 1. IDENTIFY INTEGRATION POINTS │
│ └─→ What systems does this system interact with? │
│ └─→ What APIs does it call? What events does it publish/consume? │
│ │
│ 2. GATHER CONTRACTS │
│ └─→ Load API specs from interfaces/api/ │
│ └─→ Load event schemas from interfaces/events/ │
│ │
│ 3. DESIGN TEST SCENARIOS │
│ └─→ Happy path scenarios │
│ └─→ Error scenarios │
│ └─→ Edge cases │
│ │
│ 4. SET UP TEST ENVIRONMENT │
│ └─→ Start required services (or mocks) │
│ └─→ Set up test data │
│ └─→ Configure test credentials │
│ │
│ 5. IMPLEMENT TESTS │
│ └─→ Contract tests │
│ └─→ API integration tests │
│ └─→ Event integration tests │
│ └─→ E2E tests for critical paths │
│ │
│ 6. RUN AND VERIFY │
│ └─→ Execute tests │
│ └─→ Analyze failures │
│ └─→ Fix contract mismatches │
│ │
│ 7. DOCUMENT INTEGRATION │
│ └─→ Update interface documentation │
│ └─→ Document test coverage │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Step 1: Identify Integration Points
For the system under test, document:
## Integration Points: [System Name]
### APIs Called (Outbound)
| Service | Endpoint | Method | Purpose |
|---------|----------|--------|---------|
| Auth | /auth/verify | POST | Verify JWT token |
| User | /users/{id} | GET | Get user details |
### APIs Exposed (Inbound)
| Endpoint | Method | Consumers | Purpose |
|----------|--------|-----------|---------|
| /orders | POST | Mobile App, Web App | Create order |
| /orders/{id} | GET | Mobile App, Web App, Analytics | Get order |
### Events Published
| Event | Channel | Consumers |
|-------|---------|-----------|
| OrderCreated | orders/created | Notification, Analytics |
| OrderCompleted | orders/completed | Billing, Analytics |
### Events Consumed
| Event | Channel | Publisher |
|-------|---------|-----------|
| UserCreated | users/created | User Service |
| PaymentReceived | payments/received | Billing Service |
### Data Dependencies
| System | Data | Direction | Sync Method |
|--------|------|-----------|-------------|
| User Service | User records | Read | API call |
| Inventory | Stock levels | Read | Event |
Step 2: Gather Contracts
Load contracts from domain memory:
# API contracts
cat domain-memory/{domain}/interfaces/api/auth-service.yaml
cat domain-memory/{domain}/interfaces/api/order-service.yaml
# Event contracts
cat domain-memory/{domain}/interfaces/events/order-events.yaml
If contracts don't exist, create them using templates from memory-manager/references/interface-contracts.md.
Step 3: Design Test Scenarios
Scenario Template
## Scenario: [Name]
**Type:** Contract | API | Event | E2E
**Systems Involved:** [List]
**Priority:** High | Medium | Low
### Setup
- [Precondition 1]
- [Precondition 2]
### Steps
1. [Action 1]
2. [Action 2]
3. [Action 3]
### Expected Results
- [Expected outcome 1]
- [Expected outcome 2]
### Cleanup
- [Cleanup step 1]
Scenario Categories
| Category | Scenarios to Include |
|---|---|
| Happy Path | Standard successful flow |
| Authentication | Valid token, invalid token, expired token |
| Authorization | Permitted actions, forbidden actions |
| Validation | Valid input, invalid input, edge cases |
| Error Handling | Service down, timeout, malformed response |
| Concurrency | Simultaneous requests, race conditions |
| Data Integrity | Consistent data across systems |
→ See references/scenario-templates.md
Step 4: Test Environment
Options
| Environment | Pros | Cons | Use When |
|---|---|---|---|
| Mocks | Fast, isolated | Not real services | Unit-level integration |
| Docker Compose | Realistic, reproducible | Setup complexity | Local development |
| Staging | Production-like | Shared, may be unstable | Pre-production |
| Production (read-only) | Real data | Risk | Smoke tests only |
Docker Compose Example
# docker-compose.test.yaml
version: '3.8'
services:
# System under test
order-service:
build: .
ports:
- "3000:3000"
environment:
- AUTH_SERVICE_URL=http://auth-service:3001
- DATABASE_URL=postgres://postgres:postgres@db:5432/orders
depends_on:
- auth-service
- db
# Dependency
auth-service:
image: auth-service:test
ports:
- "3001:3001"
environment:
- JWT_SECRET=test-secret
# Database
db:
image: postgres:14
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=orders
# Message broker
kafka:
image: confluentinc/cp-kafka:latest
ports:
- "9092:9092"
Test Data Setup
// test/setup.js
async function setupTestData() {
// Create test users
await authService.createUser({
id: 'test-user-1',
email: '[email protected]',
role: 'technician'
});
// Create test orders
await orderService.createOrder({
id: 'test-order-1',
customerId: 'test-customer-1',
status: 'pending'
});
}
async function teardownTestData() {
await orderService.deleteOrder('test-order-1');
await authService.deleteUser('test-user-1');
}
→ See references/test-environment-setup.md
Step 5: Implement Tests
Contract Test Example (Pact-style)
// Consumer side (Order Service expects Auth Service)
describe('Auth Service Contract', () => {
it('verifies JWT token', async () => {
// Define expected interaction
await provider.addInteraction({
state: 'a valid JWT exists',
uponReceiving: 'a request to verify token',
withRequest: {
method: 'POST',
path: '/auth/verify',
headers: { 'Content-Type': 'application/json' },
body: { token: 'valid-jwt-token' }
},
willRespondWith: {
status: 200,
body: {
valid: true,
userId: Matchers.string(),
expiresAt: Matchers.iso8601DateTime()
}
}
});
// Execute
const result = await authClient.verifyToken('valid-jwt-token');
// Verify
expect(result.valid).toBe(true);
expect(result.userId).toBeDefined();
});
});
API Integration Test Example
// test/integration/order-api.test.js
describe('Order API Integration', () => {
let authToken;
beforeAll(async () => {
// Get real auth token from Auth Service
authToken = await authService.login({
email: '[email protected]',
password: 'test-password'
});
});
describe('POST /orders', () => {
it('creates order with valid token', async () => {
const response = await request(app)
.post('/orders')
.set('Authorization', `Bearer ${authToken}`)
.send({
customerId: 'test-customer-1',
items: [{ productId: 'prod-1', quantity: 2 }]
});
expect(response.status).toBe(201);
expect(response.body.id).toBeDefined();
expect(response.body.status).toBe('pending');
// Verify in database
const order = await db.orders.findById(response.body.id);
expect(order).toBeDefined();
});
it('rejects request with invalid token', async () => {
const response = await request(app)
.post('/orders')
.set('Authorization', 'Bearer invalid-token')
.send({ customerId: 'test-customer-1' });
expect(response.status).toBe(401);
});
it('handles auth service timeout gracefully', async () => {
// Simulate Auth Service being slow
authServiceMock.setLatency(5000);
const response = await request(app)
.post('/orders')
.set('Authorization', `Bearer ${authToken}`)
.send({ customerId: 'test-customer-1' });
// Should timeout and return error
expect(response.status).toBe(503);
expect(response.body.error).toContain('Service unavailable');
});
});
});
Event Integration Test Example
// test/integration/order-events.test.js
describe('Order Events Integration', () => {
let eventCapture;
beforeAll(async () => {
// Subscribe to events
eventCapture = await kafka.subscribe('orders/#');
});
afterAll(async () => {
await kafka.unsubscribe(eventCapture);
});
it('publishes OrderCreated event on order creation', async () => {
// Create order
const order = await orderService.createOrder({
customerId: 'test-customer-1',
items: [{ productId: 'prod-1', quantity: 2 }]
});
// Wait for event
const event = await eventCapture.waitForEvent(
'orders/created',
{ orderId: order.id },
5000 // timeout
);
// Verify event
expect(event.orderId).toBe(order.id);
expect(event.customerId).toBe('test-customer-1');
expect(event.timestamp).toBeDefined();
});
it('handles duplicate events idempotently', async () => {
const event = {
eventId: 'evt-123',
orderId: 'order-123',
customerId: 'cust-123'
};
// Publish same event twice
await kafka.publish('orders/created', event);
await kafka.publish('orders/created', event);
// Should only create one record
const orders = await db.orders.findByCustomerId('cust-123');
expect(orders.length).toBe(1);
});
});
End-to-End Test Example
// test/e2e/order-workflow.test.js
describe('Order Workflow E2E', () => {
it('completes full order lifecycle', async () => {
// Step 1: Customer creates order
const createResponse = await api
.post('/orders')
.set('Authorization', `Bearer ${customerToken}`)
.send({
customerId: customerId,
items: [{ productId: 'prod-1', quantity: 1 }]
});
expect(createResponse.status).toBe(201);
const orderId = createResponse.body.id;
// Step 2: Order assigned to technician
const assignResponse = await api
.post(`/orders/${orderId}/assign`)
.set('Authorization', `Bearer ${dispatcherToken}`)
.send({ technicianId: technicianId });
expect(assignResponse.status).toBe(200);
// Verify notification sent (check Notification Service)
const notifications = await notificationService
.getNotifications(technicianId);
expect(notifications).toContainEqual(
expect.objectContaining({ type: 'ORDER_ASSIGNED', orderId })
);
// Step 3: Technician starts work
const startResponse = await api
.post(`/orders/${orderId}/start`)
.set('Authorization', `Bearer ${technicianToken}`);
expect(startResponse.status).toBe(200);
// Step 4: Technician completes order
const completeResponse = await api
.post(`/orders/${orderId}/complete`)
.set('Authorization', `Bearer ${technicianToken}`)
.send({
signature: 'base64-signature',
notes: 'Work completed successfully'
});
expect(completeResponse.status).toBe(200);
// Verify final state
const finalOrder = await api.get(`/orders/${orderId}`);
expect(finalOrder.body.status).toBe('completed');
// Verify analytics received event
const analytics = await analyticsService.getOrderMetrics(orderId);
expect(analytics.completedAt).toBeDefined();
});
});
→ See references/test-patterns.md
Step 6: Run and Verify
Running Tests
# Run all integration tests
npm run test:integration
# Run specific test file
npm run test:integration -- order-api.test.js
# Run with coverage
npm run test:integration -- --coverage
# Run in CI
npm run test:integration:ci
Analyzing Failures
| Failure Type | Likely Cause | Investigation |
|---|---|---|
| Connection refused | Service not running | Check Docker, ports |
| 401 Unauthorized | Token issue | Check token generation, expiry |
| 500 Internal Error | Bug in service | Check service logs |
| Timeout | Slow service or deadlock | Check performance, dependencies |
| Contract mismatch | API changed | Compare contract vs implementation |
| Event not received | Pub/sub issue | Check broker, subscriptions |
Contract Mismatch Resolution
When contract test fails:
- Identify mismatch — What differs from contract?
- Determine source of truth — Is contract wrong or implementation wrong?
- Coordinate change — If contract changes, notify all consumers
- Update and verify — Fix mismatch, re-run tests
Step 7: Document Integration
Integration Test Coverage
## Integration Test Coverage: [System Name]
### API Integration
| Endpoint | Test Count | Covered Scenarios |
|----------|------------|-------------------|
| POST /orders | 5 | create, validation, auth, error handling |
| GET /orders/{id} | 3 | found, not found, unauthorized |
| POST /orders/{id}/complete | 4 | success, validation, auth, already complete |
### Event Integration
| Event | Direction | Test Count | Covered Scenarios |
|-------|-----------|------------|-------------------|
| OrderCreated | Publish | 3 | publish, format, idempotency |
| PaymentReceived | Consume | 2 | process, duplicate handling |
### E2E Workflows
| Workflow | Test Count | Systems Involved |
|----------|------------|------------------|
| Order Lifecycle | 1 | Auth, Order, Notification, Analytics |
| Payment Flow | 1 | Order, Payment, Billing |
### Coverage Gaps
- [ ] Rate limiting not tested
- [ ] Bulk operations not tested
Relationship to Other Skills
| Skill | Relationship |
|---|---|
test-generation |
Creates unit tests; integration-test focuses on boundaries |
code-validation |
Validates single system; integration-test validates interactions |
memory-manager |
Provides interface contracts for testing |
loop-controller |
Invokes integration-test during VALIDATE stage |
Key Principles
Test at boundaries. Focus on where systems meet.
Use real services when practical. Mocks hide integration issues.
Test failure modes. Systems fail; test graceful degradation.
Keep tests independent. Each test should set up and clean up.
Contract tests prevent regression. They catch breaking changes early.
Mode-Specific Behavior
Integration testing approach differs by orchestrator mode:
Greenfield Mode
| Aspect | Behavior |
|---|---|
| Scope | Full coverage—all integration points |
| Approach | Comprehensive—contract, API, event, E2E |
| Patterns | Free choice—establish test infrastructure |
| Deliverables | Full test suite, Docker Compose, docs |
| Validation | All integration points tested |
| Constraints | Minimal—full control over environment |
Greenfield integration testing:
1. Define all integration contracts
2. Set up test environment (Docker Compose)
3. Create test data factories
4. Write contract tests for all interfaces
5. Write API integration tests for all endpoints
6. Write event integration tests for all events
7. Write E2E tests for critical user journeys
8. Document integration test coverage
Greenfield test priorities:
- All outbound API calls tested
- All inbound API endpoints tested
- All events (publish/consume) tested
- Critical E2E flows covered
- Contract tests for all interfaces
Brownfield-Polish Mode
| Aspect | Behavior |
|---|---|
| Scope | Gap-specific—new/modified integrations |
| Approach | Extend existing—match test patterns |
| Patterns | Should match existing test conventions |
| Deliverables | Gap tests following existing structure |
| Validation | Existing tests pass + gaps covered |
| Constraints | Don't restructure—use existing infrastructure |
Polish integration testing:
1. Identify existing integration test patterns
2. Find untested integration points
3. Add tests following existing conventions
4. Use existing test infrastructure
5. Verify new code doesn't break existing integrations
6. Update documentation for new integrations
Polish considerations:
- Match existing test file structure
- Use existing test utilities and helpers
- Follow existing naming conventions
- Don't change test configuration
- Add to existing test suites
Polish gap analysis:
## Integration Test Gaps
### Currently Tested
- Auth API integration ✓
- User API integration ✓
### Missing Tests
- Payment API integration ← Add
- Notification events ← Add
- Rate limiting ← Add
Brownfield-Enterprise Mode
| Aspect | Behavior |
|---|---|
| Scope | Change-specific—affected integrations only |
| Approach | Surgical—regression tests for changes |
| Patterns | Must conform exactly to existing tests |
| Deliverables | Regression tests, contract verification |
| Validation | Full regression—existing tests pass |
| Constraints | Requires approval—no contract changes without sign-off |
Enterprise integration testing:
1. Identify which integrations are affected by change
2. Find existing tests for those integrations
3. Add regression tests matching existing patterns
4. Verify existing integration tests still pass
5. Document any contract changes required
6. Get contract changes approved before implementing
Enterprise constraints:
- Do NOT add new integration test frameworks
- Do NOT modify test environment configuration
- Do NOT change existing contract definitions without approval
- Tests must pass in existing CI/CD pipeline
- Contract changes require consumer notification
Mode-Specific Test Types
| Test Type | Greenfield | Polish | Enterprise |
|---|---|---|---|
| Contract | Create all | Add missing | Verify existing |
| API | All endpoints | New endpoints | Changed endpoints |
| Event | All events | New events | Changed events |
| E2E | Critical flows | Gap fill | If affected |
Environment by Mode
| Mode | Environment | Setup Ownership |
|---|---|---|
| Greenfield | Docker Compose (custom) | Full control |
| Polish | Existing test infrastructure | Extend carefully |
| Enterprise | Approved environments only | No modifications |
Contract Management by Mode
Greenfield:
- Create new contracts for all interfaces
- Define contract versioning strategy
- Establish contract testing pipeline
- Full ownership of contract definitions
Polish:
- Verify against existing contracts
- Add contracts for new integrations
- Follow existing versioning patterns
- Coordinate changes with existing consumers
Enterprise:
- Read-only contract verification
- No contract modifications without approval
- Document contract change requests
- Follow change management process
Integration Test Coverage Targets
| Mode | Target Coverage |
|---|---|
| Greenfield | 100% of integration points |
| Polish | Match existing + new integrations |
| Enterprise | Changed integrations only |
References
references/scenario-templates.md: Test scenario templatesreferences/test-environment-setup.md: Environment configurationreferences/test-patterns.md: Common integration test patternsreferences/contract-testing.md: Contract testing deep dive
# Supported AI Coding Agents
This skill is compatible with the SKILL.md standard and works with all major AI coding agents:
Learn more about the SKILL.md standard and how to use these skills with your preferred AI coding agent.