name: source-ingest
version: "1.0.0"
description: Fetch and parse source materials for node implementation. Retrieves TypeScript source or API documentation based on classified source type.

Contract

autonomy_level: READ
side_effects: [net]
timeout_seconds: 120
retry:
policy: safe
max_retries: 3
backoff_seconds: 5.0
idempotency:
required: true
key_spec: "correlation_id"
max_fix_iterations: 1

Sync Celery Constraints (MANDATORY)

sync_celery:
requires_sync_execution: true
forbids_async_dependencies: true
requires_timeouts_on_external_calls: true
forbids_background_tasks: true

input_schema:
type: object
required: [correlation_id, source_type, evidence]
properties:
correlation_id:
type: string
source_type:
type: string
enum: [TYPE1, TYPE2]
evidence:
type: array

output_schema:
type: object
required: [raw_content, parsed_sections, metadata]
properties:
raw_content:
type: string
parsed_sections:
type: object
metadata:
type: object
properties:
fetch_time: { type: string, format: date-time }
source_url: { type: string }
content_hash: { type: string }

required_artifacts:
- name: raw_source.txt
type: txt
description: Raw fetched content
- name: parsed_source.json
type: json
description: Parsed and structured source content

failure_modes: [network_error, timeout, parse_error]
depends_on: [source-classify]

Source Ingest

Fetch and parse source materials based on classified source type.

TYPE1: TypeScript Source

1. Fetch TypeScript file from GitHub/local path
2. Parse and extract:
3. Node class definition
4. description object
5. properties array (parameters)
6. execute() / trigger() methods
7. Credential references
8. Store parsed content

TYPE2: Documentation

1. Fetch documentation from URLs
2. Parse format (HTML, Markdown, OpenAPI)
3. Extract:
4. Endpoints and operations
5. Parameters and schemas
6. Authentication requirements
7. Example requests/responses

Parsed Sections Structure

{
  "class_name": "TelegramNode",
  "type": "telegram",
  "version": 2.0,
  "description": { ... },
  "properties": { ... },
  "methods": ["execute"],
  "credentials": ["telegramApi"]
}

Artifacts Emitted

• artifacts/{correlation_id}/raw_source.txt
• artifacts/{correlation_id}/parsed_source.json