name: python-automation
description: Comprehensive Python automation toolkit for file processing, data manipulation, web scraping, and task automation. Use when users need to automate repetitive tasks, process files in bulk, manipulate data, scrape websites, or create Python scripts for system administration and workflow automation.

Python Automation

Overview

This skill provides patterns, utilities, and best practices for Python automation tasks including file processing, data manipulation, web scraping, and system administration.

Quick Start

For common automation needs:
1. File operations: Use scripts in scripts/file_operations.py
2. Data processing: Use scripts in scripts/data_processing.py
3. Web automation: Use scripts in scripts/web_automation.py
4. System tasks: Use scripts in scripts/system_tasks.py

Core Capabilities

1. File Operations

• Batch rename files matching patterns
• Recursive directory processing
• File content search and replace
• Archive creation and extraction
• File monitoring and watching

2. Data Processing

• CSV/JSON/Excel manipulation
• Data cleaning and transformation
• Batch data validation
• Report generation
• Database operations

3. Web Automation

• Web scraping with BeautifulSoup/Scrapy
• API interaction and data fetching
• Form submission automation
• Download management
• Browser automation with Selenium

4. System Administration

• Process management
• Log file analysis
• Scheduled task creation
• Environment setup
• Configuration management

Working with Files

When processing files, use pathlib for cross-platform compatibility:

from pathlib import Path

def process_files(directory, pattern="*.txt"):
    path = Path(directory)
    for file in path.glob(f"**/{pattern}"):
        # Process each file
        with file.open() as f:
            content = f.read()
            # Apply transformations

Data Processing Patterns

For efficient data processing:

import pandas as pd

def process_large_csv(filename, chunk_size=10000):
    for chunk in pd.read_csv(filename, chunksize=chunk_size):
        # Process chunk
        processed = chunk.apply(transform_function)
        # Save or aggregate results

Web Scraping Best Practices

Always respect robots.txt and add delays:

import requests
from time import sleep
from bs4 import BeautifulSoup

def scrape_safely(url, delay=1):
    sleep(delay)  # Be respectful
    response = requests.get(url, headers={'User-Agent': 'Automation Bot 1.0'})
    if response.status_code == 200:
        soup = BeautifulSoup(response.text, 'html.parser')
        return soup

Error Handling

Implement robust error handling for automation:

import logging
from functools import wraps

def retry_on_failure(max_attempts=3, delay=1):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    logging.warning(f"Attempt {attempt + 1} failed: {e}")
                    if attempt < max_attempts - 1:
                        sleep(delay * (attempt + 1))
                    else:
                        raise
            return wrapper
        return decorator

Resources

scripts/

• file_operations.py - File manipulation utilities
• data_processing.py - Data transformation tools
• web_automation.py - Web scraping helpers
• system_tasks.py - System administration utilities

references/

• patterns.md - Common automation patterns and recipes
• libraries.md - Recommended libraries and tools