huggingface_transformers

by @DonggangChen in AI & LLM

# Install this skill:

npx skills add DonggangChen/antigravity-agentic-skills --skill "huggingface_transformers"

Install specific skill from multi-skill repository

# Description

Hugging Face Transformers best practices including model loading, tokenization, fine-tuning workflows, and inference optimization. Use when working with transformer models, fine-tuning LLMs, implementing NLP tasks, or optimizing transformer inference.

# SKILL.md

name: huggingface_transformers
router_kit: AIKit
description: Hugging Face Transformers best practices including model loading, tokenization, fine-tuning workflows, and inference optimization. Use when working with transformer models, fine-tuning LLMs, implementing NLP tasks, or optimizing transformer inference.
metadata:
skillport:
category: auto-healed
tags: [bert, big data, cleaning, csv, data analysis, data engineering, data science, database, etl pipelines, export, gpt, huggingface transformers, import, inference, json, machine learning basics, migration, models, nlp, nosql, numpy, pandas, python data stack, query optimization, reporting, schema design, sql, statistics, transformation, visualization]

Hugging Face Transformers Best Practices

Comprehensive guide to using the Hugging Face Transformers library including model loading, tokenization, fine-tuning workflows, pipeline usage, custom datasets, and deployment optimization.

Quick Reference

When to use this skill:
- Loading and using pre-trained transformers (BERT, GPT, T5, LLaMA, etc.)
- Fine-tuning models on custom data
- Implementing NLP tasks (classification, QA, generation, etc.)
- Optimizing inference (quantization, ONNX, etc.)
- Debugging tokenization issues
- Using Hugging Face pipelines
- Deploying transformers to production

Models covered:
- Encoders: BERT, RoBERTa, DeBERTa, ALBERT
- Decoders: GPT-2, GPT-Neo, LLaMA, Mistral
- Encoder-Decoders: T5, BART, Flan-T5
- Vision: ViT, CLIP, Stable Diffusion

Part 1: Model Loading Patterns

Pattern 1: Basic Model Loading

from transformers import AutoModel, AutoTokenizer

# Load model and tokenizer
model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# For specific tasks
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    num_labels=3  # For 3-class classification
)

Pattern 2: Loading with Specific Configuration

from transformers import AutoConfig, AutoModel

# Modify configuration
config = AutoConfig.from_pretrained("bert-base-uncased")
config.hidden_dropout_prob = 0.2  # Custom dropout
config.attention_probs_dropout_prob = 0.2

# Load model with custom config
model = AutoModel.from_pretrained("bert-base-uncased", config=config)

# Or create model from scratch with config
model = AutoModel.from_config(config)

Pattern 3: Loading Quantized Models (Memory Efficient)

from transformers import AutoModel, BitsAndBytesConfig
import torch

# 8-bit quantization (50% memory reduction)
quantization_config = BitsAndBytesConfig(load_in_8bit=True)

model = AutoModel.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=quantization_config,
    device_map="auto"  # Automatic device placement
)

# 4-bit quantization (75% memory reduction)
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True
)

model = AutoModel.from_pretrained(
    "meta-llama/Llama-2-13b-hf",
    quantization_config=quantization_config,
    device_map="auto"
)

Pattern 4: Loading from Local Path

# Save model locally
model.save_pretrained("./my-model")
tokenizer.save_pretrained("./my-model")

# Load from local path
model = AutoModel.from_pretrained("./my-model")
tokenizer = AutoTokenizer.from_pretrained("./my-model")

Part 2: Tokenization Best Practices

Critical Tokenization Patterns

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# ✅ CORRECT: All required arguments
tokens = tokenizer(
    text,
    padding=True,  # Pad to longest in batch
    truncation=True,  # Truncate to max_length
    max_length=512,  # Maximum sequence length
    return_tensors="pt"  # Return PyTorch tensors
)

# Access components
input_ids = tokens['input_ids']  # Token IDs
attention_mask = tokens['attention_mask']  # Padding mask
token_type_ids = tokens.get('token_type_ids')  # Segment IDs (BERT)

# ❌ WRONG: Missing critical arguments
tokens = tokenizer(text)  # No padding, truncation, or tensor format!

Batch Tokenization

# Tokenize multiple texts efficiently
texts = ["First text", "Second text", "Third text"]

tokens = tokenizer(
    texts,
    padding=True,  # Pad all to longest in batch
    truncation=True,
    max_length=128,
    return_tensors="pt"
)

# Result shape: [batch_size, max_length]
print(tokens['input_ids'].shape)  # torch.Size([3, max_len_in_batch])

Special Token Handling

# Add special tokens
tokenizer.add_special_tokens({
    'additional_special_tokens': ['[CUSTOM]', '[MARKER]']
})

# Resize model embeddings to match
model.resize_token_embeddings(len(tokenizer))

# Encode with special tokens preserved
text = "Hello [CUSTOM] world"
tokens = tokenizer(text, add_special_tokens=True)

# Decode
decoded = tokenizer.decode(tokens['input_ids'][0], skip_special_tokens=False)

Tokenization for Different Tasks

# Text classification (single sequence)
tokens = tokenizer(
    "This movie was great!",
    padding="max_length",
    truncation=True,
    max_length=128,
    return_tensors="pt"
)

# Question answering (pair of sequences)
question = "What is the capital of France?"
context = "France is a country in Europe. Paris is its capital."

tokens = tokenizer(
    question,
    context,
    padding="max_length",
    truncation="only_second",  # Only truncate context
    max_length=384,
    return_tensors="pt"
)

# Text generation (decoder-only models)
prompt = "Once upon a time"
tokens = tokenizer(prompt, return_tensors="pt")
# No padding needed for generation input

Part 3: Fine-Tuning Workflows

Pattern 1: Simple Fine-Tuning with Trainer

from transformers import (
    AutoModelForSequenceClassification,
    AutoTokenizer,
    Trainer,
    TrainingArguments
)
from datasets import load_dataset

# 1. Load dataset
dataset = load_dataset("glue", "mrpc")

# 2. Load model
model = AutoModelForSequenceClassification.from_pretrained(
    "bert-base-uncased",
    num_labels=2
)
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# 3. Tokenize dataset
def tokenize_function(examples):
    return tokenizer(
        examples["sentence1"],
        examples["sentence2"],
        padding="max_length",
        truncation=True,
        max_length=128
    )

tokenized_datasets = dataset.map(tokenize_function, batched=True)

# 4. Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
    logging_dir="./logs",
    logging_steps=100,
    save_strategy="epoch",
    load_best_model_at_end=True,
    metric_for_best_model="accuracy",
)

# 5. Define metrics
from datasets import load_metric
import numpy as np

metric = load_metric("accuracy")

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)

# 6. Create Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"],
    compute_metrics=compute_metrics,
)

# 7. Train
trainer.train()

# 8. Save
trainer.save_model("./fine-tuned-model")

Pattern 2: LoRA Fine-Tuning (Parameter-Efficient)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model, TaskType

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    load_in_8bit=True,  # 8-bit for memory efficiency
    device_map="auto"
)

# Configure LoRA
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=8,  # LoRA rank
    lora_alpha=32,  # LoRA alpha
    lora_dropout=0.1,
    target_modules=["q_proj", "v_proj"],  # Which layers to adapt
)

# Apply LoRA
model = get_peft_model(model, lora_config)

# Check trainable parameters
model.print_trainable_parameters()
# Output: trainable params: 4.2M || all params: 6.7B || trainable%: 0.062%

# Train with Trainer (same as before)
# Only LoRA parameters are updated!

Pattern 3: Custom Training Loop

import torch
from torch.utils.data import DataLoader
from transformers import AdamW, get_scheduler

# Prepare dataloaders
train_dataloader = DataLoader(tokenized_datasets["train"], batch_size=16, shuffle=True)
eval_dataloader = DataLoader(tokenized_datasets["validation"], batch_size=16)

# Optimizer
optimizer = AdamW(model.parameters(), lr=2e-5)

# Learning rate scheduler
num_epochs = 3
num_training_steps = num_epochs * len(train_dataloader)
lr_scheduler = get_scheduler(
    "linear",
    optimizer=optimizer,
    num_warmup_steps=500,
    num_training_steps=num_training_steps
)

# Training loop
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

for epoch in range(num_epochs):
    model.train()
    for batch in train_dataloader:
        batch = {k: v.to(device) for k, v in batch.items()}

        outputs = model(**batch)
        loss = outputs.loss
        loss.backward()

        optimizer.step()
        lr_scheduler.step()
        optimizer.zero_grad()

    # Evaluation
    model.eval()
    for batch in eval_dataloader:
        batch = {k: v.to(device) for k, v in batch.items()}
        with torch.no_grad():
            outputs = model(**batch)
        # Compute metrics

Part 4: Pipeline Usage (High-Level API)

Text Classification Pipeline

from transformers import pipeline

# Load pipeline
classifier = pipeline(
    "text-classification",
    model="distilbert-base-uncased-finetuned-sst-2-english"
)

# Single prediction
result = classifier("I love this product!")
# [{'label': 'POSITIVE', 'score': 0.9998}]

# Batch prediction
results = classifier([
    "Great service!",
    "Terrible experience",
    "Average quality"
])

Question Answering Pipeline

qa_pipeline = pipeline("question-answering", model="distilbert-base-uncased-distilled-squad")

result = qa_pipeline(
    question="What is the capital of France?",
    context="France is a country in Europe. Its capital is Paris, a beautiful city."
)
# {'score': 0.98, 'start': 49, 'end': 54, 'answer': 'Paris'}

Text Generation Pipeline

generator = pipeline("text-generation", model="gpt2")

outputs = generator(
    "Once upon a time",
    max_length=50,
    num_return_sequences=3,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
    do_sample=True
)

for output in outputs:
    print(output['generated_text'])

Zero-Shot Classification Pipeline

classifier = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")

result = classifier(
    "This is a course about Python programming.",
    candidate_labels=["education", "technology", "business", "sports"]
)
# {'sequence': '...', 'labels': ['education', 'technology', ...], 'scores': [0.85, 0.12, ...]}

Part 5: Inference Optimization

Optimization 1: Batch Processing

# ❌ SLOW: Process one at a time
for text in texts:
    output = model(**tokenizer(text, return_tensors="pt"))

# ✅ FAST: Process in batches
batch_size = 32
for i in range(0, len(texts), batch_size):
    batch = texts[i:i+batch_size]
    inputs = tokenizer(batch, padding=True, truncation=True, return_tensors="pt")
    outputs = model(**inputs)

Optimization 2: Mixed Precision (AMP)

from torch.cuda.amp import autocast, GradScaler

scaler = GradScaler()

for batch in dataloader:
    optimizer.zero_grad()

    # Forward pass in mixed precision
    with autocast():
        outputs = model(**batch)
        loss = outputs.loss

    # Backward pass with scaled gradients
    scaler.scale(loss).backward()
    scaler.step(optimizer)
    scaler.update()

Optimization 3: ONNX Export

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification

# Export to ONNX
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
model.save_pretrained("./onnx-model", export=True)

# Load ONNX model (faster inference)
ort_model = ORTModelForSequenceClassification.from_pretrained("./onnx-model")

# Inference (2-3x faster)
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
inputs = tokenizer("Hello world", return_tensors="pt")
outputs = ort_model(**inputs)

Optimization 4: Dynamic Quantization

import torch

# Quantize model to int8
quantized_model = torch.quantization.quantize_dynamic(
    model,
    {torch.nn.Linear},  # Quantize Linear layers
    dtype=torch.qint8
)

# 4x smaller model, 2-3x faster inference on CPU

Part 6: Common Issues & Solutions

Issue 1: CUDA Out of Memory

Problem: RuntimeError: CUDA out of memory

Solutions:

# Solution 1: Reduce batch size
training_args = TrainingArguments(
    per_device_train_batch_size=8,  # Was 32
    gradient_accumulation_steps=4,  # Effective batch = 8*4 = 32
)

# Solution 2: Use gradient checkpointing
model.gradient_checkpointing_enable()

# Solution 3: Use 8-bit model
from transformers import BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModel.from_pretrained("model-name", quantization_config=quantization_config)

# Solution 4: Clear cache
import torch
torch.cuda.empty_cache()

Issue 2: Slow Tokenization

Problem: Tokenization is bottleneck

Solutions:

# Solution 1: Use fast tokenizers
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased", use_fast=True)

# Solution 2: Tokenize dataset once, cache it
tokenized_dataset = dataset.map(
    tokenize_function,
    batched=True,
    num_proc=4,  # Parallel processing
    remove_columns=dataset.column_names,
    load_from_cache_file=True  # Cache results
)

# Solution 3: Use larger batches for tokenization
tokenizer(
    texts,
    padding=True,
    truncation=True,
    max_length=512,
    return_tensors="pt",
    batched=True,  # Process multiple texts at once
    batch_size=1000
)

Issue 3: Inconsistent Results

Problem: Model outputs different results for same input

Solution:

# Set seeds for reproducibility
import random
import numpy as np
import torch

def set_seed(seed=42):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

set_seed(42)

# Disable dropout during inference
model.eval()

# Use deterministic generation
outputs = model.generate(
    inputs,
    do_sample=False,  # Greedy decoding
    # OR
    do_sample=True,
    temperature=1.0,
    top_k=50,
    seed=42  # For sampling
)

Issue 4: Attention Mask Errors

Problem: IndexError: index out of range in self