Performance Optimization and Caching

Intermediate⏱45 min📖2 min readLesson 4 of 6

Performance Optimization and Caching

Optimize OpenCode for speed and cost efficiency. Learn caching strategies, token optimization, model selection, and how to reduce API calls while maintaining quality.

Why Optimize?

Metric	Impact
Response Time	Faster development workflow
Token Usage	Lower API costs
Quality	Better results with less overhead
Scalability	Handle larger projects

Token Optimization

Understanding Tokens

Tokens are the basic units LLMs process:

Content	Approximate Tokens
1 word	1-2 tokens
1 line of code	5-15 tokens
1 file (100 lines)	500-1500 tokens

Reduce Token Usage

Clear conversation regularly:

> /clear

Be specific in prompts:

❌ "Fix the bug"
✅ "Fix the ZeroDivisionError in calculate_average() when list is empty"

Read only relevant files:

❌ "Read all files in src/"
✅ "Read src/utils.py"

Use grep before reading:

> Search for "TODO" in src/ before reading entire files

Model Selection Strategy

Task	Model	Reason
Simple questions	gpt-4o-mini	Fast, cheap
Code generation	gpt-4o	Balanced
Complex reasoning	claude-sonnet-4-20250514	High quality
Code review	claude-sonnet-4-20250514	Thorough analysis
Quick edits	gpt-4o-mini	Speed

Configuration

json

{
  "agents": {
    "quick": {
      "model": "gpt-4o-mini",
      "description": "Simple tasks, fast responses"
    },
    "complex": {
      "model": "gpt-4o",
      "description": "Complex tasks, high quality"
    }
  }
}

Caching Strategies

File Content Cache

Cache frequently accessed files:

python

# Pseudo-code for caching
file_cache = {}

def read_file(path):
    if path in file_cache:
        return file_cache[path]
    
    content = read_from_disk(path)
    file_cache[path] = content
    return content

Prompt Cache

Reuse prompt templates:

json

{
  "skills": {
    "code-review": {
      "prompt_template": "Review this code for security issues: {code}"
    }
  }
}

Result Cache

Cache AI responses for repeated queries:

json

{
  "cache": {
    "enabled": true,
    "ttl": 3600,
    "maxSize": 1000
  }
}

Batch Processing

Process Multiple Files

Instead of:

> Read file1.py
> Read file2.py
> Read file3.py

Use:

> Read all Python files in src/ and summarize their purposes

Parallel Requests

json

{
  "performance": {
    "parallelRequests": true,
    "maxConcurrent": 3
  }
}

Context Window Management

Monitor Context Size

> /verbose
Context: 4523 tokens (15% of 32k limit)

Optimize Context

Clear history when switching topics
Read only necessary files
Summarize long conversations
Use subagents for isolated tasks

Network Optimization

Connection Pooling

json

{
  "providers": {
    "openai": {
      "keepAlive": true,
      "maxConnections": 5
    }
  }
}

Request Batching

json

{
  "performance": {
    "batchRequests": true,
    "batchSize": 10
  }
}

Performance Monitoring

Track Metrics

json

{
  "logging": {
    "performance": true,
    "tokenUsage": true,
    "responseTime": true
  }
}

Analyze Logs

bash

# Find slow requests
grep "response_time" opencode.log | awk '{print $NF}' | sort -n

# Calculate average tokens
grep "total_tokens" opencode.log | awk '{sum+=$NF} END {print sum/NR}'