tikker/docs/PERFORMANCE.md

# Tikker Performance Optimization Guide

## Performance Benchmarks

Baseline performance metrics on standard hardware (2CPU, 2GB RAM):

### API Service (C Tools Integration)
- Health Check: ~15ms (p50), <50ms (p99)
- Daily Stats: ~80ms (p50), <150ms (p99)
- Top Words: ~120ms (p50), <250ms (p99)
- Throughput: ~40-60 req/s

### AI Service
- Health Check: ~10ms (p50), <50ms (p99)
- Text Analysis: ~2-5s (depends on text length and API availability)
- Throughput: ~0.5 req/s (limited by OpenAI API)

### Visualization Service
- Health Check: ~12ms (p50), <50ms (p99)
- Bar Chart: ~150ms (p50), <300ms (p99)
- Line Chart: ~160ms (p50), <320ms (p99)
- Pie Chart: ~140ms (p50), <280ms (p99)
- Throughput: ~5-8 req/s

## Running Benchmarks

### Quick Benchmark
```bash
python scripts/benchmark.py
```

### Benchmark Against Remote Server
```bash
python scripts/benchmark.py http://production-server
```

### Detailed Test Results
```bash
pytest tests/test_performance.py -v --tb=short
```

## Optimization Strategies

### 1. Database Optimization

#### Vacuum Database
Regular database maintenance improves query performance.

```bash
docker-compose exec api sqlite3 tikker.db "VACUUM;"
```

Impact: 5-15% query speed improvement

#### Create Indexes
Add indexes for frequently queried columns:

```sql
CREATE INDEX idx_words_frequency ON words(frequency DESC);
CREATE INDEX idx_events_timestamp ON events(timestamp);
CREATE INDEX idx_events_date ON events(date);
```

Impact: 30-50% improvement for indexed queries

#### Query Optimization
Use EXPLAIN QUERY PLAN to analyze slow queries:

```bash
sqlite3 tikker.db "EXPLAIN QUERY PLAN SELECT * FROM words ORDER BY frequency LIMIT 10;"
```

### 2. Caching Strategies

#### Redis Caching for Frequent Queries
Add Redis for popular word list caching:

```python
import redis
cache = redis.Redis(host='localhost', port=6379)

def get_top_words(limit=10):
    key = f"top_words:{limit}"
    cached = cache.get(key)
    if cached:
        return json.loads(cached)
    
    result = query_database(limit)
    cache.setex(key, 3600, json.dumps(result))
    return result
```

Impact: 10-100x improvement for cached queries

#### Add to docker-compose.yml:
```yaml
redis:
  image: redis:7-alpine
  ports:
    - "6379:6379"
  volumes:
    - redis_data:/data
```

### 3. Python Optimization

#### Enable Optimization
```dockerfile
ENV PYTHONOPTIMIZE=2
ENV PYTHONDONTWRITEBYTECODE=1
```

#### Use Async I/O
Current API already uses FastAPI (async), good baseline.

#### Profile Code
Identify bottlenecks:
```bash
python -m cProfile -s cumtime -m pytest tests/test_services.py
```

### 4. C Tools Optimization

#### Compile Flags
Update Makefile with optimization flags:

```makefile
CFLAGS = -O3 -march=native -Wall -Wextra
```

Impact: 20-40% improvement in execution speed

#### Binary Stripping
Reduce binary size:

```bash
strip build/bin/tikker-*
```

Impact: Faster loading, reduced disk I/O

### 5. Network Optimization

#### Connection Pooling
Add HTTP connection pooling in wrapper:

```python
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

session = requests.Session()
retry_strategy = Retry(
    total=3,
    backoff_factor=1,
    status_forcelist=[429, 500, 502, 503, 504]
)
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount("http://", adapter)
```

#### Service Co-location
Run services on same host to reduce latency:
- Typical inter-service latency: ~5-10ms
- Same-host latency: <1ms

### 6. Memory Optimization

#### Monitor Memory Usage
```bash
docker stats

# Or detailed analysis
docker-compose exec api ps aux
```

#### Reduce Buffer Sizes
In c_tools_wrapper.py:
```python
# Limit concurrent subprocess calls
from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(max_workers=4)
```

#### Garbage Collection Tuning
```python
import gc
gc.set_threshold(10000)
```

### 7. Container Resource Limits

Update docker-compose.yml:

```yaml
services:
  api:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G
```

### 8. Load Balancing

For production deployments with multiple instances:

```yaml
nginx:
  image: nginx:latest
  ports:
    - "8000:8000"
  volumes:
    - ./nginx.conf:/etc/nginx/nginx.conf:ro
  depends_on:
    - api1
    - api2
    - api3
```

nginx.conf:
```nginx
upstream api {
    server api1:8000;
    server api2:8000;
    server api3:8000;
}

server {
    listen 8000;
    location / {
        proxy_pass http://api;
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
    }
}
```

## Performance Tuning Checklist

- [ ] Database vacuumed and indexed
- [ ] Python optimization flags enabled
- [ ] C compilation optimizations applied
- [ ] Connection pooling configured
- [ ] Caching strategy implemented
- [ ] Memory limits set appropriately
- [ ] Load balancing configured (if needed)
- [ ] Monitoring and logging enabled
- [ ] Benchmarks show acceptable latency
- [ ] Throughput meets SLA requirements

## Monitoring Performance

### Key Metrics to Track

1. **Latency (p50, p95, p99)**
   - Target: p50 <100ms, p99 <500ms

2. **Throughput (req/s)**
   - Target: >20 req/s per service

3. **Error Rate**
   - Target: <0.1%

4. **Resource Usage**
   - CPU: <80% sustained
   - Memory: <80% allocated
   - Disk: <90% capacity

### Prometheus Metrics

Add to FastAPI apps:

```python
from prometheus_client import Counter, Histogram, generate_latest

request_count = Counter('api_requests_total', 'Total requests')
request_duration = Histogram('api_request_duration_seconds', 'Request duration')

@app.middleware("http")
async def add_metrics(request, call_next):
    start = time.time()
    response = await call_next(request)
    duration = time.time() - start
    
    request_count.inc()
    request_duration.observe(duration)
    return response

@app.get("/metrics")
def metrics():
    return generate_latest()
```

## Troubleshooting Performance Issues

### High CPU Usage
1. Profile code: `python -m cProfile`
2. Check for infinite loops in C tools
3. Reduce concurrent operations

### High Memory Usage
1. Monitor with `docker stats`
2. Check for memory leaks in C code
3. Implement garbage collection tuning
4. Use connection pooling

### Slow Queries
1. Run EXPLAIN QUERY PLAN
2. Add missing indexes
3. Verify statistics are current
4. Consider query rewriting

### Network Latency
1. Check service co-location
2. Verify DNS resolution
3. Monitor with `tcpdump`
4. Consider service mesh (istio)

### Database Lock Issues
1. Check for long-running transactions
2. Verify concurrent access limits
3. Consider read replicas
4. Increase timeout values

## Advanced Optimization

### Async Database Access
Consider async SQLite driver for true async I/O:

```python
from aiosqlite import connect

async def get_stats():
    async with connect('tikker.db') as db:
        cursor = await db.execute('SELECT * FROM events')
        return await cursor.fetchall()
```

### Compiled C Extensions
Convert performance-critical Python code to C extensions:

```c
// stats.c
PyObject* get_daily_stats(PyObject* self, PyObject* args) {
    // High-performance C implementation
}
```

### Graph Query Optimization
For complex analyses, consider graph database:

```
Events → Words → Patterns
Analysis becomes graph traversal instead of SQL joins
```

## SLA Targets

Recommended SLA targets for Tikker:

| Metric | Target | Priority |
|--------|--------|----------|
| API Availability | 99.5% | Critical |
| Health Check Latency | <50ms | Critical |
| Stats Query Latency | <200ms | High |
| Word Search Latency | <300ms | High |
| Report Generation | <5s | Medium |
| AI Analysis | <10s | Low |

## Performance Testing in CI/CD

Add performance regression testing:

```bash
# Run baseline benchmark
python scripts/benchmark.py baseline

# Run benchmark
python scripts/benchmark.py current

# Compare and fail if regression
python scripts/compare_benchmarks.py baseline current --fail-if-slower 10%
```

## Further Reading

- SQLite Performance: https://www.sqlite.org/bestcase.html
- FastAPI Performance: https://fastapi.tiangolo.com/
- Python Optimization: https://docs.python.org/3/library/profile.html