b0esche_cloud/docs/DEPLOYMENT.md

# b0esche.cloud Deployment Guide

This guide covers production deployment, server configuration, and operations.

## Production Architecture

### Server Overview

| Component | Domain | Port | Container |
|-----------|--------|------|-----------|
| Flutter Web | www.b0esche.cloud | 80 | `flutter-web` |
| Go Backend | go.b0esche.cloud | 8080 | `go-backend` |
| PostgreSQL | internal | 5432 | `go-postgres` |
| Nextcloud | storage.b0esche.cloud | 80 | `nextcloud` |
| Collabora | of.b0esche.cloud | 9980 | `collabora` |
| Traefik | - | 80, 443 | `traefik` |

### Server Directory Structure

```
/opt/
├── traefik/
│   ├── docker-compose.yml    # Traefik + Nextcloud + Collabora
│   ├── traefik.yml           # Static configuration
│   ├── .env                  # DNS credentials
│   └── acme/                 # SSL certificates
├── go/
│   ├── docker-compose.yml    # Go backend + PostgreSQL
│   ├── .env.production       # Production environment
│   └── data/
│       └── postgres/
│           └── backend/
│               └── go_cloud/ # Backend source code
├── flutter/
│   ├── docker-compose.yml    # Nginx for Flutter
│   ├── nginx.conf            # Nginx configuration
│   └── web/                  # Built Flutter files
├── scripts/
│   ├── auto-deploy.sh        # Daily auto-deployment
│   ├── deploy-now.sh         # Manual deployment trigger
│   ├── backup.sh             # Backup script
│   ├── monitor.sh            # Health monitoring
│   └── webhook-server.py     # GitLab webhook receiver
└── auto-deploy/
    └── b0esche_cloud_rollout/ # Deployment workspace
```

## Deployment Methods

### 1. Automatic Deployment (Recommended)

Deployments run automatically at 3 AM daily via cron:

```cron
0 3 * * * /opt/scripts/auto-deploy.sh >> /var/log/auto-deploy.log 2>&1
```

The auto-deploy script:
1. Pulls latest changes from GitLab
2. Builds Flutter web app
3. Rebuilds Go backend Docker image
4. Restarts services
5. Validates health checks

### 2. Manual Deployment (Immediate)

Trigger an immediate deployment:

```bash
# From local machine
ssh b0esche-cloud '/opt/scripts/deploy-now.sh'

# Or directly on server
/opt/scripts/deploy-now.sh
```

### 3. GitLab Webhook (On Push)

The webhook server listens for push events:

```bash
# Start webhook server (runs as systemd service)
systemctl start webhook-server

# Check webhook logs
journalctl -u webhook-server -f
```

## Service Management

### Starting All Services

```bash
# Start in order (dependencies first)
cd /opt/traefik && docker-compose up -d
cd /opt/go && docker-compose up -d
cd /opt/flutter && docker-compose up -d
```

### Stopping All Services

```bash
cd /opt/flutter && docker-compose down
cd /opt/go && docker-compose down
cd /opt/traefik && docker-compose down
```

### Restarting Individual Services

```bash
# Restart Go backend
cd /opt/go && docker-compose restart go-backend

# Restart Flutter frontend
cd /opt/flutter && docker-compose restart flutter-web

# Restart Traefik (caution: brief SSL interruption)
cd /opt/traefik && docker-compose restart traefik
```

### Viewing Logs

```bash
# Follow Go backend logs
docker logs -f go-backend

# Follow Flutter/Nginx logs
docker logs -f flutter-web

# Follow Traefik logs
docker logs -f traefik

# All logs with timestamps
docker logs -f --timestamps go-backend
```

## Configuration Files

### Traefik Configuration

**docker-compose.yml:**
```yaml
version: '3.8'

services:
  traefik:
    image: traefik:v2.10
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik.yml:/etc/traefik/traefik.yml:ro
      - ./acme:/etc/traefik/acme
    networks:
      - proxy
    restart: unless-stopped

networks:
  proxy:
    external: true
```

**traefik.yml:**
```yaml
entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"

certificatesResolvers:
  letsencrypt:
    acme:
      email: admin@b0esche.cloud
      storage: /etc/traefik/acme/acme.json
      dnsChallenge:
        provider: bunny
        delayBeforeCheck: 30

providers:
  docker:
    exposedByDefault: false
```

### Go Backend Configuration

**docker-compose.yml:**
```yaml
version: '3.8'

services:
  postgres:
    image: postgres:15-alpine
    container_name: go-postgres
    environment:
      POSTGRES_USER: go_backend
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: go_backend
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
    networks:
      - backend
    restart: unless-stopped

  go-backend:
    build:
      context: ./data/postgres/backend/go_cloud
      dockerfile: Dockerfile
    container_name: go-backend
    env_file: .env.production
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.go.rule=Host(`go.b0esche.cloud`)"
      - "traefik.http.routers.go.tls.certresolver=letsencrypt"
    depends_on:
      - postgres
    networks:
      - proxy
      - backend
    restart: unless-stopped

networks:
  proxy:
    external: true
  backend:
    driver: bridge
```

### Flutter/Nginx Configuration

**docker-compose.yml:**
```yaml
version: '3.8'

services:
  flutter-web:
    image: nginx:alpine
    container_name: flutter-web
    volumes:
      - ./web:/usr/share/nginx/html:ro
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.flutter.rule=Host(`www.b0esche.cloud`)"
      - "traefik.http.routers.flutter.tls.certresolver=letsencrypt"
    networks:
      - proxy
    restart: unless-stopped

networks:
  proxy:
    external: true
```

**nginx.conf:**
```nginx
server {
    listen 80;
    server_name www.b0esche.cloud;
    root /usr/share/nginx/html;
    index index.html;

    # Flutter web app routing
    location / {
        try_files $uri $uri/ /index.html;
    }

    # Cache static assets
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    # Gzip compression
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;
}
```

## Database Operations

### Running Migrations

```bash
# Enter backend container
docker exec -it go-backend sh

# Run migrations
./api migrate up

# Or from host
docker exec go-backend ./api migrate up
```

### Database Backup

```bash
# Manual backup
docker exec go-postgres pg_dump -U go_backend -Fc go_backend > backup.sqlc

# Restore from backup
docker exec -i go-postgres pg_restore -U go_backend -d go_backend < backup.sqlc
```

### Connecting to Database

```bash
# Via docker exec
docker exec -it go-postgres psql -U go_backend -d go_backend

# Common queries
\dt                          # List tables
\d users                     # Describe table
SELECT count(*) FROM users;  # Count users
```

## SSL Certificate Management

### Certificate Status

```bash
# Check certificate expiry
docker exec traefik cat /etc/traefik/acme/acme.json | jq '.letsencrypt.Certificates[].certificate.NotAfter'

# Force certificate renewal
docker restart traefik
```

### Manual Certificate Operations

```bash
# Backup certificates
cp -r /opt/traefik/acme /opt/traefik/acme.backup

# View certificate details
openssl s_client -connect www.b0esche.cloud:443 -servername www.b0esche.cloud </dev/null 2>/dev/null | openssl x509 -noout -dates
```

## Monitoring

### Health Checks

```bash
# Check all services
/opt/scripts/monitor.sh

# Manual health checks
curl -s https://go.b0esche.cloud/health
curl -s -o /dev/null -w "%{http_code}" https://www.b0esche.cloud
curl -s -o /dev/null -w "%{http_code}" https://storage.b0esche.cloud
```

### Container Status

```bash
# All containers
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Resource usage
docker stats --no-stream

# Container health
docker inspect --format='{{.State.Health.Status}}' go-backend
```

### Disk Usage

```bash
# Docker disk usage
docker system df

# PostgreSQL data size
du -sh /opt/go/data/postgres

# Log sizes
du -sh /var/lib/docker/containers/*/
```

## Backup Strategy

### Automated Backups

Backups run daily via cron:
```cron
0 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1
```

### Backup Contents

1. **PostgreSQL database** (pg_dump)
2. **Nextcloud database** (mysqldump)
3. **Traefik certificates** (/opt/traefik/acme)
4. **Configuration files** (.env, docker-compose.yml)
5. **Nextcloud data volume**

### Backup Retention

- Keep backups for 30 days
- Stored in `/opt/backups/b0esche_cloud/`
- Compressed as `.tar.gz`

### Manual Backup

```bash
# Run backup now
/opt/scripts/backup.sh

# List backups
ls -lh /opt/backups/b0esche_cloud/
```

### Restore Procedure

```bash
# 1. Stop services
cd /opt/go && docker-compose down
cd /opt/flutter && docker-compose down

# 2. Extract backup
cd /opt/backups/b0esche_cloud
tar -xzf 20260113_020000.tar.gz

# 3. Restore database
docker exec -i go-postgres pg_restore -U go_backend -d go_backend < go_backend.sqlc

# 4. Restore configurations
cp .env.production /opt/go/
cp go-docker-compose.yml /opt/go/docker-compose.yml

# 5. Restart services
cd /opt/go && docker-compose up -d
cd /opt/flutter && docker-compose up -d
```

## Troubleshooting

### Common Issues

#### Service won't start

```bash
# Check logs for errors
docker logs go-backend --tail 50

# Check container status
docker inspect go-backend | jq '.[0].State'

# Check port conflicts
netstat -tlnp | grep -E '80|443|8080'
```

#### Database connection issues

```bash
# Test database connectivity
docker exec go-backend ping -c 3 postgres

# Check PostgreSQL logs
docker logs go-postgres --tail 50

# Verify credentials
docker exec go-postgres psql -U go_backend -c "SELECT 1"
```

#### SSL certificate errors

```bash
# Check certificate status
curl -vI https://www.b0esche.cloud 2>&1 | grep -A 5 "Server certificate"

# Force renewal
docker restart traefik
sleep 60
curl -vI https://www.b0esche.cloud
```

#### Out of disk space

```bash
# Check disk usage
df -h

# Clean Docker resources
docker system prune -a --volumes

# Clean old backups
find /opt/backups -mtime +30 -delete

# Clean old logs
truncate -s 0 /var/log/auto-deploy.log
```

### Emergency Procedures

#### Rollback Deployment

```bash
# 1. Stop current services
cd /opt/go && docker-compose down
cd /opt/flutter && docker-compose down

# 2. Checkout previous version
cd /opt/auto-deploy/b0esche_cloud_rollout
git log --oneline -10  # Find last working commit
git checkout <commit-hash>

# 3. Redeploy
/opt/scripts/auto-deploy.sh
```

#### Database Recovery

```bash
# Find latest backup
ls -lt /opt/backups/b0esche_cloud/ | head -5

# Restore (see Restore Procedure above)
```

#### Full System Recovery

1. Provision new server
2. Install Docker
3. Copy `/opt` from backup
4. Start services in order
5. Restore database from backup
6. Verify health checks

## Security Checklist

- [ ] All services behind Traefik (no direct port exposure)
- [ ] SSL certificates valid and auto-renewing
- [ ] Database not accessible from internet
- [ ] Strong passwords in `.env.production`
- [ ] Regular backups verified
- [ ] Firewall configured (only 80, 443, 22 open)
- [ ] SSH key authentication only
- [ ] Auto-deploy logs monitored

## Performance Tuning

### PostgreSQL

```sql
-- Check slow queries
SELECT * FROM pg_stat_activity WHERE state = 'active';

-- Analyze tables
ANALYZE;

-- Vacuum
VACUUM ANALYZE;
```

### Nginx

```nginx
# Add to nginx.conf for better performance
worker_connections 1024;
keepalive_timeout 65;
gzip_comp_level 6;
```

### Docker

```bash
# Limit container resources
docker update --memory="512m" --cpus="1" go-backend

# Clean up unused resources
docker system prune -f
```