Add docs, scripts, and update README
- Added docs/AUTH.md with authentication system documentation - Added server scripts (auto-deploy, backup, monitor, webhook-server) - Updated README with deployment info and project structure - Added gitignore for backup archives
This commit is contained in:
Leon Bösche
2
.gitignore
vendored
Normal file
2
.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
*.tar.gz
|
||||
*_bak.tar.gz
|
||||
254
README.md
254
README.md
@@ -1,149 +1,227 @@
|
||||
# b0esche.cloud
|
||||
|
||||
A self-hosted, SaaS-style document platform with Go backend and Flutter web frontend.
|
||||
A self-hosted, SaaS-style cloud storage and document platform with a Go backend and Flutter web frontend.
|
||||
|
||||
🌐 **Live:** [b0esche.cloud](https://b0esche.cloud)
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Flutter Web │────▶│ Go Backend │────▶│ PostgreSQL │
|
||||
│ (b0esche_cloud)│ │ (go_cloud) │ │ │
|
||||
└─────────────────┘ └────────┬────────┘ └─────────────────┘
|
||||
│
|
||||
┌────────────┼────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│Nextcloud │ │Collabora │ │ Traefik │
|
||||
│(Storage) │ │ (Office) │ │ (Proxy) │
|
||||
└──────────┘ └──────────┘ └──────────┘
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
- `go_cloud/`: Go backend (control plane) with REST API
|
||||
- `b0esche_cloud/`: Flutter web frontend with BLoC architecture
|
||||
- Supporting services: Nextcloud (storage), Collabora (editing), PostgreSQL (database)
|
||||
```
|
||||
b0esche_cloud/
|
||||
├── b0esche_cloud/ # Flutter web frontend
|
||||
│ ├── lib/
|
||||
│ │ ├── blocs/ # BLoC state management
|
||||
│ │ ├── models/ # Data models
|
||||
│ │ ├── pages/ # UI pages
|
||||
│ │ ├── repositories/ # Data repositories
|
||||
│ │ ├── services/ # API services
|
||||
│ │ ├── theme/ # App theming
|
||||
│ │ ├── viewmodels/ # View models
|
||||
│ │ └── widgets/ # Reusable widgets
|
||||
│ └── web/ # Web assets
|
||||
├── go_cloud/ # Go backend
|
||||
│ ├── cmd/api/ # Main entry point
|
||||
│ ├── internal/
|
||||
│ │ ├── auth/ # Authentication (OIDC, Passkeys)
|
||||
│ │ ├── files/ # File management
|
||||
│ │ ├── org/ # Organization management
|
||||
│ │ ├── storage/ # Nextcloud/WebDAV integration
|
||||
│ │ ├── http/ # HTTP handlers & WOPI
|
||||
│ │ └── ...
|
||||
│ ├── migrations/ # Database migrations
|
||||
│ └── pkg/jwt/ # JWT utilities
|
||||
├── scripts/ # Deployment & operations scripts
|
||||
└── docs/ # Documentation
|
||||
└── AUTH.md # Authentication system docs
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- 🔐 **Authentication**: OIDC via Nextcloud + WebAuthn Passkeys
|
||||
- 📁 **File Management**: Upload, download, organize files
|
||||
- 👥 **Organizations**: Multi-tenant with roles (Owner, Admin, Member)
|
||||
- 📝 **Document Viewing**: PDF viewer, Office document preview
|
||||
- 🔄 **Real-time Sync**: Nextcloud/WebDAV backend storage
|
||||
- 🚀 **Auto-deployment**: Daily 3AM deployments via GitLab webhooks
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Go 1.21+
|
||||
- Flutter 3.10+
|
||||
- Docker and Docker Compose
|
||||
- PostgreSQL (or Docker)
|
||||
- Nextcloud instance
|
||||
- Collabora Online instance
|
||||
- Docker & Docker Compose
|
||||
- PostgreSQL 15+
|
||||
|
||||
## Local Development Setup
|
||||
## Local Development
|
||||
|
||||
### 1. Start Supporting Services
|
||||
|
||||
Use Docker Compose to start PostgreSQL, Nextcloud, and Collabora:
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
docker-compose up -d db nextcloud collabora
|
||||
# Start everything
|
||||
./scripts/dev-all.sh
|
||||
```
|
||||
|
||||
### 2. Backend Setup
|
||||
### Manual Setup
|
||||
|
||||
**Backend:**
|
||||
```bash
|
||||
cd go_cloud
|
||||
cp .env.example .env
|
||||
# Edit .env with your configuration (DB URL, Nextcloud URL, etc.)
|
||||
# Edit .env with your configuration
|
||||
go run ./cmd/api
|
||||
```
|
||||
|
||||
Or use the provided script:
|
||||
|
||||
```bash
|
||||
./scripts/dev-backend.sh
|
||||
```
|
||||
|
||||
### 3. Frontend Setup
|
||||
|
||||
**Frontend:**
|
||||
```bash
|
||||
cd b0esche_cloud
|
||||
flutter pub get
|
||||
flutter run -d chrome
|
||||
```
|
||||
|
||||
Or use the script:
|
||||
|
||||
```bash
|
||||
./scripts/dev-frontend.sh
|
||||
```
|
||||
|
||||
### 4. Full Development Environment
|
||||
|
||||
To start everything:
|
||||
|
||||
```bash
|
||||
./scripts/dev-all.sh
|
||||
```
|
||||
|
||||
This will bring up all services, backend, and frontend.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Backend (.env)
|
||||
### Backend Environment Variables
|
||||
|
||||
Copy `go_cloud/.env.example` to `go_cloud/.env` and fill in:
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `SERVER_ADDR` | Server address (default: `:8080`) |
|
||||
| `DATABASE_URL` | PostgreSQL connection string |
|
||||
| `JWT_SECRET` | Secret for JWT signing |
|
||||
| `OIDC_ISSUER_URL` | OIDC provider URL |
|
||||
| `OIDC_CLIENT_ID` | OIDC client ID |
|
||||
| `OIDC_CLIENT_SECRET` | OIDC client secret |
|
||||
| `NEXTCLOUD_URL` | Nextcloud instance URL |
|
||||
| `NEXTCLOUD_USERNAME` | Nextcloud admin username |
|
||||
| `NEXTCLOUD_PASSWORD` | Nextcloud admin password |
|
||||
| `COLLABORA_URL` | Collabora Online URL |
|
||||
|
||||
- `DATABASE_URL`: PostgreSQL connection string
|
||||
- `JWT_SECRET`: Random secret for JWT signing
|
||||
- `OIDC_*`: OIDC provider settings
|
||||
- `NEXTCLOUD_*`: Nextcloud API settings
|
||||
- `COLLABORA_*`: Collabora settings
|
||||
## Production Deployment
|
||||
|
||||
### Frontend
|
||||
The project runs on a VPS with Docker containers behind Traefik reverse proxy.
|
||||
|
||||
The frontend uses build-time environment variables for API base URL. For dev, it's hardcoded in `ApiClient` constructor.
|
||||
### Services & Domains
|
||||
|
||||
For production builds, update accordingly.
|
||||
| Domain | Service |
|
||||
|--------|---------|
|
||||
| `www.b0esche.cloud` | Flutter Web (Nginx) |
|
||||
| `go.b0esche.cloud` | Go API Backend |
|
||||
| `storage.b0esche.cloud` | Nextcloud (Storage + OIDC) |
|
||||
| `of.b0esche.cloud` | Collabora Online (Office) |
|
||||
|
||||
## Running Tests
|
||||
### Server Directory Structure
|
||||
|
||||
### Backend
|
||||
|
||||
```bash
|
||||
cd go_cloud
|
||||
go test ./...
|
||||
```
|
||||
/opt/
|
||||
├── traefik/ # Reverse proxy + SSL
|
||||
├── go/ # Go backend + PostgreSQL
|
||||
├── flutter/ # Flutter web build + Nginx
|
||||
├── scripts/ # Operations scripts
|
||||
└── auto-deploy/ # Auto-deployment workspace
|
||||
```
|
||||
|
||||
### Frontend
|
||||
### Server Scripts
|
||||
|
||||
| Script | Description |
|
||||
|--------|-------------|
|
||||
| `auto-deploy.sh` | Daily automated deployment (runs at 3AM) |
|
||||
| `deploy-now.sh` | Trigger immediate deployment |
|
||||
| `backup.sh` | Full backup (DB, configs, volumes) |
|
||||
| `monitor.sh` | Health monitoring & alerts |
|
||||
| `webhook-server.py` | GitLab webhook receiver |
|
||||
|
||||
### Deployment Commands
|
||||
|
||||
```bash
|
||||
cd b0esche_cloud
|
||||
flutter test
|
||||
# Trigger immediate deploy
|
||||
ssh b0esche-cloud '/opt/scripts/deploy-now.sh'
|
||||
|
||||
# Check backend logs
|
||||
ssh b0esche-cloud 'docker logs go-backend -f'
|
||||
|
||||
# Check service status
|
||||
ssh b0esche-cloud 'docker ps --format "table {{.Names}}\t{{.Status}}"'
|
||||
|
||||
# Health checks
|
||||
curl -s https://go.b0esche.cloud/health
|
||||
curl -s https://www.b0esche.cloud | grep -o '<title>.*</title>'
|
||||
```
|
||||
|
||||
## Building for Production
|
||||
|
||||
### Backend
|
||||
### Starting Services (Manual)
|
||||
|
||||
```bash
|
||||
cd go_cloud
|
||||
go build -o bin/api ./cmd/api
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
```bash
|
||||
cd b0esche_cloud
|
||||
flutter build web
|
||||
# Start all services in order
|
||||
ssh b0esche-cloud 'cd /opt/traefik && docker-compose up -d'
|
||||
ssh b0esche-cloud 'cd /opt/go && docker-compose up -d'
|
||||
ssh b0esche-cloud 'cd /opt/flutter && docker-compose up -d'
|
||||
```
|
||||
|
||||
## Database Migrations
|
||||
|
||||
Migrations are in `go_cloud/migrations/`.
|
||||
|
||||
To apply:
|
||||
Migrations are in `go_cloud/migrations/`:
|
||||
|
||||
```bash
|
||||
# Dev
|
||||
cd go_cloud
|
||||
go run github.com/pressly/goose/v3/cmd/goose@latest postgres "$DATABASE_URL" up
|
||||
|
||||
# Production
|
||||
# Use your deployment tool to run the migration command
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
## Backup & Recovery
|
||||
|
||||
- **Database**: Regular PostgreSQL dumps of orgs, memberships, activities
|
||||
- **Files**: Nextcloud/S3 backups handled at storage layer
|
||||
- **Recovery**: Restore DB, then files; Go control plane is stateless
|
||||
Backups run daily and include:
|
||||
- PostgreSQL database dumps
|
||||
- Nextcloud database
|
||||
- Traefik certificates
|
||||
- Docker volumes
|
||||
- Configuration files
|
||||
|
||||
## Contributing
|
||||
Backups are retained for 30 days.
|
||||
|
||||
1. Clone the repo
|
||||
2. Follow local setup
|
||||
3. Make changes
|
||||
4. Run tests
|
||||
5. Submit PR
|
||||
```bash
|
||||
# Manual backup
|
||||
ssh b0esche-cloud '/opt/scripts/backup.sh'
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Backend tests
|
||||
cd go_cloud && go test ./...
|
||||
|
||||
# Frontend tests
|
||||
cd b0esche_cloud && flutter test
|
||||
```
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Component | Technology |
|
||||
|-----------|------------|
|
||||
| Frontend | Flutter Web, BLoC |
|
||||
| Backend | Go, Chi Router |
|
||||
| Database | PostgreSQL |
|
||||
| Storage | Nextcloud (WebDAV) |
|
||||
| Office | Collabora Online |
|
||||
| Auth | OIDC, WebAuthn |
|
||||
| Proxy | Traefik |
|
||||
| CI/CD | GitLab + Webhooks |
|
||||
|
||||
## Documentation
|
||||
|
||||
- [AUTH.md](docs/AUTH.md) - Complete authentication system documentation (Passkeys, OIDC, roles)
|
||||
|
||||
## License
|
||||
|
||||
[License here]
|
||||
Private project - All rights reserved
|
||||
|
||||
Binary file not shown.
279
docs/AUTH.md
Normal file
279
docs/AUTH.md
Normal file
@@ -0,0 +1,279 @@
|
||||
# b0esche.cloud Authentication System
|
||||
|
||||
This document describes the complete passkey-first authentication and authorization system for b0esche.cloud.
|
||||
|
||||
## Overview
|
||||
|
||||
b0esche.cloud implements a modern, secure, username-only, passkey-first authentication system with comprehensive admin functionality and recovery options.
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
### Primary Authentication: Passkeys (WebAuthn)
|
||||
|
||||
- **Username + Passkey Registration**: Users create an account with just a username and register a passkey
|
||||
- **Passkey-First Login**: Primary authentication method using WebAuthn/FIDO2 standards
|
||||
- **Device Support**: Works with Touch ID, Windows Hello, YubiKey, and other FIDO2 authenticators
|
||||
- **Multiple Passkeys**: Users can register multiple devices for redundancy
|
||||
|
||||
### Fallback Options
|
||||
|
||||
- **Optional Password**: Users can add a password as a fallback authentication method
|
||||
- **Recovery Codes**: 10 single-use recovery codes generated per user
|
||||
- **Admin Recovery**: Admins can assist with account recovery if needed
|
||||
|
||||
## User Roles and Permissions
|
||||
|
||||
### Role Hierarchy
|
||||
|
||||
1. **superadmin** (Level 3): Full system access
|
||||
- User management and role assignments
|
||||
- Admin invitation system
|
||||
- System configuration access
|
||||
- All organization management
|
||||
|
||||
2. **admin** (Level 2): Administrative access
|
||||
- Organization management
|
||||
- User role promotion (within their orgs)
|
||||
- Activity monitoring
|
||||
- File management across organizations
|
||||
|
||||
3. **user** (Level 1): Standard user access
|
||||
- Personal file management
|
||||
- Organization membership
|
||||
- Basic account settings
|
||||
|
||||
## Bootstrap Process
|
||||
|
||||
### Creating the First Administrator
|
||||
|
||||
The system includes a bootstrap utility to create the first superadmin user securely:
|
||||
|
||||
```bash
|
||||
# Interactive mode (recommended for production)
|
||||
./bin/bootstrap
|
||||
|
||||
# Environment variable mode (recommended for automation)
|
||||
export BOOTSTRAP_ADMIN_USERNAME=admin
|
||||
export BOOTSTRAP_ADMIN_PASSWORD=secure_password_123
|
||||
./bin/bootstrap
|
||||
```
|
||||
|
||||
#### Bootstrap Process
|
||||
|
||||
1. **Check Existing Admin**: Verifies no superadmin user already exists
|
||||
2. **Secure Input**: Prompts for username and password (or reads from environment)
|
||||
3. **Password Security**: Enforces minimum 8-character passwords
|
||||
4. **Role Assignment**: Automatically assigns superadmin role
|
||||
5. **Secure Storage**: Passwords are bcrypt-hashed before storage
|
||||
|
||||
#### Security Guidelines
|
||||
|
||||
- **Immediate Action**: Log in immediately after bootstrap to register a passkey
|
||||
- **Passkey Priority**: Set up passkeys and consider removing the password
|
||||
- **Recovery Setup**: Generate recovery codes during first login
|
||||
- **Credential Rotation**: Never leave default credentials in production
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Authentication Endpoints
|
||||
|
||||
```
|
||||
POST /auth/passkey/register/start
|
||||
POST /auth/passkey/register/verify
|
||||
POST /auth/passkey/login/start
|
||||
POST /auth/passkey/login/verify
|
||||
```
|
||||
|
||||
### Device Management
|
||||
|
||||
```
|
||||
GET /auth/passkey/devices # List user's passkeys
|
||||
POST /auth/passkey/devices/add # Add new passkey
|
||||
DELETE /auth/passkey/devices/{id} # Remove passkey
|
||||
```
|
||||
|
||||
### Recovery System
|
||||
|
||||
```
|
||||
POST /auth/recovery/codes/generate # Generate recovery codes
|
||||
POST /auth/recovery/codes/use # Use recovery code
|
||||
DELETE /auth/recovery/codes/revoke # Revoke all codes
|
||||
```
|
||||
|
||||
### Admin Operations
|
||||
|
||||
```
|
||||
GET /auth/admin/invitations # List invitations
|
||||
POST /auth/admin/invitations # Create invitation
|
||||
POST /auth/admin/invitations/accept # Accept invitation
|
||||
DELETE /auth/admin/invitations/{id} # Revoke invitation
|
||||
```
|
||||
|
||||
### Password Fallback
|
||||
|
||||
```
|
||||
POST /auth/password/add # Add password to account
|
||||
DELETE /auth/password/remove # Remove password from account
|
||||
```
|
||||
|
||||
## Environment Configuration
|
||||
|
||||
```bash
|
||||
# Server Configuration
|
||||
SERVER_ADDR=:8080
|
||||
DATABASE_URL=postgresql://user:pass@localhost/db
|
||||
|
||||
# WebAuthn Configuration
|
||||
WEBAUTHN_RP_ID=www.b0esche.cloud
|
||||
WEBAUTHN_RP_NAME=b0esche.cloud
|
||||
WEBAUTHN_RP_ORIGIN=https://www.b0esche.cloud
|
||||
|
||||
# Security Configuration
|
||||
JWT_SECRET=your_jwt_secret_key
|
||||
PASSKEY_TIMEOUT=300000
|
||||
RECOVERY_CODE_EXPIRY=86400
|
||||
MAX_RECOVERY_CODES=10
|
||||
```
|
||||
|
||||
## Security Features
|
||||
|
||||
### WebAuthn Security
|
||||
|
||||
- **FIDO2/WebAuthn Compliant**: Full compliance with WebAuthn Level 2
|
||||
- **Origin Binding**: Credentials bound to specific domain
|
||||
- **Challenge-Response**: Cryptographic challenge verification
|
||||
- **Device Attestation**: Optional device verification
|
||||
- **Public Key Crypto**: Asymmetric cryptography with private keys never leaving devices
|
||||
|
||||
### Account Security
|
||||
|
||||
- **Rate Limiting**: Built-in protection against brute force attacks
|
||||
- **Session Management**: Secure HTTP-only sessions with configurable expiration
|
||||
- **Audit Logging**: Comprehensive logging of all authentication and admin actions
|
||||
- **Role-Based Access Control**: Hierarchical permission system
|
||||
|
||||
### Data Protection
|
||||
|
||||
- **Password Hashing**: bcrypt with automatic salt generation
|
||||
- **Recovery Code Hashing**: Secure one-way hashing of recovery codes
|
||||
- **No Plaintext Storage**: No sensitive data stored in plaintext
|
||||
- **Input Validation**: Comprehensive input sanitization and validation
|
||||
|
||||
## Database Schema
|
||||
|
||||
The authentication system uses the following key tables:
|
||||
|
||||
- **users**: User accounts with role-based access control
|
||||
- **roles**: Hierarchical role definitions
|
||||
- **passkeys**: WebAuthn credential storage
|
||||
- **recovery_codes**: One-time recovery codes
|
||||
- **admin_invitations**: Secure admin invitation system
|
||||
- **sessions**: Secure session management
|
||||
|
||||
## Development and Testing
|
||||
|
||||
### Running Locally
|
||||
|
||||
1. **Setup Database**: Ensure PostgreSQL is running with migrations applied
|
||||
2. **Bootstrap Admin**: Run the bootstrap command to create first admin
|
||||
3. **Start Server**: Run the API server with appropriate environment
|
||||
4. **Test Authentication**: Use the Flutter app or API tests
|
||||
|
||||
### Testing WebAuthn
|
||||
|
||||
WebAuthn requires HTTPS in production browsers. For local testing:
|
||||
|
||||
```bash
|
||||
# Use mkcert for local HTTPS
|
||||
mkcert -install
|
||||
mkcert localhost 127.0.0.1 ::1
|
||||
|
||||
# Set environment for local development
|
||||
export WEBAUTHN_RP_ID=localhost
|
||||
export WEBAUTHN_RP_ORIGIN=https://localhost:8080
|
||||
```
|
||||
|
||||
## User Experience
|
||||
|
||||
### Signup Flow
|
||||
|
||||
1. **Username Selection**: User chooses a unique username
|
||||
2. **Real-time Validation**: Immediate feedback on username availability
|
||||
3. **Passkey Creation**: Browser prompts for passkey registration
|
||||
4. **Account Creation**: Automatic account creation with passkey
|
||||
|
||||
### Login Flow
|
||||
|
||||
1. **Passkey Detection**: System shows available passkeys
|
||||
2. **Biometric Prompt**: Browser authenticates with passkey
|
||||
3. **Session Creation**: Secure session established
|
||||
4. **Redirect**: User directed to dashboard
|
||||
|
||||
### Security Settings
|
||||
|
||||
Users can manage their security through the Settings > Security page:
|
||||
|
||||
- **Device Management**: View, add, remove, and label passkeys
|
||||
- **Recovery Codes**: Generate new recovery codes
|
||||
- **Password Options**: Add or remove password fallback
|
||||
- **Account Recovery**: Secure account recovery options
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **WebAuthn Not Supported**: Use a modern browser (Chrome, Firefox, Safari, Edge)
|
||||
2. **HTTPS Required**: WebAuthn requires HTTPS in production environments
|
||||
3. **Device Compatibility**: Ensure devices support FIDO2/WebAuthn
|
||||
4. **Database Connection**: Verify database connection and migrations
|
||||
|
||||
### Bootstrap Issues
|
||||
|
||||
1. **Permission Denied**: Ensure proper file permissions on bootstrap binary
|
||||
2. **Database Connection**: Check DATABASE_URL configuration
|
||||
3. **Port Conflicts**: Ensure database port is accessible
|
||||
|
||||
### Recovery Process
|
||||
|
||||
If a user loses access to all passkeys:
|
||||
|
||||
1. **Use Recovery Code**: Enter one of the 10 recovery codes
|
||||
2. **Contact Admin**: Admins can assist with account recovery
|
||||
3. **Re-register Passkey**: Set up new passkeys after recovery
|
||||
4. **Generate New Recovery Codes**: Replace used recovery codes
|
||||
|
||||
## Monitoring and Maintenance
|
||||
|
||||
### Key Metrics
|
||||
|
||||
Monitor these metrics for system health:
|
||||
|
||||
- Authentication success/failure rates
|
||||
- Passkey registration and usage patterns
|
||||
- Recovery code usage frequency
|
||||
- Admin action audit logs
|
||||
- Session expiration and renewal rates
|
||||
|
||||
### Maintenance Tasks
|
||||
|
||||
Regular maintenance includes:
|
||||
|
||||
- Clean up expired sessions and recovery codes
|
||||
- Review audit logs for suspicious activity
|
||||
- Update role assignments as needed
|
||||
- Monitor WebAuthn compatibility with browser updates
|
||||
|
||||
## Support and Documentation
|
||||
|
||||
For additional support:
|
||||
|
||||
- **Technical Issues**: Check application logs and database status
|
||||
- **Security Concerns**: Review audit logs and user activity
|
||||
- **Feature Requests**: Follow the contribution guidelines
|
||||
- **Documentation Updates**: Keep this document current with system changes
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: January 2026
|
||||
**Version**: 1.0
|
||||
**Compatibility**: WebAuthn Level 2, FIDO2
|
||||
Binary file not shown.
101
scripts/auto-deploy.sh
Executable file
101
scripts/auto-deploy.sh
Executable file
@@ -0,0 +1,101 @@
|
||||
#!/bin/bash
|
||||
# Daily auto-deploy script for Flutter/Go projects
|
||||
# Runs at 3AM daily - no more disruptive deployments!
|
||||
# Run as root/admin - no extra users needed!
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
GIT_REPO="https://lab.b0esche.cloud/b0esche/b0esche_cloud.git"
|
||||
DEPLOY_DIR="/opt/auto-deploy/b0esche_cloud_rollout"
|
||||
BUILD_DIR="/opt/go/data/postgres/backend/go_cloud"
|
||||
LOG_FILE="/var/log/auto-deploy.log"
|
||||
TIMEOUT=1800 # 30 minutes for Flutter build (increased from default)
|
||||
SKIP_IF_RECENT_HOURS=2 # Skip if deployed within last 2 hours
|
||||
|
||||
# Logging
|
||||
exec > >(tee -a "$LOG_FILE") 2>&1
|
||||
|
||||
# Check if we should skip deployment
|
||||
LAST_DEPLOY_FILE="/tmp/last_deploy_time"
|
||||
if [ -f "$LAST_DEPLOY_FILE" ]; then
|
||||
LAST_DEPLOY=$(cat "$LAST_DEPLOY_FILE")
|
||||
CURRENT_TIME=$(date +%s)
|
||||
if [ $((CURRENT_TIME - LAST_DEPLOY)) -lt $((SKIP_IF_RECENT_HOURS * 3600)) ]; then
|
||||
echo "=== Skipping deployment - recent deploy was less than $SKIP_IF_RECENT_HOURS hours ago ==="
|
||||
echo "=== Auto-deploy skipped at $(date) ==="
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "=== Scheduled auto-deploy started at $(date) ==="
|
||||
|
||||
# Create deploy directory if it doesn't exist
|
||||
mkdir -p "$DEPLOY_DIR"
|
||||
cd "$DEPLOY_DIR"
|
||||
|
||||
echo "Pulling latest changes..."
|
||||
if [ -d ".git" ]; then
|
||||
git fetch origin
|
||||
git reset --hard origin/main
|
||||
else
|
||||
git clone "$GIT_REPO" .
|
||||
fi
|
||||
|
||||
echo "=== Deploying Backend ==="
|
||||
# Copy backend code to build directory
|
||||
echo "Copying backend code to build directory..."
|
||||
mkdir -p "$BUILD_DIR"
|
||||
cp -r go_cloud/* "$BUILD_DIR/"
|
||||
|
||||
# Build and start backend from traefik directory
|
||||
cd /opt/traefik
|
||||
echo "Building go-backend container..."
|
||||
docker-compose build --no-cache go-backend
|
||||
echo "Recreating go-backend container with fresh environment..."
|
||||
docker-compose up -d --force-recreate go-backend
|
||||
|
||||
echo "Backend deployed successfully!"
|
||||
|
||||
echo "=== Deploying Frontend ==="
|
||||
# Build Flutter for web with proper timeout
|
||||
echo "Building Flutter web app..."
|
||||
FRONTEND_DIR="$DEPLOY_DIR/b0esche_cloud"
|
||||
cd "$FRONTEND_DIR"
|
||||
timeout 900 sudo -u admin /opt/flutter/bin/flutter build web --release || {
|
||||
echo "Flutter build failed or timed out"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Copy built files to nginx volume
|
||||
echo "Copying built files to web server..."
|
||||
rm -rf /opt/traefik/web/*
|
||||
cp -r build/web/* /opt/traefik/web/
|
||||
|
||||
# Restart nginx container
|
||||
echo "Restarting flutter-web container..."
|
||||
cd /opt/traefik
|
||||
docker-compose up -d --force-recreate flutter-web
|
||||
|
||||
echo "=== Deployment completed successfully at $(date) ==="
|
||||
|
||||
# Record deployment time
|
||||
date +%s > "$LAST_DEPLOY_FILE"
|
||||
|
||||
# Health checks
|
||||
echo "=== Running health checks ==="
|
||||
sleep 10
|
||||
|
||||
if curl -f -s https://go.b0esche.cloud/health > /dev/null 2>&1; then
|
||||
echo "✅ Backend health check passed"
|
||||
else
|
||||
echo "❌ Backend health check failed"
|
||||
fi
|
||||
|
||||
if curl -f -s https://www.b0esche.cloud > /dev/null 2>&1; then
|
||||
echo "✅ Frontend health check passed"
|
||||
else
|
||||
echo "❌ Frontend health check failed"
|
||||
fi
|
||||
|
||||
echo "=== Scheduled auto-deploy completed ==="
|
||||
50
scripts/backup.sh
Executable file
50
scripts/backup.sh
Executable file
@@ -0,0 +1,50 @@
|
||||
#!/bin/bash
|
||||
# b0esche.cloud Backup Script
|
||||
# Usage: ./backup.sh
|
||||
|
||||
set -e
|
||||
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_DIR="/opt/backups/b0esche_cloud/$DATE"
|
||||
RETENTION_DAYS=30
|
||||
|
||||
echo "Starting backup for b0esche.cloud - $DATE"
|
||||
|
||||
# Create backup directory
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
echo "Backing up Go backend PostgreSQL database..."
|
||||
docker exec go-postgres pg_dump -U go_backend -Fc go_backend > "$BACKUP_DIR/go_backend.sqlc"
|
||||
|
||||
echo "Backing up Nextcloud database..."
|
||||
docker exec nextcloud-db mysqldump -u nextcloud -pSu11Fd02!!! nextcloud > "$BACKUP_DIR/nextcloud.sql"
|
||||
|
||||
echo "Backing up Traefik certificates..."
|
||||
cp -r /opt/traefik/acme "$BACKUP_DIR/"
|
||||
|
||||
echo "Backing up configuration files..."
|
||||
cp /opt/go/.env.production "$BACKUP_DIR/"
|
||||
cp /opt/go/docker-compose.yml "$BACKUP_DIR/go-docker-compose.yml"
|
||||
cp /opt/flutter/docker-compose.yml "$BACKUP_DIR/flutter-docker-compose.yml"
|
||||
cp /opt/flutter/nginx.conf "$BACKUP_DIR/"
|
||||
cp /opt/traefik/docker-compose.yml "$BACKUP_DIR/traefik-docker-compose.yml"
|
||||
cp /opt/traefik/traefik.yml "$BACKUP_DIR/"
|
||||
|
||||
echo "Backing up Docker volumes..."
|
||||
docker run --rm -v nextcloud_data_31:/data -v "$BACKUP_DIR":/backup alpine tar czf /backup/nextcloud_data.tar.gz -C /data .
|
||||
|
||||
echo "Compressing backup..."
|
||||
cd /opt/backups/b0esche_cloud
|
||||
tar czf "$DATE.tar.gz" "$DATE"
|
||||
rm -rf "$DATE"
|
||||
|
||||
echo "Backup completed: /opt/backups/b0esche_cloud/$DATE.tar.gz"
|
||||
|
||||
# Clean up old backups
|
||||
echo "Cleaning up backups older than $RETENTION_DAYS days..."
|
||||
find /opt/backups/b0esche_cloud -name "*.tar.gz" -mtime +$RETENTION_DAYS -delete
|
||||
|
||||
echo "Backup process completed successfully!"
|
||||
|
||||
# Show backup size
|
||||
ls -lh "/opt/backups/b0esche_cloud/$DATE.tar.gz"
|
||||
9
scripts/deploy-now.sh
Executable file
9
scripts/deploy-now.sh
Executable file
@@ -0,0 +1,9 @@
|
||||
#!/bin/bash
|
||||
# Manual deploy script - call this when you want immediate deployment
|
||||
# Usage: ./deploy-now.sh
|
||||
|
||||
echo "🚀 Starting immediate deployment..."
|
||||
|
||||
/opt/scripts/auto-deploy.sh
|
||||
|
||||
echo "✅ Manual deployment completed!"
|
||||
120
scripts/monitor.sh
Executable file
120
scripts/monitor.sh
Executable file
@@ -0,0 +1,120 @@
|
||||
#!/bin/bash
|
||||
# b0esche.cloud Monitoring Script
|
||||
# Usage: ./monitor.sh
|
||||
|
||||
set -e
|
||||
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
echo "=== b0esche.cloud Service Status ==="
|
||||
echo
|
||||
|
||||
# Check critical services
|
||||
services=("traefik" "go-backend" "go-postgres" "flutter-web" "nextcloud" "nextcloud-db" "collabora")
|
||||
|
||||
for service in "${services[@]}"; do
|
||||
if docker ps --format "table {{.Names}}" | grep -q "^$service$"; then
|
||||
echo -e "${GREEN}✓${NC} $service is running"
|
||||
else
|
||||
echo -e "${RED}✗${NC} $service is not running"
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
echo "=== Service Health Checks ==="
|
||||
echo
|
||||
|
||||
# HTTP/HTTPS health checks
|
||||
echo -n "Flutter Web (www.b0esche.cloud): "
|
||||
if curl -s --max-time 5 https://www.b0esche.cloud | grep -q "b0esche_cloud"; then
|
||||
echo -e "${GREEN}OK${NC}"
|
||||
else
|
||||
echo -e "${RED}FAILED${NC}"
|
||||
fi
|
||||
|
||||
echo -n "Go Backend (go.b0esche.cloud): "
|
||||
if curl -s --max-time 5 https://go.b0esche.cloud/health | grep -q "ok"; then
|
||||
echo -e "${GREEN}OK${NC}"
|
||||
else
|
||||
echo -e "${RED}FAILED${NC}"
|
||||
fi
|
||||
|
||||
echo -n "Nextcloud (storage.b0esche.cloud): "
|
||||
if curl -s --max-time 5 -I https://storage.b0esche.cloud | grep -q "HTTP/2 200"; then
|
||||
echo -e "${GREEN}OK${NC}"
|
||||
else
|
||||
echo -e "${RED}FAILED${NC}"
|
||||
fi
|
||||
|
||||
echo -n "Collabora (of.b0esche.cloud): "
|
||||
if curl -s --max-time 5 -I https://of.b0esche.cloud | grep -q "HTTP/2"; then
|
||||
echo -e "${GREEN}OK${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}DEGRADED${NC}"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "=== Resource Usage ==="
|
||||
echo
|
||||
|
||||
# Show container resource usage
|
||||
docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}" | grep -E "(traefik|go-backend|flutter-web|nextcloud|collabora)"
|
||||
|
||||
echo
|
||||
echo "=== Recent Error Logs ==="
|
||||
echo
|
||||
|
||||
# Show recent error logs from each service
|
||||
for service in traefik go-backend nextcloud collabora; do
|
||||
errors=$(docker logs "$service" --since=1h 2>&1 | grep -i error | tail -3 | wc -l)
|
||||
if [ "$errors" -gt 0 ]; then
|
||||
echo -e "${YELLOW}$service:${NC} $errors errors in last hour"
|
||||
docker logs "$service" --since=1h 2>&1 | grep -i error | tail -3 | sed 's/^/ /'
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
echo "=== SSL Certificate Status ==="
|
||||
echo
|
||||
|
||||
# Check certificate expiry for main domains
|
||||
domains=("www.b0esche.cloud" "go.b0esche.cloud" "storage.b0esche.cloud" "of.b0esche.cloud")
|
||||
|
||||
for domain in "${domains[@]}"; do
|
||||
expiry=$(echo | openssl s_client -servername "$domain" -connect "$domain:443" 2>/dev/null | openssl x509 -noout -dates 2>/dev/null | grep notAfter | cut -d= -f2)
|
||||
if [ -n "$expiry" ]; then
|
||||
expiry_epoch=$(date -d "$expiry" +%s)
|
||||
current_epoch=$(date +%s)
|
||||
days_left=$(( (expiry_epoch - current_epoch) / 86400 ))
|
||||
|
||||
if [ "$days_left" -lt 7 ]; then
|
||||
echo -e "${RED}$domain: ${NC}Expires in $days_left days"
|
||||
elif [ "$days_left" -lt 30 ]; then
|
||||
echo -e "${YELLOW}$domain: ${NC}Expires in $days_left days"
|
||||
else
|
||||
echo -e "${GREEN}$domain: ${NC}Expires in $days_left days"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}$domain: ${NC}Certificate check failed"
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
echo "=== Disk Usage ==="
|
||||
echo
|
||||
|
||||
# Show disk usage for critical directories
|
||||
echo "PostgreSQL data:"
|
||||
du -sh /opt/go/data/postgres 2>/dev/null || echo " Not accessible"
|
||||
|
||||
echo "Backup directory:"
|
||||
du -sh /opt/backups 2>/dev/null || echo " Not found"
|
||||
|
||||
echo "Docker volumes:"
|
||||
docker system df --format "table {{.Type}}\t{{.TotalCount}}\t{{.Size}}"
|
||||
|
||||
echo
|
||||
echo "=== Monitoring Complete ==="
|
||||
92
scripts/webhook-server.py
Executable file
92
scripts/webhook-server.py
Executable file
@@ -0,0 +1,92 @@
|
||||
#!/usr/bin/env python3
|
||||
# Simple webhook server for Gitea
|
||||
# Listens for push events and triggers auto-deploy
|
||||
|
||||
import http.server
|
||||
import socketserver
|
||||
import json
|
||||
import subprocess
|
||||
import os
|
||||
import hmac
|
||||
import hashlib
|
||||
from urllib.parse import urlparse
|
||||
|
||||
# Configuration
|
||||
PORT = 8080
|
||||
SECRET = 'your-webhook-secret' # Change this!
|
||||
DEPLOY_SCRIPT = '/opt/scripts/auto-deploy.sh'
|
||||
|
||||
class WebhookHandler(http.server.BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
if self.path == '/webhook':
|
||||
try:
|
||||
# Read the payload
|
||||
content_length = int(self.headers['Content-Length'])
|
||||
post_data = self.rfile.read(content_length)
|
||||
|
||||
# Verify signature (optional but recommended)
|
||||
signature = self.headers.get('X-Gitea-Signature', '')
|
||||
if signature:
|
||||
expected_sig = hmac.new(
|
||||
SECRET.encode(),
|
||||
post_data,
|
||||
hashlib.sha256
|
||||
).hexdigest()
|
||||
expected_sig = f'sha256={expected_sig}'
|
||||
|
||||
if not hmac.compare_digest(signature, expected_sig):
|
||||
self.send_response(401)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Invalid signature')
|
||||
return
|
||||
|
||||
# Parse the webhook payload
|
||||
data = json.loads(post_data.decode('utf-8'))
|
||||
|
||||
# Only deploy on manual trigger (with "deploy" in commit message)
|
||||
commit_message = data.get('commits', [{}])[0].get('message', '').lower()
|
||||
|
||||
if 'deploy now' in commit_message or 'manual deploy' in commit_message:
|
||||
print(f"Manual deploy requested: {commit_message}")
|
||||
|
||||
# Run the deploy script in background
|
||||
subprocess.Popen([
|
||||
'nohup', 'bash', '/opt/scripts/deploy-now.sh'
|
||||
], stdout=open('/var/log/webhook-deploy.log', 'a'),
|
||||
stderr=subprocess.STDOUT)
|
||||
|
||||
self.send_response(200)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Manual deployment triggered')
|
||||
else:
|
||||
print(f"Ignoring push - not a manual deploy trigger")
|
||||
self.send_response(200)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Push received - daily deploy scheduled for 3AM')
|
||||
|
||||
except Exception as e:
|
||||
print(f"Webhook error: {e}")
|
||||
self.send_response(500)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Internal server error')
|
||||
else:
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Not found')
|
||||
|
||||
def do_GET(self):
|
||||
if self.path == '/health':
|
||||
self.send_response(200)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'OK')
|
||||
else:
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Not found')
|
||||
|
||||
if __name__ == '__main__':
|
||||
os.makedirs('/var/log', exist_ok=True)
|
||||
|
||||
with socketserver.TCPServer(("", PORT), WebhookHandler) as httpd:
|
||||
print(f"Webhook server listening on port {PORT}")
|
||||
httpd.serve_forever()
|
||||
Reference in New Issue
Block a user