Add architecture, deployment, and development documentation for b0esche.cloud
This commit is contained in:
Leon Bösche
596
docs/API.md
Normal file
596
docs/API.md
Normal file
@@ -0,0 +1,596 @@
|
||||
# b0esche.cloud API Reference
|
||||
|
||||
Base URL: `https://go.b0esche.cloud`
|
||||
|
||||
## Authentication
|
||||
|
||||
All authenticated endpoints require a JWT token in the Authorization header:
|
||||
```
|
||||
Authorization: Bearer <token>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Health Check
|
||||
|
||||
### GET /health
|
||||
Check if the API is running.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"timestamp": "2026-01-13T19:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authentication Endpoints
|
||||
|
||||
### Passkey Registration
|
||||
|
||||
#### POST /auth/passkey/register/start
|
||||
Start passkey registration for a new user.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "johndoe"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"publicKey": {
|
||||
"challenge": "base64-encoded-challenge",
|
||||
"rp": {
|
||||
"name": "b0esche.cloud",
|
||||
"id": "www.b0esche.cloud"
|
||||
},
|
||||
"user": {
|
||||
"id": "base64-user-id",
|
||||
"name": "johndoe",
|
||||
"displayName": "johndoe"
|
||||
},
|
||||
"pubKeyCredParams": [...],
|
||||
"timeout": 300000,
|
||||
"attestation": "none"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /auth/passkey/register/verify
|
||||
Complete passkey registration.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "johndoe",
|
||||
"credential": {
|
||||
"id": "credential-id",
|
||||
"rawId": "base64-raw-id",
|
||||
"type": "public-key",
|
||||
"response": {
|
||||
"clientDataJSON": "base64-client-data",
|
||||
"attestationObject": "base64-attestation"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"user": {
|
||||
"id": "uuid",
|
||||
"username": "johndoe"
|
||||
},
|
||||
"token": "jwt-token"
|
||||
}
|
||||
```
|
||||
|
||||
### Passkey Login
|
||||
|
||||
#### POST /auth/passkey/login/start
|
||||
Start passkey authentication.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "johndoe"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"publicKey": {
|
||||
"challenge": "base64-challenge",
|
||||
"timeout": 300000,
|
||||
"rpId": "www.b0esche.cloud",
|
||||
"allowCredentials": [...]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /auth/passkey/login/verify
|
||||
Complete passkey authentication.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "johndoe",
|
||||
"credential": {
|
||||
"id": "credential-id",
|
||||
"rawId": "base64-raw-id",
|
||||
"type": "public-key",
|
||||
"response": {
|
||||
"clientDataJSON": "base64-client-data",
|
||||
"authenticatorData": "base64-auth-data",
|
||||
"signature": "base64-signature"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"user": {
|
||||
"id": "uuid",
|
||||
"username": "johndoe",
|
||||
"role": "user"
|
||||
},
|
||||
"token": "jwt-token"
|
||||
}
|
||||
```
|
||||
|
||||
### Device Management
|
||||
|
||||
#### GET /auth/passkey/devices
|
||||
List user's registered passkeys.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"devices": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"credentialId": "credential-id",
|
||||
"deviceLabel": "MacBook Pro",
|
||||
"createdAt": "2026-01-01T00:00:00Z",
|
||||
"lastUsedAt": "2026-01-13T19:00:00Z",
|
||||
"backupEligible": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /auth/passkey/devices/add
|
||||
Add a new passkey to existing account.
|
||||
|
||||
#### DELETE /auth/passkey/devices/{passkeyId}
|
||||
Remove a passkey from account.
|
||||
|
||||
### Recovery Codes
|
||||
|
||||
#### POST /auth/recovery/codes/generate
|
||||
Generate new recovery codes (invalidates old ones).
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"codes": [
|
||||
"XXXX-XXXX-XXXX",
|
||||
"YYYY-YYYY-YYYY",
|
||||
...
|
||||
],
|
||||
"expiresAt": "2027-01-13T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /auth/recovery/codes/use
|
||||
Use a recovery code to authenticate.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "johndoe",
|
||||
"code": "XXXX-XXXX-XXXX"
|
||||
}
|
||||
```
|
||||
|
||||
### Password (Optional Fallback)
|
||||
|
||||
#### POST /auth/password/add
|
||||
Add password to account.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"password": "secure-password"
|
||||
}
|
||||
```
|
||||
|
||||
#### DELETE /auth/password/remove
|
||||
Remove password from account.
|
||||
|
||||
---
|
||||
|
||||
## User Endpoints
|
||||
|
||||
### GET /api/me
|
||||
Get current user profile.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "uuid",
|
||||
"username": "johndoe",
|
||||
"email": "john@example.com",
|
||||
"displayName": "John Doe",
|
||||
"role": "user",
|
||||
"createdAt": "2026-01-01T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### PATCH /api/me
|
||||
Update user profile.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"displayName": "John D.",
|
||||
"email": "newemail@example.com"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Organization Endpoints
|
||||
|
||||
### GET /api/organizations
|
||||
List user's organizations.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"organizations": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "My Team",
|
||||
"slug": "my-team",
|
||||
"role": "owner",
|
||||
"memberCount": 5,
|
||||
"createdAt": "2026-01-01T00:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/organizations
|
||||
Create a new organization.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"name": "My New Team",
|
||||
"slug": "my-new-team"
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/organizations/{orgId}
|
||||
Get organization details.
|
||||
|
||||
### PATCH /api/organizations/{orgId}
|
||||
Update organization.
|
||||
|
||||
### DELETE /api/organizations/{orgId}
|
||||
Delete organization (owner only).
|
||||
|
||||
### GET /api/organizations/{orgId}/members
|
||||
List organization members.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"members": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"userId": "user-uuid",
|
||||
"username": "johndoe",
|
||||
"displayName": "John Doe",
|
||||
"role": "owner",
|
||||
"joinedAt": "2026-01-01T00:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/organizations/{orgId}/members
|
||||
Add member to organization.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"userId": "user-uuid",
|
||||
"role": "member"
|
||||
}
|
||||
```
|
||||
|
||||
### PATCH /api/organizations/{orgId}/members/{memberId}
|
||||
Update member role.
|
||||
|
||||
### DELETE /api/organizations/{orgId}/members/{memberId}
|
||||
Remove member from organization.
|
||||
|
||||
---
|
||||
|
||||
## File Endpoints
|
||||
|
||||
### GET /api/files
|
||||
List files in a directory.
|
||||
|
||||
**Query Parameters:**
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `path` | string | Directory path (default: `/`) |
|
||||
| `orgId` | string | Organization ID (optional) |
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"files": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "document.pdf",
|
||||
"path": "/documents/document.pdf",
|
||||
"type": "file",
|
||||
"mimeType": "application/pdf",
|
||||
"size": 1048576,
|
||||
"createdAt": "2026-01-01T00:00:00Z",
|
||||
"modifiedAt": "2026-01-13T19:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "photos",
|
||||
"path": "/photos",
|
||||
"type": "folder",
|
||||
"createdAt": "2026-01-01T00:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/files/upload
|
||||
Upload a file.
|
||||
|
||||
**Request:** `multipart/form-data`
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `file` | file | The file to upload |
|
||||
| `path` | string | Destination path |
|
||||
| `orgId` | string | Organization ID (optional) |
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"file": {
|
||||
"id": "uuid",
|
||||
"name": "uploaded-file.pdf",
|
||||
"path": "/documents/uploaded-file.pdf",
|
||||
"size": 1048576
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/files/download
|
||||
Download a file.
|
||||
|
||||
**Query Parameters:**
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `path` | string | File path |
|
||||
| `orgId` | string | Organization ID (optional) |
|
||||
|
||||
**Response:** File binary with appropriate Content-Type header.
|
||||
|
||||
### POST /api/files/folder
|
||||
Create a folder.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"path": "/new-folder",
|
||||
"orgId": "org-uuid"
|
||||
}
|
||||
```
|
||||
|
||||
### DELETE /api/files
|
||||
Delete a file or folder.
|
||||
|
||||
**Query Parameters:**
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `path` | string | Path to delete |
|
||||
| `orgId` | string | Organization ID (optional) |
|
||||
|
||||
### POST /api/files/move
|
||||
Move/rename a file or folder.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"sourcePath": "/old-name.pdf",
|
||||
"destinationPath": "/new-name.pdf",
|
||||
"orgId": "org-uuid"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/files/copy
|
||||
Copy a file or folder.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"sourcePath": "/original.pdf",
|
||||
"destinationPath": "/copy.pdf",
|
||||
"orgId": "org-uuid"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Admin Endpoints
|
||||
|
||||
*Requires admin or superadmin role.*
|
||||
|
||||
### GET /api/admin/users
|
||||
List all users.
|
||||
|
||||
**Query Parameters:**
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `page` | int | Page number (default: 1) |
|
||||
| `limit` | int | Items per page (default: 50) |
|
||||
| `search` | string | Search by username/email |
|
||||
|
||||
### GET /api/admin/users/{userId}
|
||||
Get user details.
|
||||
|
||||
### PATCH /api/admin/users/{userId}
|
||||
Update user (role, status).
|
||||
|
||||
### DELETE /api/admin/users/{userId}
|
||||
Delete user account.
|
||||
|
||||
### Admin Invitations
|
||||
|
||||
#### GET /auth/admin/invitations
|
||||
List admin invitations.
|
||||
|
||||
#### POST /auth/admin/invitations
|
||||
Create admin invitation.
|
||||
|
||||
**Request Body:**
|
||||
```json
|
||||
{
|
||||
"username": "newadmin",
|
||||
"roleId": "admin-role-uuid",
|
||||
"expiresIn": 86400
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"invitation": {
|
||||
"id": "uuid",
|
||||
"token": "invite-token",
|
||||
"expiresAt": "2026-01-14T19:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /auth/admin/invitations/{token}/accept
|
||||
Accept an admin invitation.
|
||||
|
||||
#### DELETE /auth/admin/invitations/{token}
|
||||
Revoke an invitation.
|
||||
|
||||
---
|
||||
|
||||
## Activity Endpoints
|
||||
|
||||
### GET /api/activities
|
||||
Get activity log.
|
||||
|
||||
**Query Parameters:**
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `page` | int | Page number |
|
||||
| `limit` | int | Items per page |
|
||||
| `orgId` | string | Filter by organization |
|
||||
| `userId` | string | Filter by user |
|
||||
| `action` | string | Filter by action type |
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"activities": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"userId": "user-uuid",
|
||||
"username": "johndoe",
|
||||
"action": "file.upload",
|
||||
"resourceType": "file",
|
||||
"resourceId": "/documents/report.pdf",
|
||||
"metadata": {
|
||||
"size": 1048576
|
||||
},
|
||||
"createdAt": "2026-01-13T19:00:00Z"
|
||||
}
|
||||
],
|
||||
"pagination": {
|
||||
"page": 1,
|
||||
"limit": 50,
|
||||
"total": 150
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Responses
|
||||
|
||||
All errors follow this format:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "ERROR_CODE",
|
||||
"message": "Human-readable error message",
|
||||
"details": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Common Error Codes
|
||||
|
||||
| Code | HTTP Status | Description |
|
||||
|------|-------------|-------------|
|
||||
| `UNAUTHORIZED` | 401 | Missing or invalid token |
|
||||
| `FORBIDDEN` | 403 | Insufficient permissions |
|
||||
| `NOT_FOUND` | 404 | Resource not found |
|
||||
| `VALIDATION_ERROR` | 400 | Invalid request data |
|
||||
| `CONFLICT` | 409 | Resource already exists |
|
||||
| `INTERNAL_ERROR` | 500 | Server error |
|
||||
|
||||
---
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
- **Authentication endpoints**: 10 requests/minute
|
||||
- **API endpoints**: 100 requests/minute
|
||||
- **File uploads**: 50 requests/hour
|
||||
|
||||
Rate limit headers:
|
||||
```
|
||||
X-RateLimit-Limit: 100
|
||||
X-RateLimit-Remaining: 95
|
||||
X-RateLimit-Reset: 1705172400
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Webhooks (Future)
|
||||
|
||||
Planned webhook events:
|
||||
- `user.created`
|
||||
- `user.deleted`
|
||||
- `file.uploaded`
|
||||
- `file.deleted`
|
||||
- `org.member.added`
|
||||
- `org.member.removed`
|
||||
313
docs/ARCHITECTURE.md
Normal file
313
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,313 @@
|
||||
# b0esche.cloud Architecture
|
||||
|
||||
## System Overview
|
||||
|
||||
b0esche.cloud is a self-hosted cloud storage platform inspired by Google Workspace, built with a modern microservices-style architecture.
|
||||
|
||||
## High-Level Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Internet │
|
||||
└─────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Traefik Reverse Proxy │
|
||||
│ (SSL Termination, Routing, Load Balancing) │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ www.* │ │ go.* │ │ storage.* │ │ of.* │ │
|
||||
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
|
||||
└─────────┼────────────────┼────────────────┼────────────────┼────────────────────────┘
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Flutter Web │ │ Go Backend │ │ Nextcloud │ │ Collabora │
|
||||
│ (Nginx) │ │ (API) │ │ (Storage) │ │ (Office) │
|
||||
└──────────────┘ └──────┬───────┘ └──────────────┘ └──────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ PostgreSQL │
|
||||
│ (Database) │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
### 1. Flutter Web Frontend (`b0esche_cloud/`)
|
||||
|
||||
The user-facing web application built with Flutter.
|
||||
|
||||
**Technology Stack:**
|
||||
- Flutter 3.x with Dart
|
||||
- BLoC pattern for state management
|
||||
- Material Design 3 theming
|
||||
|
||||
**Key Modules:**
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `blocs/` | Business logic components (auth, files, orgs) |
|
||||
| `models/` | Data models (User, File, Organization) |
|
||||
| `pages/` | UI screens (Home, Files, Settings, Admin) |
|
||||
| `repositories/` | Data access layer |
|
||||
| `services/` | API client, WebAuthn service |
|
||||
| `widgets/` | Reusable UI components |
|
||||
|
||||
**State Management Flow:**
|
||||
```
|
||||
User Action → BLoC Event → BLoC Logic → State Update → UI Rebuild
|
||||
↓
|
||||
Repository
|
||||
↓
|
||||
API Service
|
||||
↓
|
||||
Go Backend
|
||||
```
|
||||
|
||||
### 2. Go Backend (`go_cloud/`)
|
||||
|
||||
The API server handling business logic, authentication, and service orchestration.
|
||||
|
||||
**Technology Stack:**
|
||||
- Go 1.21+
|
||||
- Chi Router for HTTP routing
|
||||
- sqlx for database access
|
||||
- go-webauthn for passkey authentication
|
||||
|
||||
**Key Packages:**
|
||||
| Package | Purpose |
|
||||
|---------|---------|
|
||||
| `internal/auth/` | Authentication (OIDC, Passkeys, Sessions) |
|
||||
| `internal/files/` | File metadata and operations |
|
||||
| `internal/org/` | Organization and membership management |
|
||||
| `internal/storage/` | Nextcloud/WebDAV integration |
|
||||
| `internal/http/` | HTTP handlers and WOPI endpoints |
|
||||
| `internal/middleware/` | Auth, logging, CORS middleware |
|
||||
| `pkg/jwt/` | JWT token utilities |
|
||||
|
||||
**Request Flow:**
|
||||
```
|
||||
HTTP Request → Traefik → Chi Router → Middleware → Handler → Service → Response
|
||||
↓
|
||||
Database/Storage
|
||||
```
|
||||
|
||||
### 3. PostgreSQL Database
|
||||
|
||||
Stores application metadata (not files).
|
||||
|
||||
**Key Tables:**
|
||||
- `users` - User accounts and profiles
|
||||
- `roles` - Permission roles (user, admin, superadmin)
|
||||
- `passkeys` - WebAuthn credentials
|
||||
- `organizations` - Org definitions
|
||||
- `org_memberships` - User-org relationships
|
||||
- `activities` - Audit log
|
||||
|
||||
**Schema Relationships:**
|
||||
```
|
||||
users ──┬── passkeys (1:N)
|
||||
├── org_memberships (N:M) ── organizations
|
||||
├── recovery_codes (1:N)
|
||||
└── activities (1:N)
|
||||
```
|
||||
|
||||
### 4. Nextcloud (Storage)
|
||||
|
||||
File storage backend and OIDC provider.
|
||||
|
||||
**Responsibilities:**
|
||||
- File storage via WebDAV
|
||||
- User authentication (OIDC)
|
||||
- File sharing capabilities
|
||||
- Version control
|
||||
|
||||
**Integration Points:**
|
||||
- WebDAV API for file operations
|
||||
- OIDC for authentication
|
||||
- User provisioning sync
|
||||
|
||||
### 5. Collabora Online (Office)
|
||||
|
||||
Document editing service for Office files.
|
||||
|
||||
**Supported Formats:**
|
||||
- Documents: DOCX, ODT, RTF
|
||||
- Spreadsheets: XLSX, ODS, CSV
|
||||
- Presentations: PPTX, ODP
|
||||
|
||||
**Integration:**
|
||||
- WOPI protocol for document access
|
||||
- Embedded iframe in Flutter app
|
||||
|
||||
### 6. Traefik (Reverse Proxy)
|
||||
|
||||
SSL termination and request routing.
|
||||
|
||||
**Features:**
|
||||
- Automatic SSL via Let's Encrypt (DNS-01 challenge)
|
||||
- Dynamic service discovery
|
||||
- Load balancing
|
||||
- Request routing based on hostname
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Authentication Flow
|
||||
|
||||
```
|
||||
┌────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Client │───▶│ Frontend │───▶│ Backend │───▶│ Database │
|
||||
└────────┘ └──────────┘ └──────────┘ └──────────┘
|
||||
│ │
|
||||
│ 1. Username + Passkey │
|
||||
│─────────────────────────────▶│
|
||||
│ │
|
||||
│ 2. WebAuthn Challenge │
|
||||
│◀─────────────────────────────│
|
||||
│ │
|
||||
│ 3. Signed Challenge │
|
||||
│─────────────────────────────▶│
|
||||
│ │ 4. Verify Signature
|
||||
│ │ 5. Create Session
|
||||
│ 6. JWT Token │
|
||||
│◀─────────────────────────────│
|
||||
```
|
||||
|
||||
### File Upload Flow
|
||||
|
||||
```
|
||||
┌────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐
|
||||
│ Client │───▶│ Frontend │───▶│ Backend │───▶│ Nextcloud │
|
||||
└────────┘ └──────────┘ └──────────┘ └───────────┘
|
||||
│ │ │
|
||||
│ 1. Select File │ │
|
||||
│─────────────────────────────▶│ │
|
||||
│ │ │
|
||||
│ │ 2. WebDAV PUT │
|
||||
│ │───────────────▶│
|
||||
│ │ │
|
||||
│ │ 3. Success │
|
||||
│ │◀───────────────│
|
||||
│ │ │
|
||||
│ │ 4. Save Metadata
|
||||
│ │ (PostgreSQL) │
|
||||
│ 5. Confirmation │ │
|
||||
│◀─────────────────────────────│ │
|
||||
```
|
||||
|
||||
## Network Architecture
|
||||
|
||||
### Docker Networks
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ proxy (172.20.0.0/16) │
|
||||
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
|
||||
│ │ traefik │ │ flutter │ │ go │ │nextcloud│ │
|
||||
│ │ │ │ web │ │ backend │ │ │ │
|
||||
│ └─────────┘ └─────────┘ └────┬────┘ └─────────┘ │
|
||||
└───────────────────────────────┼─────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────────┼─────────────────────────────┐
|
||||
│ backend (internal) │
|
||||
│ ┌────┴────┐ │
|
||||
│ │postgres │ │
|
||||
│ └─────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Port Mapping
|
||||
|
||||
| Service | Internal Port | External |
|
||||
|---------|---------------|----------|
|
||||
| Traefik | 80, 443 | Exposed |
|
||||
| Flutter Web | 80 | Via Traefik |
|
||||
| Go Backend | 8080 | Via Traefik |
|
||||
| PostgreSQL | 5432 | Internal only |
|
||||
| Nextcloud | 80 | Via Traefik |
|
||||
| Collabora | 9980 | Via Traefik |
|
||||
|
||||
## Security Architecture
|
||||
|
||||
### Authentication Layers
|
||||
|
||||
1. **Primary**: WebAuthn Passkeys (FIDO2)
|
||||
2. **Fallback**: Optional password authentication
|
||||
3. **Legacy**: OIDC via Nextcloud (deprecated)
|
||||
4. **Recovery**: One-time recovery codes
|
||||
|
||||
### Authorization Model
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Role Hierarchy │
|
||||
│ │
|
||||
│ superadmin (Level 3) │
|
||||
│ ├── All system access │
|
||||
│ ├── User management │
|
||||
│ └── Can manage admins │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ admin (Level 2) │
|
||||
│ ├── Organization management │
|
||||
│ ├── User role management (within orgs) │
|
||||
│ └── Activity monitoring │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ user (Level 1) │
|
||||
│ ├── Personal file management │
|
||||
│ ├── Organization membership │
|
||||
│ └── Basic settings │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Organization Roles
|
||||
|
||||
Within each organization:
|
||||
- **Owner**: Full control, can delete org
|
||||
- **Admin**: Can manage members and files
|
||||
- **Member**: Read/write access to shared files
|
||||
|
||||
## Scalability Considerations
|
||||
|
||||
### Current Architecture (Single Server)
|
||||
|
||||
- All services on one VPS
|
||||
- Suitable for small teams (< 100 users)
|
||||
- Simple deployment and maintenance
|
||||
|
||||
### Future Scaling Options
|
||||
|
||||
1. **Database**: Read replicas, connection pooling
|
||||
2. **Storage**: S3-compatible backends, CDN for static assets
|
||||
3. **Backend**: Horizontal scaling with load balancer
|
||||
4. **Frontend**: CDN distribution, edge caching
|
||||
|
||||
## Monitoring & Observability
|
||||
|
||||
### Logging
|
||||
|
||||
- **Traefik**: Access logs, error logs
|
||||
- **Go Backend**: Structured JSON logs
|
||||
- **PostgreSQL**: Query logs, slow query analysis
|
||||
- **Docker**: Container logs via `docker logs`
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Backend health
|
||||
curl https://go.b0esche.cloud/health
|
||||
|
||||
# Frontend availability
|
||||
curl -I https://www.b0esche.cloud
|
||||
|
||||
# Database connectivity
|
||||
docker exec go-postgres pg_isready
|
||||
```
|
||||
|
||||
### Metrics (Future)
|
||||
|
||||
- Prometheus for metrics collection
|
||||
- Grafana for visualization
|
||||
- AlertManager for alerting
|
||||
579
docs/DEPLOYMENT.md
Normal file
579
docs/DEPLOYMENT.md
Normal file
@@ -0,0 +1,579 @@
|
||||
# b0esche.cloud Deployment Guide
|
||||
|
||||
This guide covers production deployment, server configuration, and operations.
|
||||
|
||||
## Production Architecture
|
||||
|
||||
### Server Overview
|
||||
|
||||
| Component | Domain | Port | Container |
|
||||
|-----------|--------|------|-----------|
|
||||
| Flutter Web | www.b0esche.cloud | 80 | `flutter-web` |
|
||||
| Go Backend | go.b0esche.cloud | 8080 | `go-backend` |
|
||||
| PostgreSQL | internal | 5432 | `go-postgres` |
|
||||
| Nextcloud | storage.b0esche.cloud | 80 | `nextcloud` |
|
||||
| Collabora | of.b0esche.cloud | 9980 | `collabora` |
|
||||
| Traefik | - | 80, 443 | `traefik` |
|
||||
|
||||
### Server Directory Structure
|
||||
|
||||
```
|
||||
/opt/
|
||||
├── traefik/
|
||||
│ ├── docker-compose.yml # Traefik + Nextcloud + Collabora
|
||||
│ ├── traefik.yml # Static configuration
|
||||
│ ├── .env # DNS credentials
|
||||
│ └── acme/ # SSL certificates
|
||||
├── go/
|
||||
│ ├── docker-compose.yml # Go backend + PostgreSQL
|
||||
│ ├── .env.production # Production environment
|
||||
│ └── data/
|
||||
│ └── postgres/
|
||||
│ └── backend/
|
||||
│ └── go_cloud/ # Backend source code
|
||||
├── flutter/
|
||||
│ ├── docker-compose.yml # Nginx for Flutter
|
||||
│ ├── nginx.conf # Nginx configuration
|
||||
│ └── web/ # Built Flutter files
|
||||
├── scripts/
|
||||
│ ├── auto-deploy.sh # Daily auto-deployment
|
||||
│ ├── deploy-now.sh # Manual deployment trigger
|
||||
│ ├── backup.sh # Backup script
|
||||
│ ├── monitor.sh # Health monitoring
|
||||
│ └── webhook-server.py # GitLab webhook receiver
|
||||
└── auto-deploy/
|
||||
└── b0esche_cloud_rollout/ # Deployment workspace
|
||||
```
|
||||
|
||||
## Deployment Methods
|
||||
|
||||
### 1. Automatic Deployment (Recommended)
|
||||
|
||||
Deployments run automatically at 3 AM daily via cron:
|
||||
|
||||
```cron
|
||||
0 3 * * * /opt/scripts/auto-deploy.sh >> /var/log/auto-deploy.log 2>&1
|
||||
```
|
||||
|
||||
The auto-deploy script:
|
||||
1. Pulls latest changes from GitLab
|
||||
2. Builds Flutter web app
|
||||
3. Rebuilds Go backend Docker image
|
||||
4. Restarts services
|
||||
5. Validates health checks
|
||||
|
||||
### 2. Manual Deployment (Immediate)
|
||||
|
||||
Trigger an immediate deployment:
|
||||
|
||||
```bash
|
||||
# From local machine
|
||||
ssh b0esche-cloud '/opt/scripts/deploy-now.sh'
|
||||
|
||||
# Or directly on server
|
||||
/opt/scripts/deploy-now.sh
|
||||
```
|
||||
|
||||
### 3. GitLab Webhook (On Push)
|
||||
|
||||
The webhook server listens for push events:
|
||||
|
||||
```bash
|
||||
# Start webhook server (runs as systemd service)
|
||||
systemctl start webhook-server
|
||||
|
||||
# Check webhook logs
|
||||
journalctl -u webhook-server -f
|
||||
```
|
||||
|
||||
## Service Management
|
||||
|
||||
### Starting All Services
|
||||
|
||||
```bash
|
||||
# Start in order (dependencies first)
|
||||
cd /opt/traefik && docker-compose up -d
|
||||
cd /opt/go && docker-compose up -d
|
||||
cd /opt/flutter && docker-compose up -d
|
||||
```
|
||||
|
||||
### Stopping All Services
|
||||
|
||||
```bash
|
||||
cd /opt/flutter && docker-compose down
|
||||
cd /opt/go && docker-compose down
|
||||
cd /opt/traefik && docker-compose down
|
||||
```
|
||||
|
||||
### Restarting Individual Services
|
||||
|
||||
```bash
|
||||
# Restart Go backend
|
||||
cd /opt/go && docker-compose restart go-backend
|
||||
|
||||
# Restart Flutter frontend
|
||||
cd /opt/flutter && docker-compose restart flutter-web
|
||||
|
||||
# Restart Traefik (caution: brief SSL interruption)
|
||||
cd /opt/traefik && docker-compose restart traefik
|
||||
```
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
```bash
|
||||
# Follow Go backend logs
|
||||
docker logs -f go-backend
|
||||
|
||||
# Follow Flutter/Nginx logs
|
||||
docker logs -f flutter-web
|
||||
|
||||
# Follow Traefik logs
|
||||
docker logs -f traefik
|
||||
|
||||
# All logs with timestamps
|
||||
docker logs -f --timestamps go-backend
|
||||
```
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### Traefik Configuration
|
||||
|
||||
**docker-compose.yml:**
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
traefik:
|
||||
image: traefik:v2.10
|
||||
container_name: traefik
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
- ./traefik.yml:/etc/traefik/traefik.yml:ro
|
||||
- ./acme:/etc/traefik/acme
|
||||
networks:
|
||||
- proxy
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
proxy:
|
||||
external: true
|
||||
```
|
||||
|
||||
**traefik.yml:**
|
||||
```yaml
|
||||
entryPoints:
|
||||
web:
|
||||
address: ":80"
|
||||
http:
|
||||
redirections:
|
||||
entryPoint:
|
||||
to: websecure
|
||||
scheme: https
|
||||
websecure:
|
||||
address: ":443"
|
||||
|
||||
certificatesResolvers:
|
||||
letsencrypt:
|
||||
acme:
|
||||
email: admin@b0esche.cloud
|
||||
storage: /etc/traefik/acme/acme.json
|
||||
dnsChallenge:
|
||||
provider: bunny
|
||||
delayBeforeCheck: 30
|
||||
|
||||
providers:
|
||||
docker:
|
||||
exposedByDefault: false
|
||||
```
|
||||
|
||||
### Go Backend Configuration
|
||||
|
||||
**docker-compose.yml:**
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:15-alpine
|
||||
container_name: go-postgres
|
||||
environment:
|
||||
POSTGRES_USER: go_backend
|
||||
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
||||
POSTGRES_DB: go_backend
|
||||
volumes:
|
||||
- ./data/postgres:/var/lib/postgresql/data
|
||||
networks:
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
|
||||
go-backend:
|
||||
build:
|
||||
context: ./data/postgres/backend/go_cloud
|
||||
dockerfile: Dockerfile
|
||||
container_name: go-backend
|
||||
env_file: .env.production
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.go.rule=Host(`go.b0esche.cloud`)"
|
||||
- "traefik.http.routers.go.tls.certresolver=letsencrypt"
|
||||
depends_on:
|
||||
- postgres
|
||||
networks:
|
||||
- proxy
|
||||
- backend
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
proxy:
|
||||
external: true
|
||||
backend:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
### Flutter/Nginx Configuration
|
||||
|
||||
**docker-compose.yml:**
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
flutter-web:
|
||||
image: nginx:alpine
|
||||
container_name: flutter-web
|
||||
volumes:
|
||||
- ./web:/usr/share/nginx/html:ro
|
||||
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.flutter.rule=Host(`www.b0esche.cloud`)"
|
||||
- "traefik.http.routers.flutter.tls.certresolver=letsencrypt"
|
||||
networks:
|
||||
- proxy
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
proxy:
|
||||
external: true
|
||||
```
|
||||
|
||||
**nginx.conf:**
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name www.b0esche.cloud;
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
# Flutter web app routing
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
# Cache static assets
|
||||
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
|
||||
expires 1y;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
|
||||
# Gzip compression
|
||||
gzip on;
|
||||
gzip_types text/plain text/css application/json application/javascript;
|
||||
}
|
||||
```
|
||||
|
||||
## Database Operations
|
||||
|
||||
### Running Migrations
|
||||
|
||||
```bash
|
||||
# Enter backend container
|
||||
docker exec -it go-backend sh
|
||||
|
||||
# Run migrations
|
||||
./api migrate up
|
||||
|
||||
# Or from host
|
||||
docker exec go-backend ./api migrate up
|
||||
```
|
||||
|
||||
### Database Backup
|
||||
|
||||
```bash
|
||||
# Manual backup
|
||||
docker exec go-postgres pg_dump -U go_backend -Fc go_backend > backup.sqlc
|
||||
|
||||
# Restore from backup
|
||||
docker exec -i go-postgres pg_restore -U go_backend -d go_backend < backup.sqlc
|
||||
```
|
||||
|
||||
### Connecting to Database
|
||||
|
||||
```bash
|
||||
# Via docker exec
|
||||
docker exec -it go-postgres psql -U go_backend -d go_backend
|
||||
|
||||
# Common queries
|
||||
\dt # List tables
|
||||
\d users # Describe table
|
||||
SELECT count(*) FROM users; # Count users
|
||||
```
|
||||
|
||||
## SSL Certificate Management
|
||||
|
||||
### Certificate Status
|
||||
|
||||
```bash
|
||||
# Check certificate expiry
|
||||
docker exec traefik cat /etc/traefik/acme/acme.json | jq '.letsencrypt.Certificates[].certificate.NotAfter'
|
||||
|
||||
# Force certificate renewal
|
||||
docker restart traefik
|
||||
```
|
||||
|
||||
### Manual Certificate Operations
|
||||
|
||||
```bash
|
||||
# Backup certificates
|
||||
cp -r /opt/traefik/acme /opt/traefik/acme.backup
|
||||
|
||||
# View certificate details
|
||||
openssl s_client -connect www.b0esche.cloud:443 -servername www.b0esche.cloud </dev/null 2>/dev/null | openssl x509 -noout -dates
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Check all services
|
||||
/opt/scripts/monitor.sh
|
||||
|
||||
# Manual health checks
|
||||
curl -s https://go.b0esche.cloud/health
|
||||
curl -s -o /dev/null -w "%{http_code}" https://www.b0esche.cloud
|
||||
curl -s -o /dev/null -w "%{http_code}" https://storage.b0esche.cloud
|
||||
```
|
||||
|
||||
### Container Status
|
||||
|
||||
```bash
|
||||
# All containers
|
||||
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
|
||||
|
||||
# Resource usage
|
||||
docker stats --no-stream
|
||||
|
||||
# Container health
|
||||
docker inspect --format='{{.State.Health.Status}}' go-backend
|
||||
```
|
||||
|
||||
### Disk Usage
|
||||
|
||||
```bash
|
||||
# Docker disk usage
|
||||
docker system df
|
||||
|
||||
# PostgreSQL data size
|
||||
du -sh /opt/go/data/postgres
|
||||
|
||||
# Log sizes
|
||||
du -sh /var/lib/docker/containers/*/
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Automated Backups
|
||||
|
||||
Backups run daily via cron:
|
||||
```cron
|
||||
0 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1
|
||||
```
|
||||
|
||||
### Backup Contents
|
||||
|
||||
1. **PostgreSQL database** (pg_dump)
|
||||
2. **Nextcloud database** (mysqldump)
|
||||
3. **Traefik certificates** (/opt/traefik/acme)
|
||||
4. **Configuration files** (.env, docker-compose.yml)
|
||||
5. **Nextcloud data volume**
|
||||
|
||||
### Backup Retention
|
||||
|
||||
- Keep backups for 30 days
|
||||
- Stored in `/opt/backups/b0esche_cloud/`
|
||||
- Compressed as `.tar.gz`
|
||||
|
||||
### Manual Backup
|
||||
|
||||
```bash
|
||||
# Run backup now
|
||||
/opt/scripts/backup.sh
|
||||
|
||||
# List backups
|
||||
ls -lh /opt/backups/b0esche_cloud/
|
||||
```
|
||||
|
||||
### Restore Procedure
|
||||
|
||||
```bash
|
||||
# 1. Stop services
|
||||
cd /opt/go && docker-compose down
|
||||
cd /opt/flutter && docker-compose down
|
||||
|
||||
# 2. Extract backup
|
||||
cd /opt/backups/b0esche_cloud
|
||||
tar -xzf 20260113_020000.tar.gz
|
||||
|
||||
# 3. Restore database
|
||||
docker exec -i go-postgres pg_restore -U go_backend -d go_backend < go_backend.sqlc
|
||||
|
||||
# 4. Restore configurations
|
||||
cp .env.production /opt/go/
|
||||
cp go-docker-compose.yml /opt/go/docker-compose.yml
|
||||
|
||||
# 5. Restart services
|
||||
cd /opt/go && docker-compose up -d
|
||||
cd /opt/flutter && docker-compose up -d
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Service won't start
|
||||
|
||||
```bash
|
||||
# Check logs for errors
|
||||
docker logs go-backend --tail 50
|
||||
|
||||
# Check container status
|
||||
docker inspect go-backend | jq '.[0].State'
|
||||
|
||||
# Check port conflicts
|
||||
netstat -tlnp | grep -E '80|443|8080'
|
||||
```
|
||||
|
||||
#### Database connection issues
|
||||
|
||||
```bash
|
||||
# Test database connectivity
|
||||
docker exec go-backend ping -c 3 postgres
|
||||
|
||||
# Check PostgreSQL logs
|
||||
docker logs go-postgres --tail 50
|
||||
|
||||
# Verify credentials
|
||||
docker exec go-postgres psql -U go_backend -c "SELECT 1"
|
||||
```
|
||||
|
||||
#### SSL certificate errors
|
||||
|
||||
```bash
|
||||
# Check certificate status
|
||||
curl -vI https://www.b0esche.cloud 2>&1 | grep -A 5 "Server certificate"
|
||||
|
||||
# Force renewal
|
||||
docker restart traefik
|
||||
sleep 60
|
||||
curl -vI https://www.b0esche.cloud
|
||||
```
|
||||
|
||||
#### Out of disk space
|
||||
|
||||
```bash
|
||||
# Check disk usage
|
||||
df -h
|
||||
|
||||
# Clean Docker resources
|
||||
docker system prune -a --volumes
|
||||
|
||||
# Clean old backups
|
||||
find /opt/backups -mtime +30 -delete
|
||||
|
||||
# Clean old logs
|
||||
truncate -s 0 /var/log/auto-deploy.log
|
||||
```
|
||||
|
||||
### Emergency Procedures
|
||||
|
||||
#### Rollback Deployment
|
||||
|
||||
```bash
|
||||
# 1. Stop current services
|
||||
cd /opt/go && docker-compose down
|
||||
cd /opt/flutter && docker-compose down
|
||||
|
||||
# 2. Checkout previous version
|
||||
cd /opt/auto-deploy/b0esche_cloud_rollout
|
||||
git log --oneline -10 # Find last working commit
|
||||
git checkout <commit-hash>
|
||||
|
||||
# 3. Redeploy
|
||||
/opt/scripts/auto-deploy.sh
|
||||
```
|
||||
|
||||
#### Database Recovery
|
||||
|
||||
```bash
|
||||
# Find latest backup
|
||||
ls -lt /opt/backups/b0esche_cloud/ | head -5
|
||||
|
||||
# Restore (see Restore Procedure above)
|
||||
```
|
||||
|
||||
#### Full System Recovery
|
||||
|
||||
1. Provision new server
|
||||
2. Install Docker
|
||||
3. Copy `/opt` from backup
|
||||
4. Start services in order
|
||||
5. Restore database from backup
|
||||
6. Verify health checks
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] All services behind Traefik (no direct port exposure)
|
||||
- [ ] SSL certificates valid and auto-renewing
|
||||
- [ ] Database not accessible from internet
|
||||
- [ ] Strong passwords in `.env.production`
|
||||
- [ ] Regular backups verified
|
||||
- [ ] Firewall configured (only 80, 443, 22 open)
|
||||
- [ ] SSH key authentication only
|
||||
- [ ] Auto-deploy logs monitored
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### PostgreSQL
|
||||
|
||||
```sql
|
||||
-- Check slow queries
|
||||
SELECT * FROM pg_stat_activity WHERE state = 'active';
|
||||
|
||||
-- Analyze tables
|
||||
ANALYZE;
|
||||
|
||||
-- Vacuum
|
||||
VACUUM ANALYZE;
|
||||
```
|
||||
|
||||
### Nginx
|
||||
|
||||
```nginx
|
||||
# Add to nginx.conf for better performance
|
||||
worker_connections 1024;
|
||||
keepalive_timeout 65;
|
||||
gzip_comp_level 6;
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
# Limit container resources
|
||||
docker update --memory="512m" --cpus="1" go-backend
|
||||
|
||||
# Clean up unused resources
|
||||
docker system prune -f
|
||||
```
|
||||
643
docs/DEVELOPMENT.md
Normal file
643
docs/DEVELOPMENT.md
Normal file
@@ -0,0 +1,643 @@
|
||||
# b0esche.cloud Development Guide
|
||||
|
||||
This guide covers local development setup, coding conventions, and contribution guidelines.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Software
|
||||
|
||||
| Software | Version | Installation |
|
||||
|----------|---------|--------------|
|
||||
| Go | 1.21+ | `brew install go` |
|
||||
| Flutter | 3.10+ | [flutter.dev](https://flutter.dev/docs/get-started/install) |
|
||||
| Docker | 24+ | [docker.com](https://docker.com) |
|
||||
| PostgreSQL | 15+ | `brew install postgresql@15` or Docker |
|
||||
| Git | 2.x | `brew install git` |
|
||||
|
||||
### Recommended Tools
|
||||
|
||||
- **VS Code** with extensions:
|
||||
- Go
|
||||
- Flutter
|
||||
- Dart
|
||||
- Docker
|
||||
- GitLens
|
||||
- **TablePlus** or **DBeaver** for database management
|
||||
- **Postman** or **Bruno** for API testing
|
||||
|
||||
## Project Setup
|
||||
|
||||
### 1. Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://lab.b0esche.cloud/b0esche/b0esche_cloud.git
|
||||
cd b0esche_cloud
|
||||
```
|
||||
|
||||
### 2. Backend Setup
|
||||
|
||||
```bash
|
||||
cd go_cloud
|
||||
|
||||
# Copy environment file
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env with your local settings
|
||||
# Key variables to set:
|
||||
# - DATABASE_URL=postgres://user:pass@localhost:5432/b0esche_dev?sslmode=disable
|
||||
# - JWT_SECRET=your-dev-secret
|
||||
# - WEBAUTHN_RP_ID=localhost
|
||||
# - WEBAUTHN_RP_ORIGIN=http://localhost:8080
|
||||
```
|
||||
|
||||
#### Start PostgreSQL
|
||||
|
||||
**Option A: Using Docker (Recommended)**
|
||||
```bash
|
||||
docker run -d \
|
||||
--name b0esche-postgres \
|
||||
-e POSTGRES_USER=b0esche \
|
||||
-e POSTGRES_PASSWORD=devpassword \
|
||||
-e POSTGRES_DB=b0esche_dev \
|
||||
-p 5432:5432 \
|
||||
postgres:15-alpine
|
||||
```
|
||||
|
||||
**Option B: Using local PostgreSQL**
|
||||
```bash
|
||||
createdb b0esche_dev
|
||||
```
|
||||
|
||||
#### Run Migrations
|
||||
|
||||
```bash
|
||||
# Install goose
|
||||
go install github.com/pressly/goose/v3/cmd/goose@latest
|
||||
|
||||
# Run migrations
|
||||
goose -dir migrations postgres "$DATABASE_URL" up
|
||||
```
|
||||
|
||||
#### Start Backend
|
||||
|
||||
```bash
|
||||
# Development mode with hot reload
|
||||
go run ./cmd/api
|
||||
|
||||
# Or build and run
|
||||
go build -o bin/api ./cmd/api
|
||||
./bin/api
|
||||
```
|
||||
|
||||
The backend will be available at `http://localhost:8080`.
|
||||
|
||||
### 3. Frontend Setup
|
||||
|
||||
```bash
|
||||
cd b0esche_cloud
|
||||
|
||||
# Get dependencies
|
||||
flutter pub get
|
||||
|
||||
# Run in Chrome (recommended for web development)
|
||||
flutter run -d chrome
|
||||
|
||||
# Or run with specific port
|
||||
flutter run -d chrome --web-port=3000
|
||||
```
|
||||
|
||||
The frontend will be available at `http://localhost:3000` (or the port shown).
|
||||
|
||||
### 4. Quick Start Script
|
||||
|
||||
Use the provided development script:
|
||||
|
||||
```bash
|
||||
./scripts/dev-all.sh
|
||||
```
|
||||
|
||||
This starts all services in the correct order.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Backend (`go_cloud/`)
|
||||
|
||||
```
|
||||
go_cloud/
|
||||
├── cmd/
|
||||
│ └── api/
|
||||
│ └── main.go # Application entry point
|
||||
├── internal/
|
||||
│ ├── auth/
|
||||
│ │ ├── auth.go # Authentication service
|
||||
│ │ ├── passkey.go # WebAuthn implementation
|
||||
│ │ └── auth_test.go # Tests
|
||||
│ ├── config/
|
||||
│ │ └── config.go # Configuration loading
|
||||
│ ├── database/
|
||||
│ │ └── database.go # Database connection
|
||||
│ ├── files/
|
||||
│ │ └── files.go # File operations
|
||||
│ ├── http/
|
||||
│ │ ├── routes.go # Route definitions
|
||||
│ │ ├── server.go # HTTP server setup
|
||||
│ │ └── wopi_handlers.go # WOPI protocol handlers
|
||||
│ ├── middleware/
|
||||
│ │ └── middleware.go # HTTP middleware
|
||||
│ ├── models/
|
||||
│ │ └── *.go # Data models
|
||||
│ ├── org/
|
||||
│ │ └── org.go # Organization logic
|
||||
│ ├── storage/
|
||||
│ │ ├── nextcloud.go # Nextcloud integration
|
||||
│ │ └── webdav.go # WebDAV client
|
||||
│ └── ...
|
||||
├── migrations/
|
||||
│ ├── 0001_initial.sql
|
||||
│ ├── 0002_passkeys.sql
|
||||
│ └── ...
|
||||
├── pkg/
|
||||
│ └── jwt/
|
||||
│ └── jwt.go # JWT utilities
|
||||
├── .env.example
|
||||
├── Dockerfile
|
||||
├── go.mod
|
||||
└── Makefile
|
||||
```
|
||||
|
||||
### Frontend (`b0esche_cloud/`)
|
||||
|
||||
```
|
||||
b0esche_cloud/
|
||||
├── lib/
|
||||
│ ├── main.dart # App entry point
|
||||
│ ├── injection.dart # Dependency injection
|
||||
│ ├── blocs/
|
||||
│ │ ├── auth/
|
||||
│ │ │ ├── auth_bloc.dart
|
||||
│ │ │ ├── auth_event.dart
|
||||
│ │ │ └── auth_state.dart
|
||||
│ │ ├── files/
|
||||
│ │ └── org/
|
||||
│ ├── models/
|
||||
│ │ ├── user.dart
|
||||
│ │ ├── file.dart
|
||||
│ │ └── organization.dart
|
||||
│ ├── pages/
|
||||
│ │ ├── home_page.dart
|
||||
│ │ ├── files_page.dart
|
||||
│ │ ├── settings_page.dart
|
||||
│ │ └── admin/
|
||||
│ ├── repositories/
|
||||
│ │ ├── auth_repository.dart
|
||||
│ │ └── file_repository.dart
|
||||
│ ├── services/
|
||||
│ │ ├── api_client.dart
|
||||
│ │ └── webauthn_service.dart
|
||||
│ ├── theme/
|
||||
│ │ └── app_theme.dart
|
||||
│ └── widgets/
|
||||
│ ├── file_list.dart
|
||||
│ └── ...
|
||||
├── web/
|
||||
│ └── index.html
|
||||
├── pubspec.yaml
|
||||
└── analysis_options.yaml
|
||||
```
|
||||
|
||||
## Coding Conventions
|
||||
|
||||
### Go Backend
|
||||
|
||||
#### Code Style
|
||||
- Follow [Effective Go](https://golang.org/doc/effective_go)
|
||||
- Use `gofmt` for formatting
|
||||
- Use `golint` and `go vet` for linting
|
||||
|
||||
```bash
|
||||
# Format code
|
||||
gofmt -w .
|
||||
|
||||
# Lint
|
||||
golint ./...
|
||||
go vet ./...
|
||||
```
|
||||
|
||||
#### Naming Conventions
|
||||
- **Packages**: lowercase, single word (`auth`, `files`)
|
||||
- **Exported functions**: PascalCase (`CreateUser`)
|
||||
- **Private functions**: camelCase (`validateToken`)
|
||||
- **Constants**: PascalCase (`DefaultTimeout`)
|
||||
|
||||
#### Error Handling
|
||||
```go
|
||||
// Always handle errors explicitly
|
||||
user, err := s.GetUser(ctx, id)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get user: %w", err)
|
||||
}
|
||||
|
||||
// Use custom error types for API errors
|
||||
type APIError struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
```
|
||||
|
||||
#### Project Patterns
|
||||
```go
|
||||
// Service pattern
|
||||
type AuthService struct {
|
||||
db *sqlx.DB
|
||||
config *config.Config
|
||||
}
|
||||
|
||||
func NewAuthService(db *sqlx.DB, config *config.Config) *AuthService {
|
||||
return &AuthService{db: db, config: config}
|
||||
}
|
||||
|
||||
// Handler pattern
|
||||
func (h *Handler) HandleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
// Parse request
|
||||
// Call service
|
||||
// Return response
|
||||
}
|
||||
```
|
||||
|
||||
### Flutter Frontend
|
||||
|
||||
#### Code Style
|
||||
- Follow [Effective Dart](https://dart.dev/guides/language/effective-dart)
|
||||
- Use `dart format` for formatting
|
||||
- Use `dart analyze` for linting
|
||||
|
||||
```bash
|
||||
# Format code
|
||||
dart format .
|
||||
|
||||
# Analyze
|
||||
dart analyze
|
||||
flutter analyze
|
||||
```
|
||||
|
||||
#### Naming Conventions
|
||||
- **Classes**: PascalCase (`AuthBloc`)
|
||||
- **Files**: snake_case (`auth_bloc.dart`)
|
||||
- **Variables/Functions**: camelCase (`getUserName`)
|
||||
- **Constants**: camelCase or SCREAMING_CAPS
|
||||
|
||||
#### BLoC Pattern
|
||||
```dart
|
||||
// Events
|
||||
abstract class AuthEvent {}
|
||||
class LoginRequested extends AuthEvent {
|
||||
final String username;
|
||||
LoginRequested(this.username);
|
||||
}
|
||||
|
||||
// States
|
||||
abstract class AuthState {}
|
||||
class AuthInitial extends AuthState {}
|
||||
class AuthLoading extends AuthState {}
|
||||
class AuthAuthenticated extends AuthState {
|
||||
final User user;
|
||||
AuthAuthenticated(this.user);
|
||||
}
|
||||
class AuthError extends AuthState {
|
||||
final String message;
|
||||
AuthError(this.message);
|
||||
}
|
||||
|
||||
// BLoC
|
||||
class AuthBloc extends Bloc<AuthEvent, AuthState> {
|
||||
AuthBloc() : super(AuthInitial()) {
|
||||
on<LoginRequested>(_onLoginRequested);
|
||||
}
|
||||
|
||||
Future<void> _onLoginRequested(
|
||||
LoginRequested event,
|
||||
Emitter<AuthState> emit,
|
||||
) async {
|
||||
emit(AuthLoading());
|
||||
try {
|
||||
final user = await _authRepository.login(event.username);
|
||||
emit(AuthAuthenticated(user));
|
||||
} catch (e) {
|
||||
emit(AuthError(e.toString()));
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Widget Structure
|
||||
```dart
|
||||
class MyWidget extends StatelessWidget {
|
||||
const MyWidget({super.key});
|
||||
|
||||
@override
|
||||
Widget build(BuildContext context) {
|
||||
return BlocBuilder<MyBloc, MyState>(
|
||||
builder: (context, state) {
|
||||
return switch (state) {
|
||||
MyLoading() => const CircularProgressIndicator(),
|
||||
MyLoaded(:final data) => _buildContent(data),
|
||||
MyError(:final message) => Text('Error: $message'),
|
||||
_ => const SizedBox.shrink(),
|
||||
};
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Backend Tests
|
||||
|
||||
```bash
|
||||
cd go_cloud
|
||||
|
||||
# Run all tests
|
||||
go test ./...
|
||||
|
||||
# Run with coverage
|
||||
go test -cover ./...
|
||||
|
||||
# Run specific package
|
||||
go test ./internal/auth/...
|
||||
|
||||
# Verbose output
|
||||
go test -v ./...
|
||||
```
|
||||
|
||||
Example test:
|
||||
```go
|
||||
func TestAuthService_Login(t *testing.T) {
|
||||
// Setup
|
||||
db := setupTestDB(t)
|
||||
service := NewAuthService(db, testConfig)
|
||||
|
||||
// Test
|
||||
user, err := service.Login(context.Background(), "testuser")
|
||||
|
||||
// Assert
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, "testuser", user.Username)
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend Tests
|
||||
|
||||
```bash
|
||||
cd b0esche_cloud
|
||||
|
||||
# Run all tests
|
||||
flutter test
|
||||
|
||||
# Run with coverage
|
||||
flutter test --coverage
|
||||
|
||||
# Run specific test file
|
||||
flutter test test/auth_bloc_test.dart
|
||||
|
||||
# Run integration tests
|
||||
flutter test integration_test/
|
||||
```
|
||||
|
||||
Example test:
|
||||
```dart
|
||||
void main() {
|
||||
group('AuthBloc', () {
|
||||
late AuthBloc authBloc;
|
||||
late MockAuthRepository mockRepository;
|
||||
|
||||
setUp(() {
|
||||
mockRepository = MockAuthRepository();
|
||||
authBloc = AuthBloc(authRepository: mockRepository);
|
||||
});
|
||||
|
||||
blocTest<AuthBloc, AuthState>(
|
||||
'emits [AuthLoading, AuthAuthenticated] on successful login',
|
||||
build: () => authBloc,
|
||||
act: (bloc) => bloc.add(LoginRequested('testuser')),
|
||||
expect: () => [
|
||||
AuthLoading(),
|
||||
isA<AuthAuthenticated>(),
|
||||
],
|
||||
);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Database Migrations
|
||||
|
||||
### Creating a Migration
|
||||
|
||||
```bash
|
||||
cd go_cloud
|
||||
|
||||
# Create new migration
|
||||
goose -dir migrations create add_new_table sql
|
||||
```
|
||||
|
||||
### Migration Best Practices
|
||||
|
||||
```sql
|
||||
-- migrations/0005_add_feature.sql
|
||||
|
||||
-- +goose Up
|
||||
-- Add new column with default
|
||||
ALTER TABLE users ADD COLUMN new_field TEXT DEFAULT '';
|
||||
|
||||
-- Create index for performance
|
||||
CREATE INDEX idx_users_new_field ON users(new_field);
|
||||
|
||||
-- +goose Down
|
||||
DROP INDEX IF EXISTS idx_users_new_field;
|
||||
ALTER TABLE users DROP COLUMN new_field;
|
||||
```
|
||||
|
||||
### Running Migrations
|
||||
|
||||
```bash
|
||||
# Apply all pending migrations
|
||||
goose -dir migrations postgres "$DATABASE_URL" up
|
||||
|
||||
# Rollback last migration
|
||||
goose -dir migrations postgres "$DATABASE_URL" down
|
||||
|
||||
# Check migration status
|
||||
goose -dir migrations postgres "$DATABASE_URL" status
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Backend Debugging
|
||||
|
||||
**VS Code launch.json:**
|
||||
```json
|
||||
{
|
||||
"version": "0.2.0",
|
||||
"configurations": [
|
||||
{
|
||||
"name": "Launch Go Backend",
|
||||
"type": "go",
|
||||
"request": "launch",
|
||||
"mode": "debug",
|
||||
"program": "${workspaceFolder}/go_cloud/cmd/api",
|
||||
"envFile": "${workspaceFolder}/go_cloud/.env"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Logging:**
|
||||
```go
|
||||
import "log"
|
||||
|
||||
log.Printf("User login attempt: %s", username)
|
||||
```
|
||||
|
||||
### Frontend Debugging
|
||||
|
||||
**Chrome DevTools:**
|
||||
- Press F12 in Chrome
|
||||
- Use the Flutter DevTools extension
|
||||
|
||||
**Debug print:**
|
||||
```dart
|
||||
debugPrint('Current state: $state');
|
||||
```
|
||||
|
||||
**VS Code launch.json:**
|
||||
```json
|
||||
{
|
||||
"version": "0.2.0",
|
||||
"configurations": [
|
||||
{
|
||||
"name": "Flutter Web",
|
||||
"type": "dart",
|
||||
"request": "launch",
|
||||
"program": "lib/main.dart",
|
||||
"deviceId": "chrome"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Git Workflow
|
||||
|
||||
### Branch Naming
|
||||
|
||||
- `feature/description` - New features
|
||||
- `fix/description` - Bug fixes
|
||||
- `refactor/description` - Code refactoring
|
||||
- `docs/description` - Documentation updates
|
||||
|
||||
### Commit Messages
|
||||
|
||||
Follow conventional commits:
|
||||
```
|
||||
type(scope): description
|
||||
|
||||
feat(auth): add passkey registration flow
|
||||
fix(files): correct upload progress display
|
||||
docs(readme): update deployment instructions
|
||||
refactor(api): extract common error handling
|
||||
```
|
||||
|
||||
### Pull Request Process
|
||||
|
||||
1. Create feature branch from `main`
|
||||
2. Make changes with atomic commits
|
||||
3. Run tests locally
|
||||
4. Push and create PR
|
||||
5. Wait for review
|
||||
6. Squash and merge
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Backend won't start
|
||||
```bash
|
||||
# Check if port is in use
|
||||
lsof -i :8080
|
||||
|
||||
# Check database connection
|
||||
psql $DATABASE_URL -c "SELECT 1"
|
||||
|
||||
# Check logs
|
||||
go run ./cmd/api 2>&1 | head -50
|
||||
```
|
||||
|
||||
#### Flutter build fails
|
||||
```bash
|
||||
# Clean and rebuild
|
||||
flutter clean
|
||||
flutter pub get
|
||||
flutter run -d chrome
|
||||
|
||||
# Check for dependency issues
|
||||
flutter pub deps
|
||||
```
|
||||
|
||||
#### Database migration fails
|
||||
```bash
|
||||
# Check current status
|
||||
goose -dir migrations postgres "$DATABASE_URL" status
|
||||
|
||||
# Force specific version
|
||||
goose -dir migrations postgres "$DATABASE_URL" fix
|
||||
```
|
||||
|
||||
#### WebAuthn not working locally
|
||||
- WebAuthn requires HTTPS in production
|
||||
- For localhost, use `WEBAUTHN_RP_ID=localhost`
|
||||
- Chrome allows WebAuthn on localhost without HTTPS
|
||||
|
||||
## Environment Variables Reference
|
||||
|
||||
### Backend (.env)
|
||||
|
||||
```bash
|
||||
# Server
|
||||
SERVER_ADDR=:8080
|
||||
DEV_MODE=true
|
||||
|
||||
# Database
|
||||
DATABASE_URL=postgres://user:pass@localhost:5432/dbname?sslmode=disable
|
||||
|
||||
# Authentication
|
||||
JWT_SECRET=your-secret-key
|
||||
WEBAUTHN_RP_ID=localhost
|
||||
WEBAUTHN_RP_NAME=b0esche.cloud
|
||||
WEBAUTHN_RP_ORIGIN=http://localhost:8080
|
||||
|
||||
# External Services (optional for local dev)
|
||||
NEXTCLOUD_BASE_URL=https://storage.b0esche.cloud
|
||||
NEXTCLOUD_USERNAME=admin
|
||||
NEXTCLOUD_PASSWORD=password
|
||||
COLLABORA_BASE_URL=https://of.b0esche.cloud
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
API base URL is configured in `lib/services/api_client.dart`:
|
||||
```dart
|
||||
class ApiClient {
|
||||
// For development
|
||||
static const baseUrl = 'http://localhost:8080';
|
||||
|
||||
// For production (set via build args)
|
||||
// static const baseUrl = String.fromEnvironment('API_URL');
|
||||
}
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- [Go Documentation](https://golang.org/doc/)
|
||||
- [Flutter Documentation](https://flutter.dev/docs)
|
||||
- [WebAuthn Guide](https://webauthn.guide/)
|
||||
- [BLoC Library](https://bloclibrary.dev/)
|
||||
- [Chi Router](https://github.com/go-chi/chi)
|
||||
Reference in New Issue
Block a user