CollabTable/CollabTableServer/README.md

CollabTable Server

A collaborative table management server built with Node.js, Express, TypeScript, and a pluggable database (SQLite or PostgreSQL).

Features

• RESTful API for managing lists, fields, items, and values
• Real-time synchronization via WebSocket (primary) with HTTP fallback
• SQLite (better-sqlite3) or PostgreSQL (pg) database backends
• Dockerized deployment with persistent storage
• TypeScript for type safety

Prerequisites

• Docker and Docker Compose
• Or Node.js 20+ (for local development)

Quick Start with Docker

1. Clone the repository

2. Navigate to the server directory:

   cd CollabTableServer
   

3. Copy the example environment file:

   cp .env.example .env
   

4. Start the services:

   docker-compose up -d
   

The server will be available at http://localhost:3000

Important: The default backend is SQLite with a Docker volume sqlite_data for persistence. You can also switch to PostgreSQL by setting environment variables.

Local Development

1. Install dependencies:

   npm install
   

2. Create a .env file:

   cp .env.example .env
   

3. Choose your database in .env (SQLite by default):

   # Backend: sqlite | postgres (default sqlite)
   DB_CLIENT=sqlite
   
   # SQLite
   DB_PATH=./data/collabtable.db
   
   # OR PostgreSQL
   # DATABASE_URL=postgres://user:password@localhost:5432/collabtable
   # PGHOST=localhost
   # PGPORT=5432
   # PGUSER=postgres
   # PGPASSWORD=yourpassword
   # PGDATABASE=collabtable
   

4. Start the development server:

   npm run dev
   

API Endpoints

Lists

• GET /api/lists - Get all lists
• GET /api/lists/:id - Get a specific list
• POST /api/lists - Create a new list
• PUT /api/lists/:id - Update a list
• DELETE /api/lists/:id - Delete a list (soft delete)

Fields

• GET /api/fields/list/:listId - Get all fields for a list
• POST /api/fields - Create a new field
• PUT /api/fields/:id - Update a field
• DELETE /api/fields/:id - Delete a field (soft delete)

Items

• GET /api/items/list/:listId - Get all items for a list
• GET /api/items/:itemId/values - Get all values for an item
• POST /api/items - Create a new item
• POST /api/items/values - Create/update an item value
• PUT /api/items/:id - Update an item
• DELETE /api/items/:id - Delete an item (soft delete)

Sync

• POST /api/sync - Synchronize data between client and server

WebSocket

• GET /api/ws - WebSocket endpoint. Send JSON messages:
  • {"type":"ping"} → {"type":"pong"}
  • {"type":"sync", "id":"<uuid>", "payload": { ...SyncRequest }} → {"type":"syncResponse", "id":"<uuid>", "payload": { ...SyncResponse }}

Authentication: If SERVER_PASSWORD is set, include header Authorization: Bearer <password> on HTTP and WebSocket requests.

Health Check

• GET /health - Check server health

Data Models

List

{
  id: string;
  name: string;
  createdAt: number;
  updatedAt: number;
  isDeleted: boolean;
}

Field

{
  id: string;
  listId: string;
  name: string;
  order: number;
  createdAt: number;
  updatedAt: number;
  isDeleted: boolean;
}

Item

{
  id: string;
  listId: string;
  createdAt: number;
  updatedAt: number;
  isDeleted: boolean;
}

ItemValue

{
  id: string;
  itemId: string;
  fieldId: string;
  value: string;
  updatedAt: number;
}

Sync Protocol

The sync endpoint accepts a POST request with the following structure:

{
  "lastSyncTimestamp": 1234567890,
  "lists": [...],
  "fields": [...],
  "items": [...],
  "itemValues": [...]
}

The server responds with:

{
  "lists": [...],
  "fields": [...],
  "items": [...],
  "itemValues": [...],
  "serverTimestamp": 1234567890
}

The sync process:

1. Client sends all local changes since the last sync
2. Server saves the client's changes
3. Server returns all changes made on the server since the client's last sync
4. Client applies the server's changes locally

Building for Production

Build the TypeScript code:

npm run build

Start the production server:

npm start

Docker Commands

Start services:

docker-compose up -d

Stop services:

docker-compose down

View logs:

docker-compose logs -f

Rebuild and restart:

docker-compose up -d --build

Environment Variables

• PORT - Server port (default: 3000)
• NODE_ENV - Environment (development/production)
• SERVER_PASSWORD - Optional shared password for API and WebSocket (recommended)
• DB_CLIENT - sqlite (default) or postgres
• DB_PATH - Path to SQLite database file (default: ./data/collabtable.db for local, /data/collabtable.db in Docker)
• DATABASE_URL - PostgreSQL connection URL (optional alternative to PG* variables)
• PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE - PostgreSQL connection parameters
• DB_INIT_RETRY_INTERVAL_MS - Delay between DB init retries in ms (default: 5000)
• DB_INIT_RETRY_MAX_ATTEMPTS - Max DB init retries; 0 retries forever (default: 0)
• AUTH_LOG_THROTTLE_MS - Minimum interval before repeating the same auth warning log (default: 30000)
• SYNC_MAX_FUTURE_SKEW_MS - Allowed future timestamp skew from clients during sync (default: 0)

Data Persistence

SQLite

The SQLite database file is stored in a Docker volume, ensuring data persists across container lifecycle events.

Backup:

docker cp collabtable-server:/data/collabtable.db ./backup.db

Restore:

docker cp ./backup.db collabtable-server:/data/collabtable.db
docker-compose restart

PostgreSQL

Use your normal PostgreSQL backup/restore procedures (e.g., pg_dump, pg_restore). If using the optional Dockerized Postgres service, the data is stored in a named volume (see docker-compose.yml).

License

MIT