Cloudflare Infrastructure Plan

Overview

This document details the Cloudflare infrastructure components managed via Terraform for the Raspberry Pi fleet.

Prerequisites

1. Cloudflare Account Setup

• Active Cloudflare account
• Domain tagai.uk added to Cloudflare
• Zero Trust plan enabled (Free tier supports up to 50 users)
• API Token with required permissions

2. Required API Permissions

Create an API Token with: - Zone:DNS:Edit - Manage DNS records - Zone:Zone:Read - Read zone information - Account:Cloudflare Tunnel:Edit - Manage tunnels - Account:Access:Apps and Policies:Edit - Manage Access apps - Account:Access:Organizations, Identity Providers, and Groups:Edit - Manage Access orgs

Infrastructure Components

1. DNS Configuration

Zone Configuration

# Reference existing zone
data "cloudflare_zone" "tagai" {
  name = "tagai.uk"
}

DNS Records

Create CNAME records for each device:

rpi-001.tagai.uk -> rpi-001.cfargotunnel.com
rpi-002.tagai.uk -> rpi-002.cfargotunnel.com
...
rpi-100.tagai.uk -> rpi-100.cfargotunnel.com

Terraform Approach: - Use count or for_each to create 100 DNS records - Follow naming convention: rpi-{001-100} - All records point to their respective tunnel CNAME

2. Cloudflare Tunnels

Tunnel Architecture

• One tunnel per device (100 tunnels total)
• Each tunnel has unique credentials
• Each tunnel routes to Node-RED on localhost:1880

Tunnel Configuration

For each tunnel:

tunnel: <TUNNEL_ID>
credentials-file: /etc/cloudflared/credentials.json

ingress:
  - hostname: rpi-XXX.tagai.uk
    service: http://localhost:1880
  - hostname: rpi-XXX-ssh.tagai.uk
    service: ssh://localhost:22
  - service: http_status:404

Terraform Resources per Tunnel

1. cloudflare_tunnel - Creates the tunnel
2. cloudflare_tunnel_config - Configures ingress rules
3. cloudflare_record - Creates DNS CNAME record

Implementation Strategy:

locals {
  devices = toset([for i in range(1, 101) : format("rpi-%03d", i)])
}

resource "cloudflare_tunnel" "rpi" {
  for_each   = local.devices
  account_id = var.cloudflare_account_id
  name       = each.key
  secret     = random_password.tunnel_secret[each.key].result
}

3. Cloudflare Access

Access Application

Create one Access Application per device (or use groups for efficiency):

Option A: Individual Applications (Fine-grained control) - 100 separate Access applications - One policy per device - Most granular control

Option B: Single Application with Policies (Recommended) - 1 Access application covering *.tagai.uk - Multiple policies for device groups - Easier to manage

Access Policies

Policy Structure: 1. Admin Policy - Full access to all devices 2. Region/Group Policies - Access to device subsets 3. Read-Only Policy - View-only access (if supported)

Authentication Methods: - Email OTP (Free) - Google Workspace / Azure AD (Requires plan) - GitHub (Free) - Service Tokens (For automation)

4. Service Tokens

Create service tokens for: - Monitoring systems - CI/CD pipelines - Management scripts

Terraform Structure

Directory Layout

terraform/
├── main.tf                    # Provider and backend config
├── variables.tf               # Input variables
├── outputs.tf                 # Output values
├── versions.tf               # Provider versions
├── cloudflare-dns.tf         # DNS records
├── cloudflare-tunnels.tf     # Tunnel resources
├── cloudflare-access.tf      # Access apps and policies
├── terraform.tfvars          # Variable values (gitignored)
└── modules/
    ├── rpi-tunnel/           # Reusable tunnel module
    │   ├── main.tf
    │   ├── variables.tf
    │   └── outputs.tf
    └── access-policy/        # Reusable policy module
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

Key Variables

variable "cloudflare_api_token" {
  description = "Cloudflare API token"
  type        = string
  sensitive   = true
}

variable "cloudflare_account_id" {
  description = "Cloudflare account ID"
  type        = string
}

variable "cloudflare_zone_id" {
  description = "Zone ID for tagai.uk"
  type        = string
}

variable "device_count" {
  description = "Number of RPi devices"
  type        = number
  default     = 100
}

variable "admin_emails" {
  description = "Admin email addresses for Access"
  type        = list(string)
}

Important Outputs

output "tunnel_tokens" {
  description = "Tunnel tokens for each device"
  value       = {
    for k, v in cloudflare_tunnel.rpi : k => v.tunnel_token
  }
  sensitive = true
}

output "device_urls" {
  description = "URLs for each device"
  value       = {
    for k in local.devices : k => "https://${k}.tagai.uk"
  }
}

output "tunnel_credentials" {
  description = "Tunnel credentials for provisioning"
  value = {
    for k, v in cloudflare_tunnel.rpi : k => {
      tunnel_id = v.id
      token     = v.tunnel_token
    }
  }
  sensitive = true
}

Terraform State Management

Backend Configuration

Option 1: Terraform Cloud (Recommended)

terraform {
  cloud {
    organization = "tagai"
    workspaces {
      name = "gaisro-technika-infra"
    }
  }
}

Option 2: S3 Backend

terraform {
  backend "s3" {
    bucket = "tagai-terraform-state"
    key    = "rpi-fleet/terraform.tfstate"
    region = "eu-west-1"
  }
}

Option 3: Local (Development Only)

terraform {
  backend "local" {
    path = "terraform.tfstate"
  }
}

Cost Considerations

Cloudflare Pricing

• Tunnels: Free (unlimited)
• Access:
• Free tier: Up to 50 users
• Teams plan: $7/user/month (if > 50 users)
• DNS: Free with Cloudflare
• Zero Trust: Free tier likely sufficient

Optimization

• Use device groups to reduce policy complexity
• Single Access application vs. 100 individual apps
• Service tokens for automation (don't count as users)

Security Considerations

1. API Token Storage
2. Store in environment variables or secret manager
3. Never commit to Git

4. Rotate regularly

5. Tunnel Secrets

6. Generate unique secret per tunnel
7. Store credentials securely

8. Use Terraform sensitive outputs

9. State File Security

10. Contains sensitive data (tunnel tokens)
11. Use remote backend with encryption
12. Restrict access to state file

Implementation Steps

Step 1: Initial Setup

# Set environment variables
export CLOUDFLARE_API_TOKEN="your-token"
export CLOUDFLARE_ACCOUNT_ID="your-account-id"
export CLOUDFLARE_ZONE_ID="your-zone-id"

# Initialize Terraform
cd terraform
terraform init

Step 2: Plan Infrastructure

# Create plan for first 5 devices (testing)
terraform plan -var="device_count=5"

# Review changes

Step 3: Apply Infrastructure

# Apply for test devices
terraform apply -var="device_count=5"

# After validation, scale to 100
terraform apply -var="device_count=100"

Step 4: Extract Credentials

# Export tunnel credentials for provisioning
terraform output -json tunnel_credentials > ../config/tunnel-credentials.json

Token Distribution Strategies

Overview

Each Raspberry Pi needs its unique tunnel token to establish connection. Here are the recommended approaches:

Method 1: Direct Token Extraction (Manual/Semi-Automated)

Best for: Small deployments (1-20 devices), testing

Workflow:

# 1. After Terraform apply, export all tokens
cd terraform
terraform output -json tunnel_credentials > ../config/tunnel-credentials.json

# 2. Extract token for specific device
jq -r '.["rpi-001"].tunnel_token' ../config/tunnel-credentials.json

# 3. Use token in provisioning
ssh pi@192.168.1.100
sudo ./rpi-init.sh rpi-001 "eyJhIjoiYzM5..."

Helper script (scripts/get-token.sh):

#!/bin/bash
# Get tunnel token for specific device

DEVICE_ID="${1:-}"
CREDS_FILE="${2:-config/tunnel-credentials.json}"

if [[ -z "${DEVICE_ID}" ]]; then
    echo "Usage: $0 <device-id> [credentials-file]"
    echo "Example: $0 rpi-001"
    exit 1
fi

if [[ ! -f "${CREDS_FILE}" ]]; then
    echo "Credentials file not found: ${CREDS_FILE}"
    echo "Run: terraform output -json tunnel_credentials > ${CREDS_FILE}"
    exit 1
fi

TOKEN=$(jq -r ".\"${DEVICE_ID}\".tunnel_token" "${CREDS_FILE}")

if [[ "${TOKEN}" == "null" ]]; then
    echo "No token found for ${DEVICE_ID}"
    exit 1
fi

echo "${TOKEN}"

Method 2: Pre-Configured SD Card Images (Recommended for Scale)

Best for: Large deployments (50-100 devices), standardized hardware

Workflow:

# 1. Create base image with generic setup
# Flash one RPi, run base setup (cloudflared, Node.js, etc.)
# But DON'T configure tunnel yet

# 2. Clone the SD card → base-image.img

# 3. For each device, customize the image:
./scripts/prepare-sd-card.sh rpi-001 /dev/disk2

Script (scripts/prepare-sd-card.sh):

#!/bin/bash
# Prepare SD card with device-specific configuration

set -euo pipefail

DEVICE_ID="${1:-}"
SD_CARD="${2:-}"
CREDS_FILE="config/tunnel-credentials.json"

if [[ -z "${DEVICE_ID}" || -z "${SD_CARD}" ]]; then
    echo "Usage: $0 <device-id> <sd-card-device>"
    echo "Example: $0 rpi-001 /dev/disk2"
    exit 1
fi

echo "⚠️  WARNING: This will erase ${SD_CARD}"
read -p "Continue? (yes/no): " confirm
[[ "${confirm}" != "yes" ]] && exit 0

# Flash base image
echo "Flashing base image..."
sudo dd if=images/base-rpi.img of=${SD_CARD} bs=4M status=progress

# Mount boot partition
MOUNT_POINT="/tmp/rpi-boot"
mkdir -p "${MOUNT_POINT}"
sudo mount "${SD_CARD}s1" "${MOUNT_POINT}"

# Get tunnel token
TOKEN=$(jq -r ".\"${DEVICE_ID}\".tunnel_token" "${CREDS_FILE}")

# Create first-boot script
cat > "${MOUNT_POINT}/firstboot.sh" <<EOF
#!/bin/bash
# First boot configuration for ${DEVICE_ID}

# Set hostname
hostnamectl set-hostname ${DEVICE_ID}

# Configure tunnel
cloudflared service install ${TOKEN}
systemctl enable cloudflared
systemctl start cloudflared

# Update device info
echo "DEVICE_ID=${DEVICE_ID}" > /etc/rpi-device-info
echo "PROVISIONED_DATE=\$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> /etc/rpi-device-info

# Remove this script after execution
rm -f /boot/firstboot.sh
EOF

chmod +x "${MOUNT_POINT}/firstboot.sh"

# Add to rc.local or systemd for first boot execution
cat > "${MOUNT_POINT}/firstboot.service" <<EOF
[Unit]
Description=First Boot Configuration
After=network-online.target

[Service]
Type=oneshot
ExecStart=/boot/firstboot.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

# Unmount
sudo umount "${MOUNT_POINT}"

echo "✓ SD card prepared for ${DEVICE_ID}"
echo "  Insert card and boot device"

Method 3: Centralized Token Server (Recommended for Large Scale)

Best for: 100+ devices, continuous provisioning, enterprise deployments

Architecture:

┌─────────────┐
│  Terraform  │
│   Outputs   │
└──────┬──────┘
       │
       v
┌─────────────────────┐
│  Token Server API   │
│  (Simple Flask/Go)  │
└──────┬──────────────┘
       │
       v
┌─────────────────────┐
│  Raspberry Pi       │
│  (Fetches token)    │
└─────────────────────┘

Token Server (scripts/token-server/server.py):

#!/usr/bin/env python3
"""
Simple token distribution server
Runs locally during provisioning
"""

from flask import Flask, jsonify, request
import json
import os
from functools import wraps

app = Flask(__name__)

# Load credentials
CREDS_FILE = os.getenv('CREDS_FILE', 'config/tunnel-credentials.json')
with open(CREDS_FILE) as f:
    CREDENTIALS = json.load(f)

# Simple API key authentication
API_KEY = os.getenv('API_KEY', 'change-me-in-production')

def require_api_key(f):
    @wraps(f)
    def decorated_function(*args, **kwargs):
        if request.headers.get('X-API-Key') != API_KEY:
            return jsonify({'error': 'Unauthorized'}), 401
        return f(*args, **kwargs)
    return decorated_function

@app.route('/health')
def health():
    return jsonify({'status': 'ok'})

@app.route('/token/<device_id>')
@require_api_key
def get_token(device_id):
    """Get tunnel token for specific device"""
    if device_id not in CREDENTIALS:
        return jsonify({'error': 'Device not found'}), 404

    return jsonify({
        'device_id': device_id,
        'tunnel_token': CREDENTIALS[device_id]['tunnel_token'],
        'tunnel_id': CREDENTIALS[device_id]['tunnel_id'],
        'account_id': CREDENTIALS[device_id]['account_id']
    })

@app.route('/devices')
@require_api_key
def list_devices():
    """List all available devices"""
    return jsonify({
        'devices': list(CREDENTIALS.keys()),
        'count': len(CREDENTIALS)
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Start server:

cd scripts/token-server
export API_KEY="your-secure-key"
export CREDS_FILE="../../config/tunnel-credentials.json"
python3 server.py

Modified init script (scripts/rpi-init.sh):

#!/bin/bash
# Init script with token server support

DEVICE_ID="${1:-}"
TOKEN_SOURCE="${2:-}"  # Can be token OR server URL

if [[ "${TOKEN_SOURCE}" =~ ^http ]]; then
    # Fetch token from server
    echo "Fetching token from server..."
    TUNNEL_TOKEN=$(curl -s -H "X-API-Key: ${API_KEY}" \
        "${TOKEN_SOURCE}/token/${DEVICE_ID}" | jq -r '.tunnel_token')

    if [[ "${TUNNEL_TOKEN}" == "null" ]]; then
        echo "Failed to fetch token for ${DEVICE_ID}"
        exit 1
    fi
else
    # Token provided directly
    TUNNEL_TOKEN="${TOKEN_SOURCE}"
fi

# Continue with provisioning...
echo "Configuring tunnel for ${DEVICE_ID}..."
sudo cloudflared service install "${TUNNEL_TOKEN}"

Usage:

# On RPi (with server running on 192.168.1.10:5000)
export API_KEY="your-secure-key"
sudo ./rpi-init.sh rpi-001 "http://192.168.1.10:5000"

Method 4: Batch Provisioning with Token Mapping

Best for: Provisioning multiple devices simultaneously

Create device mapping file (config/device-mapping.csv):

device_id,mac_address,ip_address,location
rpi-001,dc:a6:32:xx:xx:01,192.168.1.101,warehouse-north
rpi-002,dc:a6:32:xx:xx:02,192.168.1.102,warehouse-north
rpi-003,dc:a6:32:xx:xx:03,192.168.1.103,warehouse-south

Batch provisioning script (scripts/batch-provision.sh):

#!/bin/bash
# Batch provision devices using mapping file

set -euo pipefail

MAPPING_FILE="${1:-config/device-mapping.csv}"
CREDS_FILE="config/tunnel-credentials.json"
SSH_KEY="${HOME}/.ssh/id_rsa"

if [[ ! -f "${MAPPING_FILE}" ]]; then
    echo "Mapping file not found: ${MAPPING_FILE}"
    exit 1
fi

if [[ ! -f "${CREDS_FILE}" ]]; then
    echo "Credentials file not found: ${CREDS_FILE}"
    echo "Run: terraform output -json tunnel_credentials > ${CREDS_FILE}"
    exit 1
fi

# Skip header line
tail -n +2 "${MAPPING_FILE}" | while IFS=',' read -r DEVICE_ID MAC IP LOCATION; do
    echo "========================================"
    echo "Provisioning: ${DEVICE_ID}"
    echo "Location: ${LOCATION}"
    echo "IP: ${IP}"
    echo "========================================"

    # Get token
    TOKEN=$(jq -r ".\"${DEVICE_ID}\".tunnel_token" "${CREDS_FILE}")

    if [[ "${TOKEN}" == "null" ]]; then
        echo "❌ No token found for ${DEVICE_ID}, skipping..."
        continue
    fi

    # Check if device is reachable
    if ! ping -c 1 -W 2 "${IP}" > /dev/null 2>&1; then
        echo "❌ Device not reachable at ${IP}, skipping..."
        continue
    fi

    # Copy init script
    scp -i "${SSH_KEY}" scripts/rpi-init.sh pi@${IP}:/home/pi/

    # Run provisioning
    ssh -i "${SSH_KEY}" pi@${IP} "sudo ./rpi-init.sh ${DEVICE_ID} '${TOKEN}'"

    if [[ $? -eq 0 ]]; then
        echo "✅ ${DEVICE_ID} provisioned successfully"
    else
        echo "❌ ${DEVICE_ID} provisioning failed"
    fi

    echo ""
    sleep 2
done

echo "Batch provisioning completed"

Method 5: QR Code Distribution (Physical Deployment)

Best for: Field technicians, remote locations

Generate QR codes:

#!/bin/bash
# Generate QR codes with provisioning commands

mkdir -p qr-codes

for i in {1..100}; do
    DEVICE_ID=$(printf "rpi-%03d" $i)
    TOKEN=$(jq -r ".\"${DEVICE_ID}\".tunnel_token" config/tunnel-credentials.json)

    # Create provisioning command
    COMMAND="curl -sL https://setup.tagai.uk/init.sh | sudo bash -s ${DEVICE_ID} ${TOKEN}"

    # Generate QR code (requires qrencode)
    echo "${COMMAND}" | qrencode -o "qr-codes/${DEVICE_ID}.png"

    # Also create text file
    echo "${COMMAND}" > "qr-codes/${DEVICE_ID}.txt"
done

echo "QR codes generated in qr-codes/"

Field technician workflow: 1. Boot Raspberry Pi 2. Scan QR code with phone 3. SSH to device 4. Paste and execute command 5. Verify connection

Security Considerations

1. Token Storage

# ✅ Good: Environment variable
export TUNNEL_TOKEN="eyJhIjoiYzM5..."

# ✅ Good: Secure file with restricted permissions
chmod 600 config/tunnel-credentials.json

# ❌ Bad: Committed to Git
git add config/tunnel-credentials.json  # DON'T DO THIS

# ❌ Bad: Stored in plain text on shared drive

2. Token Server Security

• Use HTTPS in production
• Implement proper authentication (API keys, mutual TLS)
• Rate limiting
• Audit logging
• Token rotation capability

3. Transmission Security

# ✅ Good: HTTPS
curl https://token-server.internal/token/rpi-001

# ✅ Good: SSH
scp user@server:/secure/tokens.json .

# ❌ Bad: HTTP in production
curl http://token-server/token/rpi-001

# ❌ Bad: Unencrypted storage

Recommended Approach by Scale

Device Count	Recommended Method	Complexity	Setup Time
1-10	Direct Extraction	Low	Minutes
10-30	Batch Script	Medium	1-2 hours
30-50	Pre-configured Images	Medium	Half day
50-100	Token Server	High	1 day
100+	Token Server + Automation	High	2-3 days

Example: Complete Workflow for 100 Devices

# 1. Deploy infrastructure
cd terraform
terraform apply -var="device_count=100"

# 2. Export tokens
terraform output -json tunnel_credentials > ../config/tunnel-credentials.json

# 3. Start token server (in separate terminal)
cd ../scripts/token-server
export API_KEY="$(openssl rand -hex 32)"
python3 server.py &
SERVER_PID=$!

# 4. Create device mapping
cat > ../config/device-mapping.csv <<EOF
device_id,mac_address,ip_address,location
rpi-001,dc:a6:32:xx:xx:01,192.168.1.101,warehouse-1
rpi-002,dc:a6:32:xx:xx:02,192.168.1.102,warehouse-1
# ... more devices
EOF

# 5. Run batch provisioning
cd ..
./scripts/batch-provision.sh

# 6. Verify all devices
./scripts/validate-deployment.sh 100

# 7. Cleanup
kill $SERVER_PID

Maintenance Operations

Adding New Devices

# Increase device count
terraform apply -var="device_count=105"

Removing Devices

# Use terraform taint or targeted destroy
terraform destroy -target="cloudflare_tunnel.rpi[\"rpi-050\"]"

Updating Tunnel Configuration

# Modify tunnel config in terraform
terraform plan
terraform apply

Rotating Tunnel Credentials

# Taint specific tunnel to regenerate
terraform taint 'cloudflare_tunnel.rpi["rpi-001"]'
terraform apply

Testing Strategy

Phase 1: Single Device Test

1. Create infrastructure for rpi-001
2. Verify DNS resolution
3. Test tunnel connectivity
4. Validate Access authentication

Phase 2: Small Scale Test

1. Deploy 5 devices (rpi-001 to rpi-005)
2. Test different Access policies
3. Validate automation
4. Performance testing

Phase 3: Full Scale Deployment

1. Deploy all 100 devices
2. Monitor Cloudflare dashboard
3. Validate all devices accessible
4. Load testing (if applicable)

Monitoring and Observability

Metrics to Track

• Tunnel status (up/down)
• Access policy hits
• DNS query volume
• Authentication failures

Cloudflare Dashboard

• Zero Trust → Access → Logs
• Traffic → Analytics
• DNS → Analytics

Alerting

• Set up Cloudflare notifications
• Monitor tunnel connectivity
• Track authentication failures

Troubleshooting

Common Issues

Tunnel not connecting: - Check tunnel credentials - Verify DNS propagation - Check cloudflared service status

Access denied: - Verify Access policy includes user - Check authentication provider - Review Access logs

DNS not resolving: - Verify DNS record created - Check TTL and propagation - Test with dig or nslookup

Next Steps

1. Set up Cloudflare account and API access
2. Create Terraform workspace
3. Implement base Terraform code
4. Test with single device
5. Scale to full deployment

References

• Cloudflare Terraform Provider
• Cloudflare Tunnels Guide
• Cloudflare Access Setup