---
title: Storage providers
description: Afilmory can work with multiple storage providers, including S3, Git, Eagle, and local file system
createdAt: 2025-08-12T15:09:08+08:00
lastModified: 2025-10-28T19:48:05+08:00
---

# Storage Providers

Afilmory's flexible storage architecture allows you to store your photos across different platforms. The photo processing engine (`@afilmory/builder`) abstracts storage operations through a unified interface, making it easy to switch between providers such as S3, GitHub, Eagle, and the local file system.

## Supported Providers

### S3-Compatible Storage

S3-compatible storage is the recommended option for production deployments, offering scalability, CDN integration, and reliable performance.

**Configuration in `builder.config.ts`:**

```ts
import { defineBuilderConfig } from '@afilmory/builder'

export default defineBuilderConfig(() => ({
  storage: {
    provider: 's3',
    bucket: 'your-photos-bucket',
    region: 'us-east-1',
    prefix: 'photos/',
    customDomain: 'cdn.yourdomain.com',
  },
}))
```

**Environment Variables (`.env`):**

```bash
S3_ACCESS_KEY_ID=your_access_key_id
S3_SECRET_ACCESS_KEY=your_secret_access_key
```

### GitHub Storage

GitHub storage leverages Git repositories for photo storage, ideal for static sites, small galleries, or when you want version control for your photos.

**Features:**

- ✅ Free storage space (GitHub repository limit 1GB)
- ✅ Global CDN support via raw.githubusercontent.com
- ✅ Version control for your photos
- ✅ Public access without authentication
- ⚠️ GitHub API rate limits apply
- ⚠️ Not suitable for large files or frequent updates

**Configuration in `builder.config.ts`:**

```ts
export default defineBuilderConfig(() => ({
  storage: {
    provider: 'github',
    owner: 'your-username',
    repo: 'photo-storage',
    branch: 'main',
    path: 'photos',
    useRawUrl: true,
  },
}))
```

**Environment Variables:**

```bash
GIT_TOKEN=ghp_your_github_personal_access_token
```

**Setup Steps:**

1. **Create GitHub Repository**

   ```bash
   git clone https://github.com/your-username/photo-gallery.git
   cd photo-gallery
   mkdir photos
   ```

2. **Get GitHub Access Token** (optional but recommended)
   - Visit GitHub Settings > Developer settings > Personal access tokens
   - Create new Fine-grained personal access token
   - Grant "Contents" permission (read/write) to your repository

3. **Required Permissions:**
   - `Contents: Read and write` - for uploading processed photos
   - `Metadata: Read` - for repository access

### Eagle Storage

Eagle storage integrates directly with an existing Eagle 4 desktop library, making it ideal for teams that already curate assets in Eagle and want to publish selected photos without additional uploads.

**Features:**

- ✅ No upload step: reads directly from the Eagle library
- ✅ Flexible filtering: include or exclude by folder (with optional subfolders) and tags
- ✅ Automatic publishing: copies originals to a public directory on demand
- ⚠️ Requires absolute paths for `libraryPath` and `distPath`
- ⚠️ Only Eagle 4.x is validated; other versions may require testing

**Configuration in `builder.config.ts`:**

```ts
export default defineBuilderConfig(() => ({
  storage: {
    provider: 'eagle',
    libraryPath: '/Users/alice/Pictures/Eagle.library',
    distPath: '/Users/alice/workspaces/afilmory/apps/web/public/originals',
    baseUrl: '/originals/',
    include: [
      { type: 'folder', name: 'Published', includeSubfolder: true },
      { type: 'tag', name: 'Featured' },
    ],
    exclude: [{ type: 'tag', name: 'Private' }],
  },
}))
```

**Setup Steps:**

1. **Locate the Eagle library**: copy the absolute path shown in Eagle → Preferences → Library.
2. **Choose a distribution path (optional)**: The original images will be copied to `distPath` during the build process (defaults to `<workdir>/public/originals`).
3. **Align URLs**: set `baseUrl` to match how the frontend serves `distPath` (defaults to `/originals/`).
4. **Refine filters (optional)**: use `include` and `exclude` rules to control which assets are exported.

**Rule Notes:**

- Leaving `include` empty processes every supported image in the library.
- `exclude` overrides inclusion rules when both match a single asset.
- `folder` rules use folder names (not full paths); enabling `includeSubfolder` recursively matches children.
- `tag` rules are case-sensitive and match Eagle tags.

### Local File System

Local storage is suitable for development, testing, or self-hosted deployments where photos are stored on the same server.

**Features:**

- ✅ No external dependencies
- ✅ Fast access speeds
- ✅ Complete private control
- ✅ Recursive directory scanning
- ✅ Live Photos detection support
- ⚠️ Requires proper file system permissions
- ⚠️ Not suitable for distributed deployments

**Configuration in `builder.config.ts`:**

Self-hosted original image (manually serve to `basePath`):

```ts
import { defineBuilderConfig } from '@afilmory/builder'
export default defineBuilderConfig(() => ({
  storage: {
    provider: 'local',
    basePath: './photos',
    baseUrl: 'http://example.com:3000/photos/',
  },
}))
```

Hosted with frontend (serve via app static assets):

```json
export default defineBuilderConfig(() => ({
  storage: {
    provider: 'local',
    basePath: './photos',
    distPath: './apps/web/public/originals',
    baseUrl: '/originals/',
  },
}))
```

Use when you want originals deployed alongside your frontend’s static assets. On initialization/build, files from `basePath` are copied to `distPath` (overwriting same-named files).

## Photo Processing Workflow

### Image Processing Pipeline

When photos are processed, Afilmory performs these operations:

1. **Format Detection** - Identifies HEIC, TIFF, RAW, and standard formats
2. **EXIF Extraction** - Extracts camera metadata, GPS data, and Fujifilm film recipes
3. **Format Conversion** - Converts HEIC/TIFF to web-compatible formats (WebP, JPEG)
4. **Thumbnail Generation** - Creates multiple thumbnail sizes
5. **Blurhash Generation** - Creates placeholder images for smooth loading
6. **Manifest Creation** - Generates metadata JSON for the gallery

### Storage Organization

Afilmory organizes files in your storage with this structure:

```
photos/
├── 2024/
│   ├── 01-january/
│   │   ├── IMG_001.jpg
│   │   ├── IMG_001.mov      # Live Photo video
│   │   └── IMG_002.heic
│   └── 02-february/
│       └── sunset.jpg
├── 2023/
│   └── vacation/
│       ├── beach.jpg
│       └── mountain.png
└── manifest.json            # Photo metadata and index
```

### Processing Commands

Use these commands to manage your photo processing:

```bash
# Initial photo processing
pnpm run build:manifest

# Force rebuild all photos
pnpm run build:manifest -- --force

# Regenerate only thumbnails
pnpm run build:manifest -- --force-thumbnails

# Process specific directory
pnpm run build:manifest -- --path "2024/vacation"
```

## Environment-Specific Configurations

### Development Setup

For local development, use local storage:

```json
{
  "storage": {
    "provider": "local",
    "basePath": "./dev-photos",
    "baseUrl": "http://localhost:1924/photos"
  }
}
```

### Production Setup

For production, use S3-compatible storage with CDN:

```json
{
  "storage": {
    "provider": "s3",
    "bucket": "prod-photos-bucket",
    "region": "us-east-1",
    "prefix": "photos/",
    "customDomain": "cdn.yourdomain.com"
  },
  "performance": {
    "worker": {
      "enabled": true,
      "maxWorkers": 4
    }
  }
}
```

## API Limitations

### GitHub Storage Limits

- **Unauthenticated requests**: 60 requests/hour/IP
- **Authenticated requests**: 5,000 requests/hour/token
- **File size**: Maximum 100MB (via API)
- **Repository size**: Recommended not to exceed 1GB

### Storage Provider Comparison

| Feature          | S3              | GitHub                | Local                    |
| ---------------- | --------------- | --------------------- | ------------------------ |
| Storage Space    | Pay-as-you-go   | 1GB free              | Depends on disk          |
| CDN              | Additional cost | Free global CDN       | Manual setup             |
| API Limits       | Very high       | Limited               | None                     |
| Use Case         | Production      | Small projects, demos | Development, self-hosted |
| Setup Complexity | Medium          | Simple                | Minimal                  |

## Security Considerations

### S3 Bucket Policies

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-bucket/photos/*"
    }
  ]
}
```

### GitHub Repository Settings

- Set repository to public for image access
- Use fine-grained personal access tokens
- Limit token permissions to specific repositories

### Environment Variables

Store sensitive credentials securely:

```bash
# Use environment-specific .env files
# .env.production
S3_ACCESS_KEY_ID=prod_access_key
S3_SECRET_ACCESS_KEY=prod_secret_key

# .env.development
S3_ACCESS_KEY_ID=dev_access_key
S3_SECRET_ACCESS_KEY=dev_secret_key
```
