fix!: remove webp transformer

- Introduced a migration system for manifest files to support versioning and backward compatibility.
- Updated image processing to switch thumbnail format from WebP to JPEG for better compatibility.
- Added new commands for manifest migration and improved thumbnail handling in the builder.
- Enhanced documentation with architecture details and development commands.

Signed-off-by: Innei <tukon479@gmail.com>
This commit is contained in:
Innei
2025-09-13 23:11:00 +08:00
parent d3b59d9e88
commit 2df511bff7
14 changed files with 296 additions and 53 deletions

145
CLAUDE.md
View File

@@ -5,6 +5,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Commands
### Development Commands
```bash
# Start development server (runs both web and SSR)
pnpm dev
@@ -32,6 +33,7 @@ pnpm run build:manifest -- --force-manifest
```
### Database Commands (SSR app)
```bash
# Generate database migrations
pnpm --filter @afilmory/ssr db:generate
@@ -41,6 +43,7 @@ pnpm --filter @afilmory/ssr db:migrate
```
### Code Quality Commands
```bash
# Lint and fix code
pnpm lint
@@ -54,48 +57,131 @@ pnpm --filter web type-check
## Architecture
### Design Patterns & Application Structure
**Hybrid SPA + SSR Architecture**: The application uses a unique architecture where Next.js serves as both a hosting platform for the Vite-built SPA and a dynamic SEO/OG meta generator:
- **Production**: Next.js serves the pre-built SPA static assets and provides dynamic routes for SEO
- **Development**: Both servers run concurrently with the SSR app proxying to the SPA for seamless development
**Key Design Patterns**:
- **Adapter Pattern**: Builder system uses adapters for different storage providers (S3, GitHub)
- **Factory Pattern**: Photo processing pipeline with configurable workers and storage adapters
- **Observer Pattern**: Manifest changes trigger SSR meta tag updates for social sharing
- **Singleton Pattern**: PhotoLoader class provides global access to manifest data
### Monorepo Structure
This is a pnpm workspace with multiple applications and packages:
- `apps/web/` - Main frontend React application (Vite + React 19)
- `apps/ssr/` - Next.js SSR application for server-side rendering and APIs
- `packages/` - Shared packages and utilities
- `packages/builder/` - Photo processing and manifest generation tool
- `packages/webgl-viewer/` - WebGL-based photo viewer component
- `apps/web/` - Main frontend React application (Vite + React 19 SPA)
- `apps/ssr/` - Next.js 15 application serving as SPA host + dynamic SEO/OG generator
- `packages/builder/` - Photo processing and manifest generation tool with adapter pattern
- `packages/webgl-viewer/` - High-performance WebGL-based photo viewer component
- `packages/data/` - Shared data access layer and PhotoLoader singleton
- `packages/components/` - Reusable UI components across apps
### Next.js as SPA Host & SEO Provider
**Dual Server Architecture**:
- **Development Mode**: `apps/ssr/src/app/[...all]/route.ts` catches all SPA routes and serves index.html with injected manifest data
- **Production Mode**: Next.js serves pre-built Vite SPA assets while providing dynamic OG image generation
**Dynamic SEO Implementation**:
- `apps/ssr/src/index.html.ts` - Pre-compiled HTML template with manifest data injected as `window.__MANIFEST__`
- Dynamic OG images generated per photo via Next.js API routes (`/og/[photoId]/route.ts`)
- HTML meta tags dynamically replaced for social media sharing
### Configuration Architecture
**Two-Layer Configuration System**:
1. **Builder Config** (`builder.config.json`) - **Infrastructure/Processing Layer**:
```json
{
"storage": { "provider": "s3", "bucket": "...", "region": "..." },
"performance": { "worker": { "workerCount": 8, "useClusterMode": true } },
"repo": { "enable": true, "url": "...", "token": "..." }
}
```
- Controls photo processing, storage connections, and build performance
- Handles remote git repository sync for manifest/thumbnails
- Configures multi-process/cluster processing for large photo sets
2. **Site Config** (`site.config.ts` + `config.json`) - **Presentation/Content Layer**:
```typescript
{
name: "Gallery Name",
description: "...",
author: { name: "...", url: "...", avatar: "..." },
social: { twitter: "...", github: "..." },
map: ["maplibre"] // Map provider configuration
}
```
- Controls site branding, author info, social links
- Merged with user `config.json` using es-toolkit/compat
- Consumed by both SPA and SSR for consistent branding
### Manifest Generation & Data Flow
**Builder Pipeline** (`packages/builder/src/cli.ts`):
1. **Storage Sync**: Downloads photos from S3/GitHub with incremental change detection
2. **Format Processing**: HEIC→JPEG, TIFF→web formats, Live Photo detection
3. **Multi-threaded Processing**: Configurable worker pools or cluster mode for performance
4. **EXIF & Metadata Extraction**: Camera settings, GPS, Fujifilm recipes, tone analysis
5. **Thumbnail Generation**: Multiple sizes with blurhash placeholders
6. **Manifest Serialization**: Generates `photos-manifest.json` with full metadata
7. **Remote Sync**: Pushes updates to git repository if configured
**SPA Data Consumption** (`packages/data/src/index.ts`):
```typescript
class PhotoLoader {
constructor() {
this.photos = __MANIFEST__.data // Injected via window global
this.cameras = __MANIFEST__.cameras
this.lenses = __MANIFEST__.lenses
// Creates lookup maps and provides data access layer
}
}
```
**Data Flow**:
1. Builder generates manifest → `photos-manifest.json`
2. SSR injects manifest into HTML → `window.__MANIFEST__`
3. SPA PhotoLoader singleton consumes global data
4. React components access photos via `photoLoader.getPhotos()`
### Key Technologies
- **Frontend**: React 19, TypeScript, Vite, Tailwind CSS, Jotai (state), TanStack Query
- **Backend**: Next.js 15, Drizzle ORM, PostgreSQL
- **Image Processing**: Sharp, EXIF extraction, WebGL rendering
- **Storage**: S3-compatible storage, GitHub storage support
- **Build Tools**: pnpm workspaces, ESLint, Prettier
### Configuration Files
- `builder.config.json` - Photo processing and storage configuration
- `config.json` - Site configuration (name, description, author, etc.)
- `site.config.ts` - TypeScript site configuration with defaults
- `env.ts` - Environment variables validation and types
### Photo Processing Pipeline
1. **Storage Sync**: Fetches photos from configured storage (S3/GitHub)
2. **Format Conversion**: Converts HEIC, TIFF to web-compatible formats
3. **Thumbnail Generation**: Creates multiple sizes for performance
4. **EXIF Extraction**: Extracts camera settings and GPS data
5. **Manifest Generation**: Creates `photos-manifest.json` with metadata
- **Backend**: Next.js 15 (SPA host + SEO), Drizzle ORM, PostgreSQL
- **Image Processing**: Sharp, exiftool-vendored, HEIC conversion, blurhash generation
- **Storage**: S3-compatible (AWS/MinIO), GitHub repository storage
- **Build System**: pnpm workspaces, concurrent dev servers, cluster-based processing
### Development Workflow
- Web app runs on development server with hot reload
- SSR app provides APIs and server-side rendering
- Builder tool processes photos and generates metadata
- Database migrations handle schema changes
- **Concurrent Development**: `pnpm dev` runs both SPA (Vite) and SSR (Next.js) servers
- **Hot Reloading**: SPA changes reflect immediately, SSR provides SEO preview
- **Manifest Building**: `pnpm run build:manifest` processes photos and updates data
- **Type Safety**: Shared types between builder, SPA, and SSR ensure consistency
### Code Quality Rules
1. Avoid code duplication - extract common types and components
2. Keep components focused - use hooks and component composition
3. Follow React best practices - proper Context usage, state management
4. Use TypeScript strictly - leverage type safety throughout
### i18n Guidelines
- Use flat keys with `.` separation (e.g., `exif.camera.model`)
- Support pluralization with `_one` and `_other` suffixes
- Modify English first, then other languages (ESLint auto-removes unused keys)
@@ -106,6 +192,7 @@ This is a pnpm workspace with multiple applications and packages:
- Each key must be completely independent in the flat structure
### Testing Strategy
- Check README.md and package.json scripts for test commands
- Verify builds work with `pnpm build`
- Test photo processing with `pnpm run build:manifest`
@@ -114,17 +201,20 @@ This is a pnpm workspace with multiple applications and packages:
## Cursor Rules Integration
### Code Quality Standards
- Avoid code duplication - extract common types and components when used multiple times
- Keep components focused - use hooks and component splitting for large logic blocks
- Master React philosophy - proper Context usage, component composition, state management to prevent re-renders
### UI/UX Guidelines
- Use Apple UIKit color system via tailwind-uikit-colors package
- Prefer semantic color names: `text-primary`, `fill-secondary`, `material-thin`, etc.
- Follow system colors: `red`, `blue`, `green`, `mint`, `teal`, `cyan`, `indigo`, `purple`, `pink`, `brown`, `gray`
- Use material design principles with opacity-based fills and proper contrast
### i18n Development Rules
- Use flat keys with dot notation: `exif.camera.model`
- Support pluralization: `_one` and `_other` suffixes
- Always modify English (`en.json`) first, then other languages
@@ -132,8 +222,9 @@ This is a pnpm workspace with multiple applications and packages:
- ESLint automatically removes unused keys from non-English files
## Important Notes
- This is a photo gallery application that processes and displays photos from cloud storage
- The builder tool handles complex image processing workflows
- WebGL viewer provides high-performance photo viewing experience
- Map integration shows photo locations from GPS EXIF data
- Live Photo support for iOS/Apple device videos
- Live Photo support for iOS/Apple device videos

View File

@@ -14,5 +14,5 @@ const jsContent = \`export default \\\`\${html.replace(/\`/g, '\\\\\`').replace(
fs.writeFileSync('./src/index.html.ts', jsContent);
"
rm ./public/index.html
pnpm build:jpg
# pnpm build:jpg
pnpm build:next

View File

@@ -27,7 +27,6 @@ export const GET = async (
// 处理标签
const tags = photo.tags?.slice(0, 3).join(' • ') || ''
// Format EXIF information
const formatExifInfo = () => {
if (!photo.exif) return null

View File

@@ -88,7 +88,7 @@ function generateRSSFeed(
<description><![CDATA[${description}]]></description>
<pubDate>${pubDate}</pubDate>
${photo.tags.map((tag) => `<category><![CDATA[${tag}]]></category>`).join('\n ')}
<enclosure url="${photo.thumbnailUrl.startsWith('http') ? photo.thumbnailUrl : config.url + photo.thumbnailUrl}" type="image/webp" length="${photo.size}" />
<enclosure url="${photo.thumbnailUrl.startsWith('http') ? photo.thumbnailUrl : config.url + photo.thumbnailUrl}" type="image/jpeg" length="${photo.size}" />
${exifTags}
</item>`
})

View File

@@ -20,6 +20,7 @@
"extract:font": "tsx scripts/extract-font-glyphs.ts",
"format": "prettier --write \"src/**/*.ts\" ",
"lint": "eslint --fix",
"migrate:manifest": "tsx scripts/migrate-manifest.ts",
"preinstall": "sh scripts/preinstall.sh",
"prepare": "simple-git-hooks",
"update:lastmodified": "tsx scripts/update-lastmodified.ts"

View File

@@ -11,6 +11,7 @@ import {
needsUpdate,
saveManifest,
} from '../manifest/manager.js'
import { CURRENT_MANIFEST_VERSION } from '../manifest/version.js'
import type { PhotoProcessorOptions } from '../photo/processor.js'
import { processPhoto } from '../photo/processor.js'
import { StorageManager } from '../storage/index.js'
@@ -321,7 +322,7 @@ class PhotoGalleryBuilder {
): Promise<AfilmoryManifest> {
return options.isForceMode || options.isForceManifest
? {
version: 'v6',
version: CURRENT_MANIFEST_VERSION,
data: [],
cameras: [],
lenses: [],

View File

@@ -3,6 +3,7 @@ import 'dotenv-expand/config'
import { execSync } from 'node:child_process'
import cluster from 'node:cluster'
import { existsSync } from 'node:fs'
import fs from 'node:fs/promises'
import path from 'node:path'
import process from 'node:process'
@@ -182,10 +183,13 @@ async function main() {
// 创建空的 manifest 文件(如果不存在)
if (!existsSync(manifestSourcePath)) {
logger.main.info('📄 创建初始 manifest 文件...')
await $({
cwd: assetsGitDir,
stdio: 'inherit',
})`echo '{"version":"v2","data":[]}' > photos-manifest.json`
const { CURRENT_MANIFEST_VERSION } = await import('./manifest/version.js')
const initial = JSON.stringify(
{ version: CURRENT_MANIFEST_VERSION, data: [] },
null,
2,
)
await fs.writeFile(manifestSourcePath, initial)
}
// 删除 public/thumbnails 目录,并建立软连接到 assets-git/thumbnails

View File

@@ -15,7 +15,7 @@ const THUMBNAIL_WIDTH = 600
// 获取缩略图路径信息
function getThumbnailPaths(photoId: string) {
const filename = `${photoId}.webp`
const filename = `${photoId}.jpg`
const thumbnailPath = path.join(THUMBNAIL_DIR, filename)
const thumbnailUrl = `/thumbnails/${filename}`
@@ -101,9 +101,7 @@ async function generateNewThumbnail(
.resize(THUMBNAIL_WIDTH, null, {
withoutEnlargement: true,
})
.webp({
quality: THUMBNAIL_QUALITY,
})
.jpeg({ quality: THUMBNAIL_QUALITY })
.toBuffer()
// 保存到文件

View File

@@ -11,6 +11,8 @@ import type {
LensInfo,
} from '../types/manifest.js'
import type { PhotoManifestItem } from '../types/photo.js'
import { migrateManifestFileIfNeeded } from './migrate.js'
import { CURRENT_MANIFEST_VERSION } from './version.js'
const manifestPath = path.join(workdir, 'src/data/photos-manifest.json')
@@ -24,21 +26,16 @@ export async function loadExistingManifest(): Promise<AfilmoryManifest> {
'🔍 未找到 manifest 文件/解析失败,创建新的 manifest 文件...',
)
return {
version: 'v6',
version: CURRENT_MANIFEST_VERSION,
data: [],
cameras: [],
lenses: [],
}
}
if (manifest.version !== 'v6') {
logger.fs.error('🔍 无效的 manifest 版本,创建新的 manifest 文件...')
return {
version: 'v6',
data: [],
cameras: [],
lenses: [],
}
if (manifest.version !== CURRENT_MANIFEST_VERSION) {
const migrated = await migrateManifestFileIfNeeded(manifest)
if (migrated) return migrated
}
// 向后兼容:如果现有 manifest 没有 cameras 和 lenses 字段,则添加空数组
@@ -82,7 +79,7 @@ export async function saveManifest(
manifestPath,
JSON.stringify(
{
version: 'v6',
version: CURRENT_MANIFEST_VERSION,
data: sortedManifest,
cameras,
lenses,
@@ -117,7 +114,7 @@ export async function handleDeletedPhotos(
const manifestKeySet = new Set(items.map((item) => item.id))
for (const thumbnail of allThumbnails) {
if (!manifestKeySet.has(basename(thumbnail, '.webp'))) {
if (!manifestKeySet.has(basename(thumbnail, '.jpg'))) {
await fs.unlink(path.join(workdir, 'public/thumbnails', thumbnail))
deletedCount++
}

View File

@@ -0,0 +1,120 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import { workdir } from '@afilmory/builder/path.js'
import { logger } from '../logger/index.js'
import type { AfilmoryManifest } from '../types/manifest.js'
import type { ManifestVersion } from './version.js'
import { CURRENT_MANIFEST_VERSION } from './version.js'
const manifestPath = path.join(workdir, 'src/data/photos-manifest.json')
// Placeholder migration scaffolding (chain-of-executors)
// Supports sequential migrations: v1 -> v2 -> v3 -> ... -> CURRENT
export type MigrationContext = {
from: ManifestVersion | string
to: ManifestVersion
}
export type ManifestMigrator = (
raw: AfilmoryManifest,
ctx: MigrationContext,
) => AfilmoryManifest
export type MigrationStep = {
from: ManifestVersion | string
to: ManifestVersion
exec: ManifestMigrator
}
// Registry of ordered migration steps. Keep empty until concrete steps are added.
const MIGRATION_STEPS: MigrationStep[] = [
{
from: 'v1',
to: 'v6',
exec: () => {
logger.fs.error('🔍 无效的 manifest 版本,创建新的 manifest 文件...')
return {
version: 'v6',
data: [],
cameras: [],
lenses: [],
}
},
},
{
from: 'v6',
to: 'v7',
exec: (raw) => {
raw.data.forEach((item) => {
if (typeof item.thumbnailUrl === 'string') {
item.thumbnailUrl = item.thumbnailUrl.replace(/\.webp$/, '.jpg')
}
})
// 更新版本号为目标版本
;(raw as any).version = 'v7'
return raw
},
},
]
function noOpBumpVersion(raw: any, _target: ManifestVersion): AfilmoryManifest {
return raw
}
export function migrateManifest(
raw: AfilmoryManifest,
target: ManifestVersion = CURRENT_MANIFEST_VERSION,
): AfilmoryManifest {
let current: ManifestVersion | string = (raw?.version as any) ?? 'unknown'
let working = raw
// Iterate through chain-of-executors until reaching target.
// If no matching step is found for the current version, fallback to a no-op bump.
const guard = new Set<string>()
while (current !== target) {
const guardKey = `${String(current)}->${String(target)}`
if (guard.has(guardKey)) {
logger.main.warn('⚠️ 检测到潜在迁移循环,使用占位升级直接跳转到目标版本')
return noOpBumpVersion(working, target)
}
guard.add(guardKey)
const step = MIGRATION_STEPS.find((s) => s.from === current)
if (!step) {
// No concrete step for this source version; do a simple version bump once.
logger.main.info(
`🔄 迁移占位:${String(current)} -> ${target}(无匹配步骤,直接提升版本)`,
)
return noOpBumpVersion(working, target)
}
const ctx: MigrationContext = { from: step.from, to: step.to }
logger.main.info(
`🔁 执行迁移步骤:${String(step.from)} -> ${String(step.to)}`,
)
working = step.exec(working, ctx)
current = (working?.version as any) ?? step.to
}
// Already at target
return working as AfilmoryManifest
}
export async function migrateManifestFileIfNeeded(
parsed: AfilmoryManifest,
): Promise<AfilmoryManifest | null> {
try {
if (parsed?.version === CURRENT_MANIFEST_VERSION) return null
const migrated = migrateManifest(parsed, CURRENT_MANIFEST_VERSION)
await fs.mkdir(path.dirname(manifestPath), { recursive: true })
await fs.writeFile(manifestPath, JSON.stringify(migrated, null, 2))
logger.main.success(`✅ Manifest 版本已更新为 ${CURRENT_MANIFEST_VERSION}`)
return migrated
} catch (e) {
logger.main.error('❌ Manifest 迁移失败(占位实现):', e)
throw e
}
}

View File

@@ -0,0 +1,3 @@
export type ManifestVersion = `v${number}`
export const CURRENT_MANIFEST_VERSION: ManifestVersion = 'v7'

View File

@@ -49,10 +49,10 @@ export async function processThumbnailAndBlurhash(
const thumbnailPath = path.join(
workdir,
'public/thumbnails',
`${photoId}.webp`,
`${photoId}.jpg`,
)
const thumbnailBuffer = await fs.readFile(thumbnailPath)
const thumbnailUrl = `/thumbnails/${photoId}.webp`
const thumbnailUrl = `/thumbnails/${photoId}.jpg`
loggers.blurhash.info(`复用现有 blurhash: ${photoId}`)
loggers.thumbnail.info(`复用现有缩略图:${photoId}`)

View File

@@ -1,3 +1,4 @@
import type { ManifestVersion } from '../manifest/version'
import type { PhotoManifestItem } from './photo'
export interface CameraInfo {
@@ -13,7 +14,7 @@ export interface LensInfo {
}
export type AfilmoryManifest = {
version: 'v6'
version: ManifestVersion
data: PhotoManifestItem[]
cameras: CameraInfo[] // Unique cameras found in all photos
lenses: LensInfo[] // Unique lenses found in all photos

View File

@@ -0,0 +1,28 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import { migrateManifest } from '../packages/builder/src/manifest/migrate'
import { CURRENT_MANIFEST_VERSION } from '../packages/builder/src/manifest/version'
async function run() {
const manifestPath = path.resolve(
process.cwd(),
'src/data/photos-manifest.json',
)
try {
const raw = await fs.readFile(manifestPath, 'utf-8')
const parsed = JSON.parse(raw)
if (parsed?.version === CURRENT_MANIFEST_VERSION) {
console.info(`Manifest 已是 ${CURRENT_MANIFEST_VERSION},跳过迁移`)
return
}
const migrated = migrateManifest(parsed, CURRENT_MANIFEST_VERSION)
await fs.writeFile(manifestPath, JSON.stringify(migrated, null, 2))
console.info(`✅ Manifest 迁移完成 -> ${CURRENT_MANIFEST_VERSION}`)
} catch (e) {
console.error('❌ 迁移失败:', e)
process.exitCode = 1
}
}
run()