feat: implement remote repository support for manifest updates

- Added functionality to push updated manifests to a remote Git repository after building.
- Introduced a Git token configuration in the builder settings to authenticate pushes.
- Enhanced the CLI to handle remote repository synchronization, including cloning and pulling updates.
- Updated the builder to return detailed results on whether updates occurred during the build process.
- Revised documentation to reflect new configuration options and usage instructions.

Signed-off-by: Innei <tukon479@gmail.com>
This commit is contained in:
Innei
2025-06-28 13:34:35 +08:00
parent 6668425fba
commit 6d430faa9a
9 changed files with 963 additions and 401 deletions

View File

@@ -0,0 +1,742 @@
<!-- Generated by SpecStory -->
# 更新流程和 git 上传配置 (2025-06-27 01:14:44)
_**User**_
这里需要 pull或者 clone 之后,再跑一下增量更新的流程,如果有更新,还需要吧更新之后生成的 manifest 的上传到 git 仓库。
上传到 git 仓库需要提供 api token如果没有就跳过。
记得增加 logger 输出
---
_**Assistant**_
我需要查看一下相关的配置文件和构建器代码,了解当前的结构,然后修改这段代码来实现你要求的功能。
---
<details>
<summary>Searched codebase "builderConfig repo" • **24** results</summary>
| File | Lines |
|------|-------|
| `builder.config.example.json` | L1-6 |
| `builder.config.ts` | L71-113 |
| `packages/builder/src/storage/interfaces.ts` | L59-69 |
| `packages/builder/src/builder/builder.ts` | L42-64 |
| `builder.config.ts` | L128-130 |
| `builder.config.ts` | L1-71 |
| `builder.config.ts` | L113-128 |
| `scripts/preinstall.sh` | L1-3 |
| `packages/builder/src/storage/interfaces.ts` | L69-70 |
| `builder.config.ts` | L130-136 |
| `packages/builder/src/storage/interfaces.ts` | L46-59 |
| `README.md` | L219-311 |
| `packages/builder/src/builder/builder.ts` | L362-369 |
| `packages/builder/tsconfig.json` | L1-43 |
| `packages/builder/src/builder/builder.ts` | L27-42 |
| `apps/web/tsconfig.json` | L1-48 |
| `packages/builder/src/index.ts` | L1-2 |
| `packages/builder/src/index.ts` | L2-3 |
| `packages/builder/src/builder/builder.ts` | L1-27 |
| `packages/builder/src/storage/factory.ts` | L1-21 |
| `packages/builder/src/builder/index.ts` | L1-2 |
| `README.md` | L123-219 |
| `packages/builder/src/builder/index.ts` | L1-1 |
| `packages/builder/README.md` | L1-58 |
</details>
---
<details>
<summary>Searched codebase "defaultBuilder buildManifest" • **25** results</summary>
| File | Lines |
|------|-------|
| `packages/data/package.json` | L1-13 |
| `packages/builder/src/builder/builder.ts` | L281-292 |
| `packages/builder/src/builder/builder.ts` | L69-78 |
| `packages/builder/src/index.ts` | L34-42 |
| `README.md` | L219-311 |
| `packages/builder/src/builder/builder.ts` | L1-27 |
| `packages/builder/src/builder/builder.ts` | L228-281 |
| `packages/builder/src/builder/builder.ts` | L78-137 |
| `README.md` | L123-219 |
| `packages/builder/src/cli.ts` | L145-179 |
| `packages/builder/src/builder/builder.ts` | L329-355 |
| `scripts/photo-loader.ts` | L1-35 |
| `packages/builder/src/manifest/manager.ts` | L1-30 |
| `packages/builder/src/manifest/manager.ts` | L44-67 |
| `packages/builder/src/types/manifest.ts` | L1-6 |
| `packages/builder/src/builder/builder.ts` | L137-167 |
| `apps/web/src/global.d.ts` | L1-42 |
| `packages/builder/src/manifest/manager.ts` | L30-44 |
| `README.zh.md` | L219-323 |
| `builder.config.ts` | L113-128 |
| `README.zh.md` | L125-219 |
| `packages/builder/src/cli.ts` | L63-104 |
| `packages/builder/README.md` | L1-58 |
| `builder.config.ts` | L71-113 |
| `apps/web/plugins/vite/feed-sitemap.ts` | L1-55 |
</details>
---
<details>
<summary>Searched codebase "builder.config" • **2** results</summary>
| File |
|------|
| `undefined` |
| `undefined` |
</details>
---
现在让我查看一下构建器的更多细节,特别是如何检测更新和当前的 git 操作支持:
---
Read file: builder.config.ts
---
<details>
<summary>Searched codebase "git push commit upload manifest" • **23** results</summary>
| File | Lines |
|------|-------|
| `scripts/build-update-remote-repo.sh` | L1-14 |
| `packages/builder/src/manifest/manager.ts` | L44-67 |
| `packages/builder/src/index.ts` | L34-42 |
| `packages/builder/src/manifest/manager.ts` | L30-44 |
| `packages/builder/src/manifest/manager.ts` | L1-30 |
| `packages/builder/src/builder/builder.ts` | L228-281 |
| `apps/web/scripts/pull-remote.ts` | L1-49 |
| `.git/refs/remotes/origin/main` | L1-2 |
| `README.md` | L219-311 |
| `apps/web/plugins/vite/manifest-inject.ts` | L1-50 |
| `package.json` | L1-61 |
| `README.md` | L123-219 |
| `packages/builder/src/runAsWorker.ts` | L20-26 |
| `packages/builder/src/builder/builder.ts` | L69-78 |
| `packages/builder/src/builder/builder.ts` | L78-137 |
| `packages/builder/src/builder/builder.ts` | L281-292 |
| `packages/builder/src/runAsWorker.ts` | L174-237 |
| `packages/data/package.json` | L1-13 |
| `apps/web/plugins/vite/__internal__/constants.ts` | L1-7 |
| `apps/web/vite.config.ts` | L36-103 |
| `.git/refs/heads/main` | L1-2 |
| `packages/builder/src/types/manifest.ts` | L1-6 |
| `apps/web/src/pages/(data)/manifest.tsx` | L187-226 |
</details>
---
现在我需要查看 env.ts 文件以了解可能的环境变量配置:
---
Read file: env.ts
---
现在我基本了解了当前的结构。我需要修改以下几个文件:
1. 修改 `builder.config.ts` 中的 `BuilderConfig` 接口,添加 git token 配置
2. 修改 `env.ts` 添加 git token 环境变量
3. 修改 `packages/builder/src/cli.ts` 中的逻辑,在 pull/clone 后运行增量更新,并在有更新时上传到 git 仓库
4. 创建一个 git 操作的工具函数
让我开始实现:
---
```diff
PG_CONNECTION_STRING: z.string().min(1).optional(),
+
+ // Git token for uploading updated manifest to remote repository
+ GIT_TOKEN: z.string().optional(),
},
runtimeEnv: process.env,
```
---
```diff
enable: boolean
url: string
+ // Git token for pushing updates back to the repository
+ token?: string
}
// 存储配置
```
---
```diff
enable: false,
url: '',
+ token: env.GIT_TOKEN,
},
```
---
现在我需要创建一个 git 操作的工具函数:
---
```diff
import { workdir } from './path.js'
import { runAsWorker } from './runAsWorker.js'
+
+ /**
+ * 推送更新后的 manifest 到远程仓库
+ */
+ async function pushManifestToRemoteRepo(): Promise<boolean> {
+ if (!builderConfig.repo.enable || !builderConfig.repo.token) {
+ if (!builderConfig.repo.enable) {
+ logger.main.info('🔧 远程仓库未启用,跳过推送')
+ } else {
+ logger.main.warn('⚠️ 未提供 Git Token跳过推送到远程仓库')
+ }
+ return false
+ }
+
+ try {
+ const assetsGitDir = path.resolve(workdir, 'assets-git')
+
+ if (!existsSync(assetsGitDir)) {
+ logger.main.error('❌ assets-git 目录不存在,无法推送')
+ return false
+ }
+
+ logger.main.info('📤 开始推送更新到远程仓库...')
+
+ // 检查是否有变更
+ const status = await $({
+ cwd: assetsGitDir,
+ stdio: 'pipe',
+ })`git status --porcelain`
+
+ if (!status.stdout.trim()) {
+ logger.main.info('💡 没有变更需要推送')
+ return false
+ }
+
+ logger.main.info('📋 检测到以下变更:')
+ logger.main.info(status.stdout)
+
+ // 配置 git 凭据
+ const repoUrl = builderConfig.repo.url
+ const token = builderConfig.repo.token
+
+ // 解析仓库 URL添加 token
+ let authenticatedUrl = repoUrl
+ if (repoUrl.startsWith('https://github.com/')) {
+ const urlWithoutProtocol = repoUrl.replace('https://', '')
+ authenticatedUrl = `https://${token}@${urlWithoutProtocol}`
+ }
+
+ // 设置远程仓库 URL包含 token
+ await $({
+ cwd: assetsGitDir,
+ stdio: 'pipe',
+ })`git remote set-url origin ${authenticatedUrl}`
+
+ // 添加所有变更
+ await $({
+ cwd: assetsGitDir,
+ stdio: 'inherit',
+ })`git add .`
+
+ // 提交变更
+ const commitMessage = `chore: update photos-manifest.json and thumbnails - ${new Date().toISOString()}`
+ await $({
+ cwd: assetsGitDir,
+ stdio: 'inherit',
+ })`git commit -m ${commitMessage}`
+
+ // 推送到远程仓库
+ await $({
+ cwd: assetsGitDir,
+ stdio: 'inherit',
+ })`git push origin HEAD`
+
+ logger.main.success('✅ 成功推送更新到远程仓库')
+ return true
+
+ } catch (error) {
+ logger.main.error('❌ 推送到远程仓库失败:', error)
+ return false
+ }
+ }
async function main() {
```
---
现在我需要修改 main 函数的逻辑,在 pull/clone 远程仓库后运行增量更新,然后推送更新:
---
---
现在我需要在构建完成后调用推送函数:
---
---
现在我需要修改 builder 构建器,让它返回是否有更新的信息。首先查看 builder 的返回类型:
---
Read file: packages/builder/src/builder/builder.ts
---
我需要修改 `BuilderOptions``buildManifest` 方法,使其返回构建结果信息:
---
---
---
我需要修改 `#buildManifest` 方法的返回类型和返回值:
---
---
我需要修改方法末尾,使其返回 BuilderResult以及处理早期返回的情况
---
---
现在我需要在方法的最后添加返回语句:
---
---
现在我需要更新 `builder/index.ts` 导出新的类型:
---
```diff
- export type { BuilderOptions } from './builder.js'
+ export type { BuilderOptions, BuilderResult } from './builder.js'
export { defaultBuilder } from './builder.js'
```
---
现在我需要修改 CLI 代码,使用返回的构建结果来决定是否推送:
---
---
现在我需要更新 `builder.config.example.json` 文件,添加 token 字段的示例:
---
Read file: builder.config.example.json
---
```diff
"repo": {
"enable": true,
- "url": "https://github.com/Innei/gallery-public"
+ "url": "https://github.com/Innei/gallery-public",
+ "token": "your_github_token_here"
}
}
```
---
最后,我需要更新帮助信息,说明新的 token 配置:
---
---
现在我需要在 `--config` 选项中显示 token 的状态信息:
---
---
现在我测试一下代码是否正常工作。让我运行构建命令查看配置:
---
```bash
cd packages/builder && pnpm run cli -- --config
```
```
> @afilmory/builder@0.0.1 cli /Users/innei/git/vibe-coding/photo-gallery-site/pa
ckages/builder
> tsx src/cli.ts -- --config
🔄 同步远程仓库...
MAIN 5:05:04 PM
🔄 拉取远程仓库更新...
MAIN 5:05:04 PM
Already up to date.
✔ ✅ 远程仓库同步完成
MAIN 5:05:05 PM
🔧 当前配置:
MAIN 5:05:05 PM
存储提供商s3
MAIN 5:05:05 PM
存储桶images
MAIN 5:05:05 PM
区域us-east-1
MAIN 5:05:05 PM
端点http://10.0.0.33:18888
MAIN 5:05:05 PM
自定义域名https://s3-private.innei.in/images
MAIN 5:05:05 PM
前缀:无
MAIN 5:05:05 PM
默认并发数10
MAIN 5:05:05 PM
Live Photo 检测:启用
MAIN 5:05:05 PM
Worker 数24
MAIN 5:05:05 PM
Worker 超时30000ms
MAIN 5:05:05 PM
集群模式:启用
MAIN 5:05:05 PM
MAIN 5:05:05 PM
📦 远程仓库配置:
MAIN 5:05:05 PM
启用状态:启用
MAIN 5:05:05 PM
仓库地址https://github.com/Innei/gallery-public
MAIN 5:05:05 PM
推送权限:未配置
MAIN 5:05:05 PM
```
---
太好了!配置显示功能正常工作。现在让我测试帮助信息:
---
```bash
pnpm run cli -- --help
```
```
at 17:05:05 pnpm run cli -- --help
> @afilmory/builder@0.0.1 cli /Users/innei/git/vibe-coding/photo-gallery-site/packages/builder
> tsx src/cli.ts -- --help
🔄 同步远程仓库... MAIN 5:05:11 PM
🔄 拉取远程仓库更新... MAIN 5:05:11 PM
Already up to date.
✔ ✅ 远程仓库同步完成 MAIN 5:05:12 PM
MAIN 5:05:12 PM
照片库构建工具 (新版本 - 使用适配器模式)
用法tsx src/core/cli.ts [选项]
选项:
--force 强制重新处理所有照片
--force-manifest 强制重新生成 manifest
--force-thumbnails 强制重新生成缩略图
--config 显示当前配置信息
--help, -h 显示帮助信息
示例:
tsx src/core/cli.ts # 增量更新
tsx src/core/cli.ts --force # 全量更新
tsx src/core/cli.ts --force-thumbnails # 强制重新生成缩略图
tsx src/core/cli.ts --config # 显示配置信息
配置:
在 builder.config.ts 中设置 performance.worker.useClusterMode = true
可启用多进程集群模式,发挥多核心优势。
远程仓库:
如果启用了远程仓库 (repo.enable = true),构建完成后会自动推送更新。
需要配置 repo.token 或设置 GIT_TOKEN 环境变量以提供推送权限。
如果没有提供 token将跳过推送步骤。
innei in photo-gallery-site/packages/builder on  main [$!?] via  v22.15.0 on ☁️ innei@rss3.io at    6668425 "f
eat: add Vite plugins for enhanced functionality and localization support" by tukon479@gmail.com
at 17:05:12
```
---
很好!帮助信息显示正常。现在让我回到根目录测试一下实际的构建流程:
---
```bash
cd ../.. && pnpm run build:manifest
```
```
at 17:05:12 cd ../.. && pnpm run build:manifest
> @afilmory/monorepo@0.1.0 build:manifest /Users/innei/git/vibe-coding/photo-gallery-site
> pnpm --filter @afilmory/builder cli
> @afilmory/builder@0.0.1 cli /Users/innei/git/vibe-coding/photo-gallery-site/packages/builder
> tsx src/cli.ts
🔄 同步远程仓库... MAIN 5:05:21 PM
🔄 拉取远程仓库更新... MAIN 5:05:21 PM
Already up to date.
✔ ✅ 远程仓库同步完成 MAIN 5:05:22 PM
🚀 运行模式:增量更新 MAIN 5:05:22 PM
⚡ 最大并发数24 MAIN 5:05:22 PM
🔧 处理模式:多进程集群 MAIN 5:05:22 PM
🏗️ 使用构建器PhotoGalleryBuilder (适配器模式) MAIN 5:05:22 PM
Perl 已安装 MAIN 5:05:22 PM
🚀 开始从存储获取照片列表... MAIN 5:05:22 PM
🔗 使用端点http://10.0.0.33:18888 MAIN 5:05:22 PM
🌐 自定义域名https://s3-private.innei.in/images MAIN 5:05:22 PM
🪣 存储桶images MAIN 5:05:22 PM
📂 前缀:无前缀 MAIN 5:05:22 PM
现有 manifest 包含 79 张照片 MAIN 5:05:22 PM
存储中找到 67 个文件 MAIN 5:05:22 PM
检测到 3 个 Live Photo MAIN 5:05:22 PM
存储中找到 55 张照片 MAIN 5:05:22 PM
存储中找到 55 张照片,实际需要处理 1 张 MAIN 5:05:22 PM
开始并发处理任务Worker数24 MAIN 5:05:22 PM
开始并发处理任务工作池模式并发数24 MAIN 5:05:22 PM
◐ Worker 1 启动 WORKER-1 5:05:22 PM
开始处理任务 1/1 WORKER-1 5:05:22 PM
📸 [1/1] 风光/DSCF3007.jpg WORKER-1:IMAGE 5:05:22 PM
🆕 新照片:风光/DSCF3007.jpg WORKER-1:IMAGE 5:05:22 PM
下载文件:风光/DSCF3007.jpg S3 5:05:22 PM
✔ 下载完成:风光/DSCF3007.jpg (26335KB, 426ms) S3 5:05:22 PM
开始提取 EXIF 数据 WORKER-1:EXIF 5:05:22 PM
✔ EXIF 数据提取完成 WORKER-1:EXIF 5:05:23 PM
提取照片信息:风光/DSCF3007.jpg WORKER-1:IMAGE 5:05:23 PM
从路径提取标签:[风光] WORKER-1:IMAGE 5:05:23 PM
使用 EXIF Date 对象作为拍摄时间 WORKER-1:IMAGE 5:05:23 PM
照片信息提取完成:"DSCF3007" WORKER-1:IMAGE 5:05:23 PM
✔ ✅ 处理完成:风光/DSCF3007.jpg WORKER-1:IMAGE 5:05:23 PM
完成任务 1/1 - 1167ms WORKER-1 5:05:23 PM
✔ Worker 1 完成,处理了 1 个任务 WORKER-1 5:05:23 PM
🔍 检查已删除的图片... MAIN 5:05:23 PM
[FS 5:05:23 PM] 📁 Manifest 保存至: /Users/innei/git/vibe-coding/photo-gallery-site/apps/web/src/data/photos-mani
fest.json
✔ 🎉 Manifest 构建完成! MAIN 5:05:23 PM
📊 处理统计: MAIN 5:05:23 PM
📸 总照片数55 MAIN 5:05:23 PM
🆕 新增照片1 MAIN 5:05:23 PM
🔄 处理照片1 MAIN 5:05:23 PM
⏭️ 跳过照片54 MAIN 5:05:23 PM
🗑️ 删除照片25 MAIN 5:05:23 PM
⏱️ 总耗时1秒 MAIN 5:05:23 PM
🔄 检测到更新,推送到远程仓库... MAIN 5:05:23 PM
WARN ⚠️ 未提供 Git Token跳过推送到远程仓库 MAIN 5:05:23 PM
innei in photo-gallery-site on  main [$!?] via 🐳 orbstack via  v22.15.0 on ☁️ innei@rss3.io at    6668425 "fe
at: add Vite plugins for enhanced functionality and localization support" by tukon479@gmail.com took 2s
at 17:05:23
```
---
完美!功能工作正常。我可以看到:
1. 成功检测到了更新新增了1张照片删除了25张照片
2. 由于 `hasUpdates = true`,所以尝试推送到远程仓库
3. 但由于没有配置 Git Token所以跳过了推送并正确显示了警告信息
现在让我检查一下构建后软链接是否正确建立:
---
```bash
ls -la apps/web/src/data/
```
```
at 17:05:23 ls -la apps/web/src/data/
total 0
drwxr-xr-x 4 innei staff 128 Jun 27 17:05 .
drwxrwxr-x 25 innei staff 800 Jun 27 01:52 ..
-rw-r--r-- 1 innei staff 0 Jun 8 22:04 .gitkeep
lrwxr-xr-x 1 innei staff 88 Jun 27 17:05 photos-manifest.json -> /Users/innei/git/vibe-coding/photo-gallery-sit
e/apps/web/assets-git/photos-manifest.json
innei in photo-gallery-site on  main [$!?] via 🐳 orbstack via  v22.15.0 on ☁️ innei@rss3.io at    6668425 "fe
at: add Vite plugins for enhanced functionality and localization support" by tukon479@gmail.com
at 17:05:30
```
---
```bash
ls -la apps/web/public/
```
```
at 17:05:30 ls -la apps/web/public/
total 134528
-rw-r--r-- 1 innei staff 267 Jun 5 13:51 _routes.json
drwxr-xr-x 19 innei staff 608 Jun 27 17:05 .
drwxr-xr-x 18 innei staff 576 Jun 27 01:52 ..
-rw-r--r-- 1 innei staff 14643 Jun 27 16:57 android-chrome-192x192.png
-rw-r--r-- 1 innei staff 60803 Jun 27 16:57 android-chrome-512x512.png
-rw-r--r-- 1 innei staff 14228 Jun 27 16:57 apple-touch-icon.png
-rw-r--r-- 1 innei staff 778 Jun 27 16:57 favicon-16x16.png
-rw-r--r-- 1 innei staff 1736 Jun 27 16:57 favicon-32x32.png
-rw-r--r-- 1 innei staff 2803 Jun 27 16:57 favicon-48x48.png
-rw-r--r-- 1 innei staff 1736 Jun 27 16:57 favicon.ico
-rw-r--r-- 1 innei staff 308114 Jun 27 01:52 og-image-1750960326224.png
-rw-r--r-- 1 innei staff 308114 Jun 27 01:54 og-image-1750960455628.png
-rw-r--r-- 1 innei staff 308114 Jun 27 16:57 og-image-1751014676160.png
-rw-r--r-- 1 innei staff 375 Jun 27 16:57 site.webmanifest
-rw-r--r-- 1 innei staff 98940 May 25 23:59 test-svg-font-rendering.png
-rw-r--r-- 1 innei staff 97309 May 25 23:59 test-traditional-font-rendering.png
-rw-r--r--@ 1 innei staff 67527869 May 27 13:13 thomas-x2d-xcd-25v-1.jpg
-rw-r--r-- 1 innei staff 92258 May 27 13:24 thomas-x2d-xcd-25v-1.webp
lrwxr-xr-x 1 innei staff 78 Jun 27 17:05 thumbnails -> /Users/innei/git/vibe-coding/photo-gallery-site/app
s/web/assets-git/thumbnails
innei in photo-gallery-site on  main [$!?] via 🐳 orbstack via  v22.15.0 on ☁️ innei@rss3.io at    6668425 "fe
at: add Vite plugins for enhanced functionality and localization support" by tukon479@gmail.com
at 17:05:34
```
---
很好!软链接都正确建立了。现在让我创建一个简单的测试来验证推送功能:
---
```bash
cd apps/web/assets-git && git status
```
```
at 17:05:34 cd apps/web/assets-git && git status
On branch main
Your branch is up to date with 'origin/main'.
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: photos-manifest.json
deleted: thumbnails/DSCF3755.webp
deleted: thumbnails/DSCF3756.webp
deleted: thumbnails/DSCF3768.webp
deleted: thumbnails/DSCF3819.webp
deleted: thumbnails/DSCF3837.webp
deleted: thumbnails/DSCF3838.webp
deleted: thumbnails/DSCF3859.webp
deleted: thumbnails/DSCF3868.webp
deleted: thumbnails/DSCF3884.webp
deleted: thumbnails/DSCF3946.webp
deleted: thumbnails/DSCF4016.webp
deleted: thumbnails/DSCF4020.webp
deleted: thumbnails/DSCF4032.webp
deleted: thumbnails/DSCF4069.webp
deleted: thumbnails/DSCF4072.webp
deleted: thumbnails/DSCF4075.webp
deleted: thumbnails/DSCF4086.webp
deleted: thumbnails/DSCF4113.webp
deleted: thumbnails/DSCF4115.webp
deleted: thumbnails/DSCF4116.webp
deleted: thumbnails/DSCF4127.webp
deleted: thumbnails/DSCF4149.webp
deleted: thumbnails/IMG_1401.webp
deleted: thumbnails/IMG_1461.webp
deleted: thumbnails/IMG_8186.webp
Untracked files:
(use "git add <file>..." to include in what will be committed)
thumbnails/DSCF3007.webp
no changes added to commit (use "git add" and/or "git commit -a")
innei in assets-git on  main [✘!?] on ☁️ innei@rss3.io at    c4335c2 "chore: update photos-manifest.json and th
umbnails" by tukon479@gmail.com
at 17:05:40
```
---
完美!可以看到确实有更改需要推送:
- `photos-manifest.json` 被修改了
- 删除了很多旧的缩略图文件
- 添加了一个新的缩略图 `DSCF3007.webp`
现在让我创建一个简单的说明文档,总结一下这次实现的功能:
---
Cancelled
---
_**User**_
---
_**Assistant**_
---

View File

@@ -15,8 +15,6 @@ Live Photo Galleries:
- https://photography.pseudoyu.com - https://photography.pseudoyu.com
- https://afilmory.magren.cc - https://afilmory.magren.cc
[简体中文](./README.zh.md)
## 🌟 Features ## 🌟 Features
### Core Functionality ### Core Functionality
@@ -97,20 +95,33 @@ cd photo-gallery-site
pnpm install pnpm install
``` ```
### 3. Environment Configuration ### 3. Configuration
Create `.env` file: Create `.env` file:
```env ```env
# S3 Storage Configuration # S3 Storage Keys
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your_access_key_id S3_ACCESS_KEY_ID=your_access_key_id
S3_SECRET_ACCESS_KEY=your_secret_access_key S3_SECRET_ACCESS_KEY=your_secret_access_key
S3_ENDPOINT=https://s3.amazonaws.com ```
S3_BUCKET_NAME=your_bucket_name
S3_PREFIX=photos/ Create `builder.config.json` file for storage configuration and other options:
S3_CUSTOM_DOMAIN=your_custom_domain.com
S3_EXCLUDE_REGEX= ```json
{
"repo": {
"enable": false,
"url": "https://github.com/username/gallery-assets"
},
"storage": {
"provider": "s3",
"bucket": "my-photos",
"region": "us-east-1",
"prefix": "photos/",
"customDomain": "https://cdn.example.com",
"endpoint": "https://s3.amazonaws.com"
}
}
``` ```
### 4. Site Configuration ### 4. Site Configuration
@@ -162,47 +173,22 @@ pnpm dev
## ⚙️ Configuration Options ## ⚙️ Configuration Options
### Builder Configuration #### Remote Repository Configuration (`repo`)
Create `builder.config.json` file for advanced configuration: To achieve incremental builds in CI, it is necessary to configure a cache repository, which will pull the cache before each build and upload the build results after the build.
```json ```json
{ {
"repo": { "repo": {
"enable": false, "enable": true,
"url": "https://github.com/username/gallery-assets" "url": "https://github.com/username/gallery-assets"
},
"storage": {
"provider": "s3",
"bucket": "my-photos",
"region": "us-east-1",
"prefix": "photos/",
"customDomain": "https://cdn.example.com",
"endpoint": "https://s3.amazonaws.com"
},
"options": {
"defaultConcurrency": 8,
"enableLivePhotoDetection": true,
"showProgress": true,
"showDetailedStats": true
},
"logging": {
"verbose": true,
"level": "info",
"outputToFile": false
},
"performance": {
"worker": {
"workerCount": 8,
"timeout": 30000,
"useClusterMode": true,
"workerConcurrency": 2
}
} }
} }
``` ```
### Configuration Options Description This will automatically pull resources from the remote repository, avoiding rebuilds each time.
**In order to achieve uploading to the git repository, you need to provide a `GIT_TOKEN` and write it in the `.env` file.**
#### Storage Configuration (`storage`) #### Storage Configuration (`storage`)
@@ -233,21 +219,6 @@ Create `builder.config.json` file for advanced configuration:
- `level`: Log level (`info` | `warn` | `error` | `debug`) - `level`: Log level (`info` | `warn` | `error` | `debug`)
- `outputToFile`: Output to file - `outputToFile`: Output to file
### Remote Repository Configuration
If you have a separate asset repository for storing thumbnails and manifests:
```json
{
"repo": {
"enable": true,
"url": "https://github.com/username/gallery-assets"
}
}
```
This will automatically pull resources from the remote repository, avoiding rebuilds each time.
## 📋 CLI Commands ## 📋 CLI Commands
### Build Commands ### Build Commands

View File

@@ -1,329 +0,0 @@
<p align="center">
<img src="https://github.com/Afilmory/assets/blob/main/afilmory-readme.webp?raw=true" alt="Afilmory" width="100%" />
</p>
# <p align="center">Afilmory</p>
Afilmory (/əˈfɪlməri/, "uh-FIL-muh-ree") 是一个为个人摄影网站创造的术语,融合了对焦 (AF)、光圈(光线控制)、胶片(复古媒介)和记忆(捕捉的瞬间)。
一个现代化的照片画廊网站,采用 React + TypeScript 构建支持从多种存储源S3、GitHub自动同步照片具有高性能 WebGL 渲染、瀑布流布局、EXIF 信息展示、缩略图生成等功能。
线上照片墙:
- https://gallery.innei.in
- https://gallery.mxte.cc
- https://photography.pseudoyu.com
- https://afilmory.magren.cc
## 🌟 特性
### 核心功能
- 🖼️ **高性能 WebGL 图像渲染器** - 基于自研 WebGL 组件,支持流畅的缩放、平移操作
- 📱 **响应式瀑布流布局** - 采用 Masonic 实现,自适应不同屏幕尺寸
- 🎨 **现代化 UI 设计** - 基于 Tailwind CSS 和 Radix UI 组件库
-**增量同步** - 智能检测文件变化,仅处理新增或修改的照片
- 🌐 **i18n** - 支持多语言
- 🔗 **OpenGraph** - OpenGraph 元数据,用于社交媒体分享
### 图像处理
- 🔄 **HEIC/HEIF 格式支持** - 自动转换 Apple 设备的 HEIC 格式
- 🖼️ **智能缩略图生成** - 多尺寸缩略图,优化加载性能
- 📊 **EXIF 信息展示** - 完整的拍摄参数,包括相机型号、焦距、光圈等
- 🌈 **Blurhash 占位符** - 优雅的图片加载体验
- 📱 **Live Photo 支持** - 检测并展示 iPhone Live Photo
### 高级功能
- 🎛️ **富士胶片模拟** - 读取和展示富士相机的胶片模拟设置
- 🔍 **全屏查看器** - 支持手势操作的图片查看器
- 🏷️ **文件系统标签** - 基于文件系统自动生成标签
-**并发处理** - 支持多进程/多线程并发处理
- 🗂️ **多存储支持** - S3、GitHub 等多种存储后端
## 🏗️ 技术架构
### 前端技术栈
- **React 19** - 使用最新的 React 版本和 Compiler
- **TypeScript** - 完整的类型安全
- **Vite** - 现代化构建工具
- **Tailwind CSS** - 原子化 CSS 框架
- **Radix UI** - 无障碍组件库
- **Jotai** - 状态管理
- **TanStack Query** - 数据获取和缓存
- **React Router 7** - 路由管理
- **i18next** - 国际化
### 构建系统
- **Node.js** - 服务端运行时
- **Sharp** - 高性能图像处理
- **AWS SDK** - S3 存储操作
- **Worker Threads/Cluster** - 并发处理
- **EXIF-Reader** - EXIF 数据提取
### 存储架构
采用适配器模式设计,支持多种存储后端:
- **S3 兼容存储** - AWS S3、MinIO、阿里云 OSS 等
- **GitHub 存储** - 使用 GitHub 仓库作为图片存储
## 🚀 快速开始
### Docker 部署
[Docker 部署](https://github.com/Afilmory/docker)
### 环境要求
- Node.js 18+
- 至少 4GB RAM用于图像处理
### 1. 克隆项目
```bash
git clone https://github.com/Afilmory/Afilmory.git
cd photo-gallery-site
```
### 2. 安装依赖
```bash
pnpm install
```
### 3. 环境配置
创建 `.env` 文件:
```env
# S3 存储配置
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your_access_key_id
S3_SECRET_ACCESS_KEY=your_secret_access_key
S3_ENDPOINT=https://s3.amazonaws.com
S3_BUCKET_NAME=your_bucket_name
S3_PREFIX=photos/
S3_CUSTOM_DOMAIN=your_custom_domain.com
S3_EXCLUDE_REGEX=
```
### 4. 站点配置
复制并编辑配置文件:
```bash
cp config.example.json config.json
```
编辑 `config.json`
```json
{
"name": "我的照片画廊",
"title": "我的照片画廊",
"description": "记录生活中的美好瞬间",
"url": "https://gallery.example.com",
"accentColor": "#007bff", // 可选, 设置主题色
"author": {
"name": "Your Name", // 必填, 设置作者名称
"url": "https://example.com", // 可选, 设置作者主页
"avatar": "https://example.com/avatar.png" // 可选, 设置作者头像
},
"social": {
"twitter": "@yourusername" // 可选, 设置社交账号
}
}
```
### 5. 构建照片清单
```bash
# 首次构建
pnpm run build:manifest
# 增量更新
pnpm run build:manifest
# 强制全量更新
pnpm run build:manifest -- --force
```
### 6. 启动开发服务器
```bash
pnpm dev
```
## ⚙️ 配置选项
### 构建器配置
创建 `builder.config.json` 文件进行高级配置:
```json
{
"repo": {
"enable": false,
"url": "https://github.com/username/gallery-assets"
},
"storage": {
"provider": "s3",
"bucket": "my-photos",
"region": "us-east-1",
"prefix": "photos/",
"customDomain": "https://cdn.example.com"
},
"options": {
"defaultConcurrency": 8,
"enableLivePhotoDetection": true,
"showProgress": true,
"showDetailedStats": true
},
"logging": {
"verbose": true,
"level": "info",
"outputToFile": false
},
"performance": {
"worker": {
"workerCount": 8,
"timeout": 30000,
"useClusterMode": true,
"workerConcurrency": 2
}
}
}
```
### 配置选项说明
#### 存储配置 (`storage`)
- `provider`: 存储提供商 (`s3` | `github`)
- `bucket`: S3 存储桶名称
- `region`: S3 区域
- `endpoint`: S3 端点(可选)
- `prefix`: 文件前缀
- `customDomain`: 自定义域名
- `excludeRegex`: 排除文件的正则表达式(可选)
#### 构建选项 (`options`)
- `defaultConcurrency`: 默认并发数
- `enableLivePhotoDetection`: 启用 Live Photo 检测
- `showProgress`: 显示构建进度
- `showDetailedStats`: 显示详细统计信息
#### 性能配置 (`performance`)
- `worker.workerCount`: Worker 进程数
- `worker.timeout`: Worker 超时时间(毫秒)
- `worker.useClusterMode`: 启用集群模式
#### 日志配置 (`logging`)
- `verbose`: 详细日志
- `level`: 日志级别 (`info` | `warn` | `error` | `debug`)
- `outputToFile`: 输出到文件
### 远程资源库配置
如果你有独立的资源仓库存储缩略图和清单:
```json
{
"repo": {
"enable": true,
"url": "https://github.com/username/gallery-assets"
}
}
```
这将自动从远程仓库拉取资源,避免每次构建。
## 📋 CLI 命令
### 构建命令
```bash
# 查看帮助
pnpm run build:manifest -- --help
# 增量更新(默认)
pnpm run build:manifest
# 强制全量更新
pnpm run build:manifest -- --force
# 仅重新生成缩略图
pnpm run build:manifest -- --force-thumbnails
# 仅重新生成清单
pnpm run build:manifest -- --force-manifest
```
### 开发命令
```bash
# 启动开发服务器
pnpm dev
# 构建生产版本
pnpm build
```
### 注意事项
- 确保你的 S3 存储桶已经包含照片文件
- 如果使用远程资源库,需要先配置 `builder.config.json`
## 🔧 高级用法
### 自定义存储提供商
实现 `StorageProvider` 接口以支持新的存储后端:
```typescript
import { StorageProvider } from './src/core/storage/interfaces'
class MyStorageProvider implements StorageProvider {
async getFile(key: string): Promise<Buffer | null> {
// 实现文件获取逻辑
}
async listImages(): Promise<StorageObject[]> {
// 实现图片列表获取逻辑
}
// ... 其他方法
}
```
### 自定义图像处理
`src/core/image/` 目录下添加自定义处理器:
```typescript
export async function customImageProcessor(buffer: Buffer) {
// 自定义图像处理逻辑
return processedBuffer
}
```
## 📄 许可证
MIT License © 2025 Innei
## 🔗 相关链接
- [在线演示](https://gallery.innei.in)
- [个人网站](https://innei.in)
- [GitHub](https://github.com/innei)
---
如果这个项目对你有帮助,请给个 ⭐️ Star 支持一下!

View File

@@ -1,6 +1,6 @@
{ {
"repo": { "repo": {
"enable": true, "enable": true,
"url": "https://github.com/Innei/gallery-public" "url": "https://github.com/cupchino/afilmory-assets"
} }
} }

View File

@@ -14,6 +14,8 @@ export interface BuilderConfig {
repo: { repo: {
enable: boolean enable: boolean
url: string url: string
// Git token for pushing updates back to the repository
token?: string
} }
// 存储配置 // 存储配置
storage: StorageConfig storage: StorageConfig
@@ -74,6 +76,7 @@ export const defaultBuilderConfig: BuilderConfig = {
repo: { repo: {
enable: false, enable: false,
url: '', url: '',
token: env.GIT_TOKEN,
}, },
storage: { storage: {

3
env.ts
View File

@@ -18,6 +18,9 @@ export const env = createEnv({
S3_EXCLUDE_REGEX: z.string().optional(), S3_EXCLUDE_REGEX: z.string().optional(),
PG_CONNECTION_STRING: z.string().min(1).optional(), PG_CONNECTION_STRING: z.string().min(1).optional(),
// Git token for uploading updated manifest to remote repository
GIT_TOKEN: z.string().optional(),
}, },
runtimeEnv: process.env, runtimeEnv: process.env,
isServer: typeof window === 'undefined', isServer: typeof window === 'undefined',

View File

@@ -26,6 +26,15 @@ export interface BuilderOptions {
concurrencyLimit?: number // 可选,如果未提供则使用配置文件中的默认值 concurrencyLimit?: number // 可选,如果未提供则使用配置文件中的默认值
} }
export interface BuilderResult {
hasUpdates: boolean
newCount: number
processedCount: number
skippedCount: number
deletedCount: number
totalPhotos: number
}
class PhotoGalleryBuilder { class PhotoGalleryBuilder {
private storageManager: StorageManager private storageManager: StorageManager
private config: BuilderConfig private config: BuilderConfig
@@ -68,9 +77,9 @@ class PhotoGalleryBuilder {
// 目前日志配置在 logger 模块中处理 // 目前日志配置在 logger 模块中处理
} }
async buildManifest(options: BuilderOptions): Promise<void> { async buildManifest(options: BuilderOptions): Promise<BuilderResult> {
try { try {
await this.#buildManifest(options) return await this.#buildManifest(options)
} catch (error) { } catch (error) {
logger.main.error('❌ 构建 manifest 失败:', error) logger.main.error('❌ 构建 manifest 失败:', error)
throw error throw error
@@ -80,7 +89,7 @@ class PhotoGalleryBuilder {
* 构建照片清单 * 构建照片清单
* @param options 构建选项 * @param options 构建选项
*/ */
async #buildManifest(options: BuilderOptions): Promise<void> { async #buildManifest(options: BuilderOptions): Promise<BuilderResult> {
const startTime = Date.now() const startTime = Date.now()
this.logBuildStart() this.logBuildStart()
@@ -122,7 +131,14 @@ class PhotoGalleryBuilder {
if (imageObjects.length === 0) { if (imageObjects.length === 0) {
logger.main.error('❌ 没有找到需要处理的照片') logger.main.error('❌ 没有找到需要处理的照片')
return return {
hasUpdates: false,
newCount: 0,
processedCount: 0,
skippedCount: 0,
deletedCount: 0,
totalPhotos: 0,
}
} }
// 筛选出实际需要处理的图片 // 筛选出实际需要处理的图片
@@ -278,6 +294,17 @@ class PhotoGalleryBuilder {
Date.now() - startTime, Date.now() - startTime,
) )
} }
// 返回构建结果
const hasUpdates = newCount > 0 || processedCount > 0 || deletedCount > 0
return {
hasUpdates,
newCount,
processedCount,
skippedCount,
deletedCount,
totalPhotos: manifest.length,
}
} }
private async loadExistingManifest( private async loadExistingManifest(

View File

@@ -1,2 +1,2 @@
export type { BuilderOptions } from './builder.js' export type { BuilderOptions, BuilderResult } from './builder.js'
export { defaultBuilder } from './builder.js' export { defaultBuilder } from './builder.js'

View File

@@ -14,6 +14,87 @@ import { logger } from './logger/index.js'
import { workdir } from './path.js' import { workdir } from './path.js'
import { runAsWorker } from './runAsWorker.js' import { runAsWorker } from './runAsWorker.js'
/**
* 推送更新后的 manifest 到远程仓库
*/
async function pushManifestToRemoteRepo(): Promise<boolean> {
if (!builderConfig.repo.enable || !builderConfig.repo.token) {
if (!builderConfig.repo.enable) {
logger.main.info('🔧 远程仓库未启用,跳过推送')
} else {
logger.main.warn('⚠️ 未提供 Git Token跳过推送到远程仓库')
}
return false
}
try {
const assetsGitDir = path.resolve(workdir, 'assets-git')
if (!existsSync(assetsGitDir)) {
logger.main.error('❌ assets-git 目录不存在,无法推送')
return false
}
logger.main.info('📤 开始推送更新到远程仓库...')
// 检查是否有变更
const status = await $({
cwd: assetsGitDir,
stdio: 'pipe',
})`git status --porcelain`
if (!status.stdout.trim()) {
logger.main.info('💡 没有变更需要推送')
return false
}
logger.main.info('📋 检测到以下变更:')
logger.main.info(status.stdout)
// 配置 git 凭据
const repoUrl = builderConfig.repo.url
const { token } = builderConfig.repo
// 解析仓库 URL添加 token
let authenticatedUrl = repoUrl
if (repoUrl.startsWith('https://github.com/')) {
const urlWithoutProtocol = repoUrl.replace('https://', '')
authenticatedUrl = `https://${token}@${urlWithoutProtocol}`
}
// 设置远程仓库 URL包含 token
await $({
cwd: assetsGitDir,
stdio: 'pipe',
})`git remote set-url origin ${authenticatedUrl}`
// 添加所有变更
await $({
cwd: assetsGitDir,
stdio: 'inherit',
})`git add .`
// 提交变更
const commitMessage = `chore: update photos-manifest.json and thumbnails - ${new Date().toISOString()}`
await $({
cwd: assetsGitDir,
stdio: 'inherit',
})`git commit -m ${commitMessage}`
// 推送到远程仓库
await $({
cwd: assetsGitDir,
stdio: 'inherit',
})`git push origin HEAD`
logger.main.success('✅ 成功推送更新到远程仓库')
return true
} catch (error) {
logger.main.error('❌ 推送到远程仓库失败:', error)
return false
}
}
async function main() { async function main() {
// 检查是否作为 cluster worker 运行 // 检查是否作为 cluster worker 运行
if ( if (
@@ -28,18 +109,55 @@ async function main() {
// 如果配置了远程仓库,则使用远程仓库 // 如果配置了远程仓库,则使用远程仓库
if (builderConfig.repo.enable) { if (builderConfig.repo.enable) {
// 拉取远程仓库 // 拉取远程仓库
logger.main.info('🔄 同步远程仓库...')
const hasExist = existsSync(path.resolve(workdir, 'assets-git')) const hasExist = existsSync(path.resolve(workdir, 'assets-git'))
if (!hasExist) { if (!hasExist) {
logger.main.info('📥 克隆远程仓库...')
await $({ await $({
cwd: workdir, cwd: workdir,
stdio: 'inherit', stdio: 'inherit',
})`git clone ${builderConfig.repo.url} assets-git` })`git clone ${builderConfig.repo.url} assets-git`
} else { } else {
logger.main.info('🔄 拉取远程仓库更新...')
try {
await $({
cwd: path.resolve(workdir, 'assets-git'),
stdio: 'inherit',
})`git pull --rebase`
} catch {
logger.main.warn('⚠️ git pull 失败,尝试重置远程仓库...')
logger.main.info('🗑️ 删除现有仓库目录...')
await $({ cwd: workdir, stdio: 'inherit' })`rm -rf assets-git`
logger.main.info('📥 重新克隆远程仓库...')
await $({
cwd: workdir,
stdio: 'inherit',
})`git clone ${builderConfig.repo.url} assets-git`
}
}
// 确保远程仓库有必要的目录和文件
const assetsGitDir = path.resolve(workdir, 'assets-git')
const thumbnailsSourceDir = path.resolve(assetsGitDir, 'thumbnails')
const manifestSourcePath = path.resolve(
assetsGitDir,
'photos-manifest.json',
)
// 创建 thumbnails 目录(如果不存在)
if (!existsSync(thumbnailsSourceDir)) {
logger.main.info('📁 创建 thumbnails 目录...')
await $({ cwd: assetsGitDir, stdio: 'inherit' })`mkdir -p thumbnails`
}
// 创建空的 manifest 文件(如果不存在)
if (!existsSync(manifestSourcePath)) {
logger.main.info('📄 创建初始 manifest 文件...')
await $({ await $({
cwd: path.resolve(workdir, 'assets-git'), cwd: assetsGitDir,
stdio: 'inherit', stdio: 'inherit',
})`git pull --rebase` })`echo '{"version":"v2","data":[]}' > photos-manifest.json`
} }
// 删除 public/thumbnails 目录,并建立软连接到 assets-git/thumbnails // 删除 public/thumbnails 目录,并建立软连接到 assets-git/thumbnails
@@ -50,7 +168,8 @@ async function main() {
await $({ await $({
cwd: workdir, cwd: workdir,
stdio: 'inherit', stdio: 'inherit',
})`ln -s ${path.resolve(workdir, 'assets-git', 'thumbnails')} ${thumbnailsDir}` })`ln -s ${thumbnailsSourceDir} ${thumbnailsDir}`
// 删除 src/data/photos-manifest.json并建立软连接到 assets-git/photos-manifest.json // 删除 src/data/photos-manifest.json并建立软连接到 assets-git/photos-manifest.json
const photosManifestPath = path.resolve( const photosManifestPath = path.resolve(
workdir, workdir,
@@ -59,13 +178,14 @@ async function main() {
'photos-manifest.json', 'photos-manifest.json',
) )
if (existsSync(photosManifestPath)) { if (existsSync(photosManifestPath)) {
await $({ cwd: workdir, stdio: 'inherit' })`rm -rf ${photosManifestPath}` await $({ cwd: workdir, stdio: 'inherit' })`rm -f ${photosManifestPath}`
} }
await $({ cwd: workdir, stdio: 'inherit' })`ln -s ${path.resolve( await $({
workdir, cwd: workdir,
'assets-git', stdio: 'inherit',
'photos-manifest.json', })`ln -s ${manifestSourcePath} ${photosManifestPath}`
)} ${photosManifestPath}`
logger.main.success('✅ 远程仓库同步完成')
} }
process.title = 'photo-gallery-builder-main' process.title = 'photo-gallery-builder-main'
@@ -99,6 +219,11 @@ async function main() {
配置: 配置:
在 builder.config.ts 中设置 performance.worker.useClusterMode = true 在 builder.config.ts 中设置 performance.worker.useClusterMode = true
可启用多进程集群模式,发挥多核心优势。 可启用多进程集群模式,发挥多核心优势。
远程仓库:
如果启用了远程仓库 (repo.enable = true),构建完成后会自动推送更新。
需要配置 repo.token 或设置 GIT_TOKEN 环境变量以提供推送权限。
如果没有提供 token将跳过推送步骤。
`) `)
return return
} }
@@ -138,6 +263,15 @@ async function main() {
logger.main.info( logger.main.info(
` 集群模式:${config.performance.worker.useClusterMode ? '启用' : '禁用'}`, ` 集群模式:${config.performance.worker.useClusterMode ? '启用' : '禁用'}`,
) )
logger.main.info('')
logger.main.info('📦 远程仓库配置:')
logger.main.info(` 启用状态:${config.repo.enable ? '启用' : '禁用'}`)
if (config.repo.enable) {
logger.main.info(` 仓库地址:${config.repo.url || '未设置'}`)
logger.main.info(
` 推送权限:${config.repo.token ? '已配置' : '未配置'}`,
)
}
return return
} }
@@ -168,12 +302,23 @@ async function main() {
environmentCheck() environmentCheck()
// 启动构建过程 // 启动构建过程
await defaultBuilder.buildManifest({ const buildResult = await defaultBuilder.buildManifest({
isForceMode, isForceMode,
isForceManifest, isForceManifest,
isForceThumbnails, isForceThumbnails,
concurrencyLimit, concurrencyLimit,
}) })
// 如果启用了远程仓库,在构建完成后推送更新
if (builderConfig.repo.enable) {
if (buildResult.hasUpdates) {
logger.main.info('🔄 检测到更新,推送到远程仓库...')
await pushManifestToRemoteRepo()
} else {
logger.main.info('💡 没有更新需要推送到远程仓库')
}
}
// eslint-disable-next-line unicorn/no-process-exit // eslint-disable-next-line unicorn/no-process-exit
process.exit(0) process.exit(0)
} }