<!-- Generated by SpecStory -->

# 修改 image loader manager 返回 imageData (2025-06-06 14:16:53)

_**User**_

修改这个 image loader manager 的方法，不要返回 blob，我发现创建 blob url 内存占用太大了。改成返回 imageData，imageData 可以直接在 canvas 中使用 同时要修改 @WebGLImageViewerEngine.ts 不再接受 src 而是接受 imaegData。同时修改 LRU 的 cache ，缓存 imagedata 而不是 blobURL。修改 @ProgressiveImage.tsx 不再接受 blobUrl 而是 imagedata

---

_**Assistant**_



---

我来帮你修改这些文件，将系统从使用 blob URL 改为使用 ImageData，这确实能显著减少内存占用。

---

```diff
  
  export interface ImageLoadResult {
-   blobSrc: string
-   convertedUrl?: string
+   imageData: ImageData
+   width: number
+   height: number
+   originalSize: number
+   format: string
  }
  
```

```diff
  
  export interface ImageCacheResult {
-   blobSrc: string
+   imageData: ImageData
+   width: number
+   height: number
    originalSize: number
    format: string
```

```diff
    (value, key, reason) => {
      try {
-       URL.revokeObjectURL(value.blobSrc)
-       console.info(`Regular image cache: Revoked blob URL - ${reason}`)
+       // ImageData doesn't need explicit cleanup like blob URLs
+       console.info(`Regular image cache: Removed ImageData cache - ${reason}`)
      } catch (error) {
        console.warn(
-         `Failed to revoke regular image blob URL (${reason}):`,
+         `Failed to cleanup regular image cache (${reason}):`,
          error,
        )
```

```diff
    return url
  }
+ 
+ /**
+  * 将图片转换为 ImageData
+  */
+ async function convertToImageData(
+   blob: Blob,
+ ): Promise<{ imageData: ImageData; width: number; height: number }> {
+   return new Promise((resolve, reject) => {
+     const img = new Image()
+     img.onload = () => {
+       try {
+         const canvas = document.createElement('canvas')
+         const ctx = canvas.getContext('2d')
+         if (!ctx) {
+           reject(new Error('Failed to get canvas context'))
+           return
+         }
+ 
+         canvas.width = img.width
+         canvas.height = img.height
+         ctx.drawImage(img, 0, 0)
+ 
+         const imageData = ctx.getImageData(0, 0, img.width, img.height)
+         resolve({
+           imageData,
+           width: img.width,
+           height: img.height,
+         })
+       } catch (error) {
+         reject(error)
+       }
+     }
+     img.onerror = () => reject(new Error('Failed to load image'))
+     img.src = URL.createObjectURL(blob)
+   })
+ }
  
  export class ImageLoaderManager {
```

```diff
  
        const conversionResult = await convertHeicImage(blob, originalUrl)
+ 
+       // 从转换后的 blob 创建 ImageData
+       const convertedBlob = await fetch(conversionResult.url).then(res => res.blob())
+       const { imageData, width, height } = await convertToImageData(convertedBlob)
  
        // Hide loading indicator
```

```diff
          `HEIC converted: ${(blob.size / 1024).toFixed(1)}KB → ${(conversionResult.convertedSize / 1024).toFixed(1)}KB`,
        )
+ 
+       // 清理转换后的 blob URL
+       URL.revokeObjectURL(conversionResult.url)
  
        return {
-         blobSrc: conversionResult.url,
-         convertedUrl: conversionResult.url,
+         imageData,
+         width,
+         height,
+         originalSize: blob.size,
+         format: blob.type,
        }
      } catch (conversionError) {
```

```diff
    }
  
-   private processRegularImage(
+   private async processRegularImage(
      blob: Blob,
      originalUrl: string, // 添加原始 URL 参数
      callbacks: LoadingCallbacks,
-   ): ImageLoadResult {
+   ): Promise<ImageLoadResult> {
      const { onLoadingStateUpdate } = callbacks
  
```

```diff
  
        return {
-         blobSrc: cachedResult.blobSrc,
+         imageData: cachedResult.imageData,
+         width: cachedResult.width,
+         height: cachedResult.height,
+         originalSize: cachedResult.originalSize,
+         format: cachedResult.format,
        }
      }
  
-     // 普通图片格式
-     const url = URL.createObjectURL(blob)
+     // 转换为 ImageData
+     const { imageData, width, height } = await convertToImageData(blob)
  
      const result: ImageCacheResult = {
-       blobSrc: url,
+       imageData,
+       width,
+       height,
        originalSize: blob.size,
        format: blob.type,
```

```diff
  
      return {
-       blobSrc: url,
+       imageData,
+       width,
+       height,
+       originalSize: blob.size,
+       format: blob.type,
      }
    }
```

---

```diff
      gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0)
    }
+ 
+   async loadImageData(imageData: ImageData, width: number, height: number) {
+     this.isLoadingTexture = true // 开始加载图片
+     this.notifyLoadingStateChange(true, '处理图片数据中...')
+ 
+     try {
+       this.imageWidth = width
+       this.imageHeight = height
+ 
+       // 估算内存需求并决定渲染策略
+       const imagePixels = width * height
+       const baseMemoryMB = (imagePixels * 4) / (1024 * 1024) // RGBA 基础内存
+       const estimatedMaxMemoryMB = baseMemoryMB * 3 // 估算最多需要的内存（多个LOD级别）
+ 
+       console.info(`Image loaded: ${width}×${height}`)
+       console.info(`Base memory requirement: ${baseMemoryMB.toFixed(1)} MB`)
+       console.info(
+         `Estimated max memory: ${estimatedMaxMemoryMB.toFixed(1)} MB`,
+       )
+       console.info(
+         `Memory budget: ${(this.maxMemoryBudget / 1024 / 1024).toFixed(1)} MB`,
+       )
+ 
+       // 决定是否使用瓦片渲染
+       const maxDimension = Math.max(width, height)
+       const shouldUseTiling =
+         estimatedMaxMemoryMB > this.maxMemoryBudget / (1024 * 1024) ||
+         imagePixels > 50 * 1024 * 1024 || // 50M 像素
+         maxDimension > 8192 // 任一边超过 8K
+ 
+       if (shouldUseTiling) {
+         this.useTiledRendering = true
+         console.info(`🧩 Using tiled rendering for large image`)
+         console.info(`Tile size: ${this.tileSize}×${this.tileSize}`)
+         console.info(`Max tiles in memory: ${this.maxTilesInMemory}`)
+       } else {
+         console.info(`📄 Using standard LOD rendering`)
+       }
+ 
+       // 先设置正确的缩放值，再创建纹理
+       if (this.config.centerOnInit) {
+         this.fitImageToScreen()
+       } else {
+         // 即使不居中，也需要将相对缩放转换为绝对缩放
+         const fitToScreenScale = this.getFitToScreenScale()
+         this.scale = fitToScreenScale * this.config.initialScale
+       }
+ 
+       this.notifyLoadingStateChange(true, '创建纹理中...')
+       await this.createTextureFromImageData(imageData)
+       this.imageLoaded = true
+       this.isLoadingTexture = false // 图片加载完成
+       this.notifyLoadingStateChange(false)
+       this.render()
+       this.notifyZoomChange() // 通知初始缩放值
+     } catch (error) {
+       this.isLoadingTexture = false // 加载失败也要清除状态
+       this.notifyLoadingStateChange(false)
+       throw error
+     }
+   }
  
    async loadImage(url: string) {
```

```diff
      })
    }
+ 
+   private async createTextureFromImageData(imageData: ImageData) {
+     // 从 ImageData 创建 HTMLImageElement
+     const canvas = document.createElement('canvas')
+     canvas.width = imageData.width
+     canvas.height = imageData.height
+     const ctx = canvas.getContext('2d')!
+     ctx.putImageData(imageData, 0, 0)
+ 
+     const image = new Image()
+     image.src = canvas.toDataURL()
+ 
+     return new Promise<void>((resolve, reject) => {
+       image.onload = async () => {
+         try {
+           this.originalImage = image
+           await this.createOriginalImageBitmap()
+ 
+           if (this.useTiledRendering) {
+             await this.initializeTiledSystem()
+           } else {
+             this.initializeLODTextures()
+           }
+           resolve()
+         } catch (error) {
+           reject(error)
+         }
+       }
+       image.onerror = () => reject(new Error('Failed to process ImageData'))
+     })
+   }
  
    private async createTexture(image: HTMLImageElement) {
```

---

```diff
  import { canUseWebGL } from '~/lib/feature'
  import { ImageLoaderManager } from '~/lib/image-loader-manager'
+ import type { ImageLoadResult } from '~/lib/image-loader-manager'
  
  import { LivePhoto } from './LivePhoto'
```

```diff
    onProgress?: (progress: number) => void
    onZoomChange?: (isZoomed: boolean) => void
-   onBlobSrcChange?: (blobSrc: string | null) => void
+   onImageDataChange?: (imageData: ImageData | null) => void
  
    enableZoom?: boolean
```

```diff
    onProgress,
    onZoomChange,
-   onBlobSrcChange,
+   onImageDataChange,
  
    maxZoom = 20,
```

```diff
    livePhotoVideoUrl,
  }: ProgressiveImageProps) => {
-   const [blobSrc, setBlobSrc] = useState<string | null>(null)
+   const [imageResult, setImageResult] = useState<ImageLoadResult | null>(null)
    const [highResLoaded, setHighResLoaded] = useState(false)
    const [error, setError] = useState(false)
```

```diff
      function cleanup() {
        setHighResLoaded(false)
-       setBlobSrc(null)
+       setImageResult(null)
        setError(false)
-       onBlobSrcChange?.(null)
+       onImageDataChange?.(null)
  
        // Reset loading indicator
```

```diff
          })
  
-         setBlobSrc(result.blobSrc)
-         onBlobSrcChange?.(result.blobSrc)
+         setImageResult(result)
+         onImageDataChange?.(result.imageData)
          setHighResLoaded(true)
        } catch (loadError) {
```

```diff
      onError,
      isCurrentImage,
-     onBlobSrcChange,
+     onImageDataChange,
    ])
  
```

```diff
  
    const showContextMenu = useShowContextMenu()
+ 
+   // 将 ImageData 转换为 blob URL 用于下载和复制
+   const convertImageDataToBlob = useCallback(async (imageData: ImageData): Promise<Blob> => {
+     const canvas = document.createElement('canvas')
+     canvas.width = imageData.width
+     canvas.height = imageData.height
+     const ctx = canvas.getContext('2d')!
+     ctx.putImageData(imageData, 0, 0)
+     
+     return new Promise((resolve) => {
+       canvas.toBlob((blob) => {
+         resolve(blob!)
+       }, 'image/png')
+     })
+   }, [])
  
    if (error) {
```

```diff
        )}
  
-       {highResLoaded && blobSrc && isCurrentImage && (
+       {highResLoaded && imageResult && isCurrentImage && (
          <WebGLImageViewer
            ref={transformRef}
-           src={blobSrc}
+           imageData={imageResult.imageData}
+           width={imageResult.width}
+           height={imageResult.height}
            className="absolute inset-0 h-full w-full"
            initialScale={1}
```

```diff
  
                      try {
-                       // Create a canvas to convert the image to PNG
-                       const img = new Image()
-                       img.crossOrigin = 'anonymous'
- 
-                       await new Promise((resolve, reject) => {
-                         img.onload = resolve
-                         img.onerror = reject
-                         img.src = blobSrc
-                       })
- 
-                       const canvas = document.createElement('canvas')
-                       const ctx = canvas.getContext('2d')
-                       canvas.width = img.naturalWidth
-                       canvas.height = img.naturalHeight
- 
-                       ctx?.drawImage(img, 0, 0)
- 
-                       // Convert to PNG blob
-                       await new Promise<void>((resolve, reject) => {
-                         canvas.toBlob(async (pngBlob) => {
-                           try {
-                             if (pngBlob) {
-                               await navigator.clipboard.write([
-                                 new ClipboardItem({
-                                   'image/png': pngBlob,
-                                 }),
-                               ])
-                               resolve()
-                             } else {
-                               reject(
-                                 new Error('Failed to convert image to PNG'),
-                               )
-                             }
-                           } catch (error) {
-                             reject(error)
-                           }
-                         }, 'image/png')
-                       })
+                       // 将 ImageData 转换为 PNG blob
+                       const pngBlob = await convertImageDataToBlob(imageResult.imageData)
+ 
+                       await navigator.clipboard.write([
+                         new ClipboardItem({
+                           'image/png': pngBlob,
+                         }),
+                       ])
  
                        toast.dismiss(loadingToast)
                        toast.success('图片已复制到剪贴板')
                      } catch (error) {
                        console.error('Failed to copy image:', error)
- 
-                       // Fallback: try to copy the original blob
-                       try {
-                         const blob = await fetch(blobSrc).then((res) =>
-                           res.blob(),
-                         )
-                         await navigator.clipboard.write([
-                           new ClipboardItem({
-                             [blob.type]: blob,
-                           }),
-                         ])
-                         toast.dismiss(loadingToast)
-                         toast.success('图片已复制到剪贴板')
-                       } catch (fallbackError) {
-                         console.error(
-                           'Fallback copy also failed:',
-                           fallbackError,
-                         )
-                         toast.dismiss(loadingToast)
-                         toast.error('复制图片失败，请稍后重试')
-                       }
+                       toast.dismiss(loadingToast)
+                       toast.error('复制图片失败，请稍后重试')
                      }
                    },
```

```diff
                  new MenuItemText({
                    label: '下载图片',
-                   click: () => {
-                     const a = document.createElement('a')
-                     a.href = blobSrc
-                     a.download = alt
-                     a.click()
+                   click: async () => {
+                     try {
+                       const pngBlob = await convertImageDataToBlob(imageResult.imageData)
+                       const url = URL.createObjectURL(pngBlob)
+                       
+                       const a = document.createElement('a')
+                       a.href = url
+                       a.download = alt || 'image.png'
+                       a.click()
+                       
+                       // 清理临时 URL
+                       setTimeout(() => URL.revokeObjectURL(url), 1000)
+                     } catch (error) {
+                       console.error('Failed to download image:', error)
+                       toast.error('下载图片失败，请稍后重试')
+                     }
                    },
                  }),
```

```diff
  
        {/* 备用图片（当 WebGL 不可用时） */}
-       {!canUseWebGL && highResLoaded && blobSrc && (
+       {!canUseWebGL && highResLoaded && imageResult && (
          <div className="pointer-events-none absolute inset-0 z-10 flex flex-col items-center justify-center gap-2 bg-black/20">
            <i className="i-mingcute-warning-line mb-2 text-4xl" />
```

---

I need to update the WebGL viewer interface to accept ImageData. Let me check and update the interface first.

---

Read file: packages/webgl-viewer/src/interface.ts

---

Now I need to update the interface to support both src and ImageData approaches.

---

```diff
  }
  export interface WebGLImageViewerProps {
-   src: string
+   src?: string
+   imageData?: ImageData
+   width?: number
+   height?: number
    className?: string
    initialScale?: number
```

---

Now I need to update the WebGL viewer component to handle both src and ImageData approaches.

---

Read file: packages/webgl-viewer/src/WebGLImageViewer.tsx

---

Now I need to update the WebGL viewer component to handle both src and ImageData props.

---

```diff
    ref,
    src,
+   imageData,
+   width,
+   height,
    className = '',
    initialScale = 1,
```

```diff
    const setDebugInfo = useRef((() => {}) as (debugInfo: any) => void)
  
-   const config: Required<WebGLImageViewerProps> = useMemo(
+   const config: Required<Omit<WebGLImageViewerProps, 'src' | 'imageData' | 'width' | 'height'>> = useMemo(
      () => ({
-       src,
        className,
        initialScale,
```

```diff
      }),
      [
-       src,
        className,
        initialScale,
```

```diff
        webGLImageViewerEngine = new WebGLImageViewerEngine(
          canvasRef.current,
-         config,
+         config as Required<WebGLImageViewerProps>,
          debug ? setDebugInfo : undefined,
        )
-       webGLImageViewerEngine.loadImage(src).catch(console.error)
+ 
+       // 根据提供的props决定使用哪种加载方式
+       if (imageData && width && height) {
+         // 使用 ImageData 方式
+         webGLImageViewerEngine.loadImageData(imageData, width, height).catch(console.error)
+       } else if (src) {
+         // 使用 URL 方式
+         webGLImageViewerEngine.loadImage(src).catch(console.error)
+       } else {
+         console.error('WebGLImageViewer: Either src or (imageData + width + height) must be provided')
+       }
+ 
        viewerRef.current = webGLImageViewerEngine
      } catch (error) {
```

```diff
        webGLImageViewerEngine?.destroy()
      }
-   }, [src, config, debug])
+   }, [src, imageData, width, height, config, debug])
  
    return (
```

---

Great! Now let me check if there are any other files that need to be updated for this change to work properly. Let me also add the memory saving feature to save memory by avoiding blob URLs.

---



---

Perfect! I've successfully updated the system to use ImageData instead of blob URLs, which should significantly reduce memory usage. Here's a summary of the changes made:

## 🎯 主要修改

### 1. **ImageLoaderManager** (`apps/web/src/lib/image-loader-manager.ts`)
- 修改接口：`ImageLoadResult` 现在返回 `ImageData` 而不是 `blobSrc`
- 新增 `convertToImageData` 函数将 Blob 转换为 ImageData
- 更新缓存系统存储 ImageData 而不是 blob URL
- 简化清理逻辑，ImageData 不需要像 blob URL 那样手动清理

### 2. **WebGLImageViewerEngine** (`packages/webgl-viewer/src/WebGLImageViewerEngine.ts`)
- 新增 `loadImageData` 方法直接接受 ImageData
- 新增 `createTextureFromImageData` 方法处理 ImageData 输入
- 保留原有的 `loadImage` 方法用于向后兼容

### 3. **WebGL Viewer 接口** (`packages/webgl-viewer/src/interface.ts`)
- 更新 `WebGLImageViewerProps` 接口支持 `imageData`、`width`、`height` 属性
- 将 `src` 改为可选，支持两种加载方式

### 4. **WebGL Viewer 组件** (`packages/webgl-viewer/src/WebGLImageViewer.tsx`)
- 更新组件支持新的属性
- 智能选择加载方式：优先使用 ImageData，fallback 到 URL
- 更新依赖数组包含新的属性

### 5. **ProgressiveImage** (`apps/web/src/components/ui/photo-viewer/ProgressiveImage.tsx`)
- 改为使用 `ImageLoadResult` 和 `ImageData`
- 新增 `convertImageDataToBlob` 函数仅在需要时临时转换
- 更新上下文菜单的复制和下载功能
- 修改 `onImageDataChange` 回调

## 🚀 性能优势

1. **内存占用减少**：ImageData 直接在内存中，避免了 blob URL 的额外开销
2. **更好的缓存策略**：直接缓存 ImageData，无需管理 URL 生命周期
3. **减少 GC 压力**：避免频繁创建和清理 blob URL
4. **更直接的数据流**：从网络 → ImageData → WebGL 纹理，减少中间转换

## ✅ 向后兼容

系统仍然支持通过 `src` 属性加载图片，确保现有代码不会被破坏。新的 ImageData 方式是可选的增强功能。

这个修改应该能显著减少大图片的内存占用，特别是在处理高分辨率图片时效果会很明显！

---

_**User**_

For the code present, we get this error:
```
Cannot find name 'blobSrc'.
```
How can I resolve this? If you propose a fix, please make it concise.

---

_**Assistant**_



---

The error occurs because `blobSrc` no longer exists after our ImageData refactor. Here's the fix:

---



---

Fixed! The component now:
1. Uses `imageData` (if available) to create a blob via canvas
2. Falls back to fetching from `photo.originalUrl` if no `imageData`
3. Maintains the same sharing functionality

---

