Body Array 대표 이미지

Body Array

이 작품은 신체를 아름답게 재현하는 장치가 아니라, 신체를 기억처럼 오인하게 만드는 오류 장치이다. 화면은 내가 보는 이 신체가 나의 현재인지, AI가 생성한 과거인지 질문을 남긴다.

작업일
2026.07.04–2026.07.26
디자인 분야
크리에이티브인터랙션·모션
용도
코드·프로토타입이미지
사용 도구
PhotoshopAfter EffectsChatGPT · 5.6 sol
작성자
강정빈

작업 배경

프로젝트 초기, 터치 디자이너를 활용하여 실시간으로 카메라 피드 인풋을 받아 화면 분할을 시도하고자 하였습니다. 다만 여러 개의 스크린과 각각 다른 기기를 활용해야하는 상황 속에서, 특히 모바일 환경에서 실시간으로 터치 디자이너를 활용하는 것이 가능하지 않았기에, 다양한 환경에서도 최적화 가능한 웹으로 구현하는 방향으로 변경하였고, 짧은 기간 안에 이를 효율적으로 구현하기 위하여 Codex Agent를 활용하였습니다.

작업 단계

TOE 테스트

ChatGPT

인터랙션 테스트

You are a senior creative-coding engineer, TouchDesigner migration specialist,
and exhibition playback systems engineer.

Your task is to port exactly three final TouchDesigner screen outputs into one
production-ready web application that can display three independent visuals on
three separate external monitors.

Do not create three unrelated repositories or three duplicated websites.
Create one maintainable web application with three independent routes:

- /screen/1
- /screen/2
- /screen/3

Also create:

- /control
- /status
- /calibrate/1
- /calibrate/2
- /calibrate/3

The intended exhibition hardware is:

1. LG Gram laptop
   - Runs the local production server.
   - Opens /screen/1 locally.
   - Connects to a TG Sambo TW4510 external monitor through HDMI.

2. iPad
   - Opens /screen/2.
   - Connects to another TG Sambo TW4510 through the appropriate HDMI adapter.
   - The iPad model may support either true external-display mode or only
     display mirroring, so the application must handle both 16:9 and 4:3
     browser viewports.

3. Samsung Galaxy S10e
   - Runs Samsung DeX.
   - Opens /screen/3.
   - Connects through a USB-C HDMI adapter or PD-enabled HDMI hub to another
     TG Sambo TW4510.

The target monitors are Full HD displays. Use a default design coordinate
system of 1920 × 1080 for landscape screens. Allow any individual screen to be
configured as 1080 × 1920 when physically mounted in portrait orientation.

IMPORTANT REALITY

A TouchDesigner .toe file cannot simply be treated as a browser application.
Do not pretend there is an automatic one-to-one converter. Inspect the
TouchDesigner network, identify the three final output graphs, trace their
dependencies, and reimplement their rendering behavior using appropriate web
technologies.

Use the following source material, when available:

- The original .toe project.
- An ASCII-expanded TouchDesigner directory generated using toeexpand.
- External media assets used by the .toe project.
- Reference screenshots and recordings for all three outputs.
- Any GLSL, Python, DAT, JSON, CSV, image, video, font, and geometry files.

FIRST: AUDIT THE SOURCE

1. Inspect the entire repository before changing files.
2. Locate all .toe files, expanded TouchDesigner files, external assets,
   shaders, scripts, and reference media.
3. Identify the three final TouchDesigner screen outputs.
4. Trace every important upstream dependency for each output.
5. Create docs/td-migration-map.md with a table containing:
   - TouchDesigner operator path
   - operator type
   - visual or timing purpose
   - dependent assets
   - web equivalent
   - migration status
   - known fidelity risks
6. Identify missing external assets and list them in
   docs/missing-assets.md.
7. Do not invent missing visual details. Use clearly labeled temporary
   placeholders only when an asset is genuinely unavailable.
8. If only a binary .toe file exists and TouchDesigner/toeexpand is not
   available in the environment, create docs/TOE_EXPORT_INSTRUCTIONS.md,
   continue building the complete display, calibration, networking, and
   routing framework, and isolate temporary screen renderers so they can be
   replaced easily when the expanded source arrives.

Write a concise implementation plan to docs/implementation-plan.md and then
continue implementing the project. Do not stop after writing the plan.

TECHNOLOGY AND PROJECT STRUCTURE

Use:

- Vite
- TypeScript
- React for routing, setup UI, control UI, and diagnostics
- Canvas 2D, SVG, CSS, or WebGL2 depending on the needs of each screen
- Three.js only when 3D rendering or shader-based rendering is genuinely useful
- A small Node.js production server
- WebSocket communication using the ws package
- npm scripts, not a complex monorepo tool

Do not make WebGPU mandatory. The application must run on:

- current Chromium on Windows
- Samsung Internet or Chrome in Samsung DeX
- Safari/WebKit on iPadOS

Use WebGL2 where GPU rendering is required, and provide a clear fallback or
error screen when a required capability is unavailable.

Suggested source structure:

src/
  app/
  screens/
    screen1/
    screen2/
    screen3/
  display/
    AspectStage.tsx
    FullscreenController.ts
    WakeLockController.ts
    CalibrationView.tsx
  sync/
    SyncClient.ts
    Clock.ts
  config/
    screens.ts
  diagnostics/
server/
  index.ts
public/
  assets/
docs/

ONE APP, THREE SCREEN ROUTES

Each screen route must load only its own renderer and required assets.

Use this configuration model:

screen 1:
- designWidth
- designHeight
- orientation
- fitMode
- rotation
- offsetX
- offsetY
- zoom
- maxDevicePixelRatio
- backgroundColor

Repeat the same configuration for screens 2 and 3.

Store the default configuration in src/config/screens.ts.

Allow runtime overrides through URL query parameters:

?fit=contain
?fit=cover
?fit=stretch
?rotate=0
?rotate=90
?rotate=180
?rotate=270
?zoom=1
?offsetX=0
?offsetY=0
?dpr=1.5
?debug=1

Persist calibration overrides in localStorage independently on each playback
device.

ASPECT-RATIO AND EXTERNAL-DISPLAY REQUIREMENTS

This part is critical.

Build a reusable AspectStage component that renders all artwork in a fixed
design coordinate system and maps it to the actual browser viewport.

Use window.innerWidth, window.innerHeight, ResizeObserver, and
window.visualViewport when available.

Never choose layout dimensions based only on the device name or user agent.
The built-in device screen ratio is not the rendering target. The actual
browser viewport on the external monitor is the rendering target.

Implement the following fit modes:

1. contain
   - scale = min(viewportWidth / designWidth,
                 viewportHeight / designHeight)
   - preserve all content
   - allow letterboxing
   - center the stage

2. cover
   - scale = max(viewportWidth / designWidth,
                 viewportHeight / designHeight)
   - fill the entire display
   - allow symmetric cropping
   - center the stage

3. stretch
   -
TOE 테스트 단계 이미지

구현을 위한 프롬프트 작성 및 웹 송출

ChatGPT

구현을 위한 프로토타입 페이지 제작

You are a senior creative-coding engineer, TouchDesigner migration specialist,
and exhibition playback systems engineer.

Your task is to port exactly three final screen outputs into one
production-ready web application that can display five independent visuals on
three separate external monitors.

Do not create five unrelated repositories or three duplicated websites.
Create one maintainable web application with three independent routes:

- /screen/1
- /screen/2
- /screen/3
- /screen/4
- /screen/5


- /control
- /status
- /calibrate/1
- /calibrate/2
- /calibrate/3

The intended exhibition hardware is:

1. LG Gram laptop
   - Runs the local production server.
   - Opens /screen/1 locally.
   - Connects to a TG Sambo TW4510 external monitor through HDMI.

2. iPad
   - Opens /screen/2.
   - Connects to another TG Sambo TW4510 through the appropriate HDMI adapter.
   - The iPad model may support either true external-display mode or only
     display mirroring, so the application must handle both 16:9 and 4:3
     browser viewports.

3. Samsung Galaxy S10e
   - Runs Samsung DeX.
   - Opens /screen/3.
   - Connects through a USB-C HDMI adapter or PD-enabled HDMI hub to another
     TG Sambo TW4510.

The target monitors are Full HD displays. Use a default design coordinate
system of 1920 × 1080 for landscape screens. Allow any individual screen to be
configured as 1080 × 1920 when physically mounted in portrait orientation.

IMPORTANT REALITY

A TouchDesigner .toe file cannot simply be treated as a browser application.
Do not pretend there is an automatic one-to-one converter. Inspect the
TouchDesigner network, identify the three final output graphs, trace their
dependencies, and reimplement their rendering behavior using appropriate web
technologies.

Use the following source material, when available:

- The original .toe project.
- An ASCII-expanded TouchDesigner directory generated using toeexpand.
- External media assets used by the .toe project.
- Reference screenshots and recordings for all three outputs.
- Any GLSL, Python, DAT, JSON, CSV, image, video, font, and geometry files.

FIRST: AUDIT THE SOURCE

1. Inspect the entire repository before changing files.
2. Locate all .toe files, expanded TouchDesigner files, external assets,
   shaders, scripts, and reference media.
3. Identify the three final TouchDesigner screen outputs.
4. Trace every important upstream dependency for each output.
5. Create docs/td-migration-map.md with a table containing:
   - TouchDesigner operator path
   - operator type
   - visual or timing purpose
   - dependent assets
   - web equivalent
   - migration status
   - known fidelity risks
6. Identify missing external assets and list them in
   docs/missing-assets.md.
7. Do not invent missing visual details. Use clearly labeled temporary
   placeholders only when an asset is genuinely unavailable.
8. If only a binary .toe file exists and TouchDesigner/toeexpand is not
   available in the environment, create docs/TOE_EXPORT_INSTRUCTIONS.md,
   continue building the complete display, calibration, networking, and
   routing framework, and isolate temporary screen renderers so they can be
   replaced easily when the expanded source arrives.

Write a concise implementation plan to docs/implementation-plan.md and then
continue implementing the project. Do not stop after writing the plan.

TECHNOLOGY AND PROJECT STRUCTURE

Use:

- Vite
- TypeScript
- React for routing, setup UI, control UI, and diagnostics
- Canvas 2D, SVG, CSS, or WebGL2 depending on the needs of each screen
- Three.js only when 3D rendering or shader-based rendering is genuinely useful
- A small Node.js production server
- WebSocket communication using the ws package
- npm scripts, not a complex monorepo tool

Do not make WebGPU mandatory. The application must run on:

- current Chromium on Windows
- Samsung Internet or Chrome in Samsung DeX
- Safari/WebKit on iPadOS

Use WebGL2 where GPU rendering is required, and provide a clear fallback or
error screen when a required capability is unavailable.

Suggested source structure:

src/
  app/
  screens/
    screen1/
    screen2/
    screen3/
  display/
    AspectStage.tsx
    FullscreenController.ts
    WakeLockController.ts
    CalibrationView.tsx
  sync/
    SyncClient.ts
    Clock.ts
  config/
    screens.ts
  diagnostics/
server/
  index.ts
public/
  assets/
docs/

ONE APP, THREE SCREEN ROUTES

Each screen route must load only its own renderer and required assets.

Use this configuration model:

screen 1:
- designWidth
- designHeight
- orientation
- fitMode
- rotation
- offsetX
- offsetY
- zoom
- maxDevicePixelRatio
- backgroundColor

Repeat the same configuration for screens 2 and 3.

Store the default configuration in src/config/screens.ts.

Allow runtime overrides through URL query parameters:

?fit=contain
?fit=cover
?fit=stretch
?rotate=0
?rotate=90
?rotate=180
?rotate=270
?zoom=1
?offsetX=0
?offsetY=0
?dpr=1.5
?debug=1

Persist calibration overrides in localStorage independently on each playback
device.

ASPECT-RATIO AND EXTERNAL-DISPLAY REQUIREMENTS

This part is critical.

Build a reusable AspectStage component that renders all artwork in a fixed
design coordinate system and maps it to the actual browser viewport.

Use window.innerWidth, window.innerHeight, ResizeObserver, and
window.visualViewport when available.

Never choose layout dimensions based only on the device name or user agent.
The built-in device screen ratio is not the rendering target. The actual
browser viewport on the external monitor is the rendering target.

Implement the following fit modes:

1. contain
   - scale = min(viewportWidth / designWidth,
                 viewportHeight / designHeight)
   - preserve all content
   - allow letterboxing
   - center the stage

2. cover
   - scale = max(viewportWidth / designWidth,
                 viewportHeight / designHeight)
   - fill the entire display
   - allow symmetric cropping
   - center the stage

3. stretch
   - scale
구현을 위한 프롬프트 작성 및 웹 송출 단계 이미지

회고

성과와 한계

우선 주어진 기간 내에 학습된 AI 에이전트와 함께 일하며 작업의 호율성은 올라갔습니다. 하지만 여전히 원하는 것을 구현하기 위해선 세부적인 프롬프트 수정이 필수적입니다. 웹 기반+실시간 송출로 인해 표현하고자 했던 시각 표현이 아쉬웠습니다.

시행착오와 개선점

다양한 기기에 광범위하게 사용될 수 있는 하나의 웹 제작을 하며 각 기기마다의 화면비에 맞게 분할되는 인터랙션을 제작하는 것과 구형 기기들의 최적화 문제를 마주하면서, 각 기기 정보와 화면비율을 리서치하며 이에 맞게 최적화할 수 있도록 에이전트와 대화하며 작업을 진행했습니다.

적합한 활용 맥락

인터랙티브 미디어, 웹 기반 아트 측면에서는 AI 에이전트를 활용하는 것이 매우 호율적입니다. 특히 스케치하듯이 큰 틀을 먼저 작성해두고, 이후 시각, 인터랙션, 등의 디테일을 수정해가면 AI의 오류를 줄일 수 있습니다.

댓글

0

작성자 정보를 확인하고 있습니다.