mirror of
https://github.com/oso95/scroll-world.git
synced 2026-09-19 01:08:37 +08:00
v0.3.0: mobile is a native 9:16 portrait chain
The mobile version is now its own camera chain rendered natively in portrait (portrait start canvases -> 9:16 dives + connectors frame-locked against their own renders -> 720-wide encodes -> stillMobile posters), not a centre-crop of the landscape film. The 16:9 crop encodes remain only as an explicitly-approved stopgap. Engine: stillMobile poster support; mobile QA checklist verifies the served clips are natively portrait. Interview: mobile is an opt-in question with the ~2x credit cost stated. Also: ignore .gstack/ session dir.
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "scroll-world",
|
||||
"description": "Build an immersive scroll-scrubbed 'fly through the world' landing page for any industry or brand using Higgsfield. As the visitor scrolls, a pre-rendered camera flies from outside each scene into its interior, then flows on to the next scene with no cuts — one continuous connected flight. Interviews the user for the topic, story beats, and brand kit, then generates cohesive scenes + seamless camera clips and wires a portable, framework-agnostic scroll-scrub engine. Use for a '3D world' / 'browse-through-the-industry' hero, a scroll cinematic, or a diorama landing.",
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"author": {
|
||||
"name": "cyw",
|
||||
"email": "cyw@cywang.me"
|
||||
|
||||
@@ -1,2 +1,3 @@
|
||||
.DS_Store
|
||||
node_modules/
|
||||
.gstack/
|
||||
|
||||
@@ -91,15 +91,21 @@ default. Cover:
|
||||
section is usually the hero product + the CTA.
|
||||
5. **Mobile version (beta) — ALWAYS ask this; never silently generate both.** Ask as a
|
||||
two-option choice (`AskUserQuestion` in Claude Code; a plain question elsewhere):
|
||||
*"Want a mobile-optimized version too? Mobile support is in
|
||||
**beta** — the scroll-scrub mechanic is desktop-native; on phones you get lighter
|
||||
encodes and engine hardening, but portrait crops the 16:9 frame and low-end devices
|
||||
may still stutter."* Options: "Desktop only" / "Desktop + mobile (beta)". The beta
|
||||
disclaimer must be stated to the user, not just implied. What the answer gates:
|
||||
- **Yes** → produce the `-m.mp4` mobile encodes (Step 6) and wire
|
||||
`clipMobile`/`connectorsMobile` (Step 7); run the full mobile QA (Step 8). If any
|
||||
scene's focal subject sits off-centre, offer the 9:16 hero-variant escape hatch
|
||||
(extra Higgsfield credits — say so).
|
||||
*"Want a mobile-optimized version too? Mobile support is in **beta** — the scroll-scrub
|
||||
mechanic is desktop-native. The mobile version is a second camera chain rendered
|
||||
natively in **9:16 portrait** — composed for phones, not a crop of the landscape film —
|
||||
which roughly doubles the Higgsfield credit spend (state the estimated number)."*
|
||||
Options: "Desktop only" / "Desktop + mobile (beta, native 9:16 — ~2× credits)". The
|
||||
beta disclaimer and the credit cost must be stated to the user, not just implied.
|
||||
What the answer gates:
|
||||
- **Yes** → render the parallel 9:16 portrait chain and ship it as the mobile variants
|
||||
(Step 6 / pipeline.md §6b): portrait start canvases → 9:16 dives + connectors
|
||||
frame-locked against their own renders → 720-wide `-m.mp4` encodes → `stillMobile`
|
||||
portrait posters. Wire `clipMobile`/`connectorsMobile`/`stillMobile` (Step 7); run
|
||||
the full mobile QA (Step 8). Budget ~2N-1 extra video gens + NSFW re-rolls.
|
||||
**Never ship the centre-crop as the mobile version by default** — if credits can't
|
||||
cover the portrait chain, say so and offer the crop encodes (pipeline.md §6) as an
|
||||
explicitly-labelled stopgap the user must approve.
|
||||
- **No** → skip the mobile encodes and wiring entirely. The engine's phone hardening
|
||||
(seek-coalescing, iOS priming, safe-area CSS) is always on regardless — that's not
|
||||
a "mobile version," it's just the page not breaking when a phone visits — so a
|
||||
@@ -364,15 +370,17 @@ ffmpeg -i src.mp4 -an -vf "unsharp=5:5:0.8:5:5:0.0" \
|
||||
|
||||
Encode all 2N-1 clips (dives + connectors) with the same settings for uniform quality.
|
||||
|
||||
**Mobile encodes (beta — only if the user opted in at Step 1.5).** Phone video decoders seek
|
||||
far slower than a laptop's, and seek cost scales with GOP length, so the 1080p `-g 8` master
|
||||
that scrubs smoothly on desktop can stutter on a phone. Produce a lighter `-m.mp4` sibling for
|
||||
every clip — **720p, `-g 4`** (more keyframes = cheaper seeks), crf 23 — and wire them as
|
||||
`clipMobile` / `connectorsMobile` (Step 7). The engine serves them automatically on phones and
|
||||
falls back to the desktop clip when absent. The exact `encm()` script is in
|
||||
`references/pipeline.md` §6. If the user chose desktop-only, skip this — the engine still
|
||||
hardens phone scrubbing regardless (seek-coalescing, iOS priming), so the page degrades
|
||||
gracefully rather than breaking.
|
||||
**Mobile encodes (beta — only if the user opted in at Step 1.5).** The mobile version is
|
||||
the **native 9:16 portrait chain** (pipeline.md §6b): portrait renders of every dive and
|
||||
connector, encoded **720 wide (`scale=720:-2`), `-g 4`** (more keyframes = cheaper seeks —
|
||||
phone decoders' seek cost scales with GOP length), crf 23 — wired as `clipMobile` /
|
||||
`connectorsMobile`, with each portrait dive's first frame extracted as the section's
|
||||
`stillMobile` poster (Step 7). The engine serves them automatically on phones and falls
|
||||
back to the desktop clip when absent. The 16:9 centre-crop `encm()` encodes
|
||||
(pipeline.md §6) are a **fallback only** — for when credits can't cover the portrait
|
||||
chain — and shipping them must be called out to the user, never silent. If the user chose
|
||||
desktop-only, skip this — the engine still hardens phone scrubbing regardless
|
||||
(seek-coalescing, iOS priming), so the page degrades gracefully rather than breaking.
|
||||
|
||||
---
|
||||
|
||||
@@ -388,7 +396,9 @@ mountScrollWorld(document.getElementById('world'), {
|
||||
diveScroll: 1.3, connScroll: 0.9, // viewport-heights of scroll per clip
|
||||
sections: [
|
||||
{ id:'farm', label:'The Farms', still:'assets/farm.webp',
|
||||
clip:'assets/vid/farm.mp4', clipMobile:'assets/vid/farm-m.mp4', // mobile beta only
|
||||
clip:'assets/vid/farm.mp4',
|
||||
clipMobile:'assets/vid/farm-m.mp4', // mobile beta only: native 9:16 render
|
||||
stillMobile:'assets/farm-m.webp', // its first frame as the portrait poster
|
||||
scroll: 1.6, linger: 0.45, // optional pacing: longer dwell + camera settles mid-scene
|
||||
accent:'#8FB98A', eyebrow:'From leaf to last sip', title:'It starts in the hills.',
|
||||
body:'…', tags:['Single-origin','Hand-picked'] },
|
||||
@@ -446,12 +456,15 @@ is the thing most likely to be wrong:
|
||||
over the instant you scroll — no blank/black scene (the iOS priming fix). Test iOS Safari
|
||||
specifically; it's the one that goes blank if this regresses.
|
||||
- Verify the `-m.mp4` variant is actually served on mobile (Network panel), and the
|
||||
heavy 1080p master on desktop.
|
||||
heavy 1080p master on desktop. The mobile clips must be **natively portrait**
|
||||
(`videoWidth < videoHeight` — not a downscaled 16:9 file), and the `stillMobile`
|
||||
posters must be served and match each portrait clip's first frame (no
|
||||
landscape→portrait flash when the video paints).
|
||||
- Slowly scroll so the URL bar collapses — the page must **not jump** (height-only resizes
|
||||
are ignored on touch). Rotate the device — layout should recompose cleanly.
|
||||
- Portrait crops a 16:9 clip to its centre; confirm the focal subject still reads. If a
|
||||
hero scene's subject sits off-centre and gets cut, recompose it (prompts.md) or generate
|
||||
a 9:16 variant for that scene.
|
||||
- Only if the crop **fallback** shipped (no credits for the portrait chain): portrait
|
||||
crops a 16:9 clip to its centre — confirm the focal subject still reads, and remind
|
||||
the user this is the stopgap, not the mobile version.
|
||||
- Check reduced-motion (should fall back to the stills, no video, no particles).
|
||||
|
||||
---
|
||||
@@ -506,10 +519,11 @@ is the thing most likely to be wrong:
|
||||
- **Copy hidden behind the URL bar / notch on mobile** → use the engine's safe-area-aware
|
||||
bottom offset (`env(safe-area-inset-bottom)` + `dvh`); make sure the page's
|
||||
`<meta viewport>` includes `viewport-fit=cover` (the template does).
|
||||
- **Portrait crops the scene** → a 16:9 clip on a tall phone shows only its centre. Keep each
|
||||
scene's focal subject centred with a little headroom (prompts.md), or generate a 9:16 hero
|
||||
for the scenes that matter most. The engine centre-crops (`object-fit:cover`); it can't
|
||||
un-crop a widescreen composition.
|
||||
- **Portrait crops the scene** → a 16:9 clip on a tall phone shows only its centre — which
|
||||
is why the mobile version is the native 9:16 chain (§6b), never the crop. If you're seeing
|
||||
this on a mobile build, either the crop fallback shipped (call it out to the user) or the
|
||||
9:16 encodes aren't actually being served (check `videoWidth < videoHeight`). Keeping each
|
||||
scene's focal subject centred (prompts.md) still matters for the desktop film itself.
|
||||
- **`--generate-audio` errors on seedance** → omit it; mute in HTML and `-an` on encode.
|
||||
- **Kling rejects your flags** → `kling3_0` has **no `--resolution` param** (don't pass
|
||||
one; encode at whatever native res ffprobe reports) and **sound defaults on** — pass
|
||||
@@ -521,6 +535,10 @@ is the thing most likely to be wrong:
|
||||
- **White-box scenes** → `gpt_image_2` returns a solid bg; either match the page bg to it
|
||||
or knock it out (Step 3).
|
||||
- **bash 3.2** on macOS → no associative arrays in scripts.
|
||||
- **Connector grabs the wrong scene's frames** (or errors on a frame that doesn't exist
|
||||
yet) → the array loop ran in **zsh** (macOS default interactive shell), where arrays are
|
||||
1-indexed, not bash's 0-indexed. Keep every array-driven chain step in a `#!/bin/bash`
|
||||
script run via `bash script.sh` — never inline array loops in the interactive shell.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -119,13 +119,17 @@ i=0; for f in "$WORK"/conn_*.mp4; do i=$((i+1)); enc "$f" "$ASSETS/vid/conn$i.mp
|
||||
Now the engine config's `sections[k].clip = assets/vid/<name>.mp4` and
|
||||
`connectors = [assets/vid/conn1.mp4, …]` (length N-1, in order).
|
||||
|
||||
## 6. Mobile encodes (Step 6) — mobile beta, only if the user opted in
|
||||
## 6. Centre-crop mobile encodes — FALLBACK ONLY, not the mobile version
|
||||
|
||||
**Skip this section unless the user chose the mobile (beta) version in the Step 1
|
||||
interview.** Scrubbing sets `currentTime` every frame, and a phone decoder's **seek cost scales with
|
||||
how many frames it must decode from the nearest keyframe** — so a 1080p `-g 8` master
|
||||
that scrubs fine on a laptop stutters on a phone. A **smaller frame + tighter GOP** fixes
|
||||
that (and halves the bytes on cellular). Produce a `-m.mp4` sibling for every clip:
|
||||
**The mobile version is the native 9:16 portrait chain (§6b).** This section's crop
|
||||
encodes exist for one case: the user opted into mobile but credits can't cover the
|
||||
portrait chain — and shipping them must be called out and approved, never silent
|
||||
(portrait phones will see the landscape film's centre ~26%). The encode mechanics
|
||||
matter either way: scrubbing sets `currentTime` every frame, and a phone decoder's
|
||||
**seek cost scales with how many frames it must decode from the nearest keyframe** — so
|
||||
a 1080p `-g 8` master that scrubs fine on a laptop stutters on a phone. A **smaller
|
||||
frame + tighter GOP** fixes that (and halves the bytes on cellular). The crop `-m.mp4`
|
||||
sibling per clip:
|
||||
|
||||
```bash
|
||||
# 720p, GOP 4 (twice the keyframes = ~half the seek-decode work), crf 23, same sharpen/faststart.
|
||||
@@ -151,6 +155,33 @@ If phone scrubbing still stutters, tighten the GOP further (`-g 2`, or `-g 1` fo
|
||||
the mobile encode still pays off — the tighter GOP is what makes phone seeks cheap. All-mobile encodes stay 16:9 — the engine
|
||||
centre-crops them; see the portrait note in SKILL Step 8 / prompts.md.
|
||||
|
||||
## 6b. Native 9:16 portrait chain — THE mobile version (Step 1.5 opt-in)
|
||||
|
||||
When the user opts into mobile, this is what they get: a **parallel 9:16 chain** rendered
|
||||
natively for phones and shipped as the mobile variants — never the §6 crops (those are the
|
||||
no-credits stopgap). Same seam laws as the main chain — the portrait chain frame-locks
|
||||
against its own rendered frames, never the landscape ones. Budget ~2N-1 video gens +
|
||||
re-rolls (interiors trip the NSFW filter in portrait too); state the credit cost at the
|
||||
Step 1.5 interview.
|
||||
|
||||
1. **Portrait start canvases.** Don't hand the video model a 3:2 still and hope: composite
|
||||
each scene onto a 1080×1920 canvas in the page bg colour (island at ~94% width, visual
|
||||
centre at ~45% height). The render then opens exactly on what the portrait poster shows.
|
||||
For knocked-out stills, composite the RGBA over the bg colour first.
|
||||
2. **Dives/legs**: same prompt templates with a portrait clause up front ("Vertical
|
||||
portrait composition, the diorama centered with generous [bg] space above and below"),
|
||||
`--aspect_ratio 9:16`, same model/params as the main chain. Review each last frame
|
||||
before chaining, as ever.
|
||||
3. **Connectors**: extract first/last frames **from the 9:16 renders** and generate 9:16
|
||||
connectors between them. A native 9:16 scene mixed into cropped-16:9 neighbours pops at
|
||||
both seams — the portrait chain must be complete, not partial.
|
||||
4. **Encode** with the §6 settings but portrait-oriented scale: `scale=720:-2` (720 wide),
|
||||
`-g 4`, crf 23 → these ARE the `-m.mp4` mobile files (and they replace any §6 crop
|
||||
stopgaps that shipped earlier).
|
||||
5. **Posters**: extract each 9:16 dive's first frame → webp → wire as the section's
|
||||
`stillMobile` so the poster matches the portrait video's frame 0 (no landscape→portrait
|
||||
flash when the clip paints). Engine support: `sections[k].stillMobile`.
|
||||
|
||||
## Notes
|
||||
|
||||
- `.[0].result_url` is the field on the `--wait --json` job object. `.min_result_url` is
|
||||
|
||||
@@ -13,7 +13,10 @@ Collect and write down:
|
||||
- `TONE` — a word or two (cozy/premium, playful, industrial…).
|
||||
- `STYLE` — the art direction (default below).
|
||||
- `SECTIONS[]` — ordered list; for each: `id`, `label`, `subject` (what's in the diorama), `eyebrow`, `title`, `body` (≤ 1 sentence), `tags[]` (0–3). Last section = hero product + CTA.
|
||||
- `MOBILE` — yes/no. **Always asked** (SKILL Step 1.5), presented to the user as **beta**. Gates the `-m.mp4` encodes (pipeline §6) + `clipMobile`/`connectorsMobile` wiring + the full mobile QA.
|
||||
- `MOBILE` — yes/no. **Always asked** (SKILL Step 1.5), presented to the user as **beta**
|
||||
with the ~2× credit cost stated. Yes = the **native 9:16 portrait chain** (pipeline §6b):
|
||||
portrait renders of every dive/connector + `clipMobile`/`connectorsMobile`/`stillMobile`
|
||||
wiring + the full mobile QA. The §6 crop encodes are a no-credits stopgap only.
|
||||
|
||||
## Style preamble (default: clay diorama)
|
||||
|
||||
@@ -47,12 +50,12 @@ Tips:
|
||||
- For the final "hero product" section, drop the diorama-island framing and prompt a
|
||||
single oversized product centerpiece floating on the same background with a few small
|
||||
orbiting props.
|
||||
- **Compose for the centre.** The page renders every clip `object-fit:cover`, and a portrait
|
||||
phone crops a 16:9 frame to roughly its centre half. Keep the focal subject horizontally
|
||||
centred with a little headroom, and don't park anything essential at the far left/right
|
||||
edges — it will be cut off on phones. This also keeps the dive's focal point (which the
|
||||
camera flies toward) inside the mobile crop. For a scene that absolutely must show its full
|
||||
width on mobile, generate a separate 9:16 variant for it.
|
||||
- **Compose for the centre.** The page renders every clip `object-fit:cover`. Keep the
|
||||
focal subject horizontally centred with a little headroom, and don't park anything
|
||||
essential at the far left/right edges. Mobile ships its own native 9:16 chain
|
||||
(pipeline §6b), so this is not about surviving a crop — but a centred composition makes
|
||||
the portrait renders open cleanly from the same still, and it keeps the dive's focal
|
||||
point where the camera actually flies.
|
||||
- Aspect `3:2`, `--resolution 2k --quality high`.
|
||||
|
||||
## Leg prompt — architecture A, continuous forward take (Step 4)
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
nav: true, // show the top section nav
|
||||
atmosphere: true, // subtle gradient + drifting particles behind the clips
|
||||
sections: [
|
||||
{ id, label, still, clip, clipMobile, accent,
|
||||
{ id, label, still, stillMobile, clip, clipMobile, accent,
|
||||
scroll: 1.6, // optional per-section override of diveScroll — more scroll
|
||||
// distance = a slower, longer dwell in this scene
|
||||
linger: 0.5, // optional 0..1 — remaps time so the camera settles mid-scene
|
||||
@@ -35,6 +35,10 @@
|
||||
tighter-GOP — seek cost on a phone decoder is dominated by frames-from-keyframe,
|
||||
so a 720p, -g 4 file scrubs far smoother than the 1080p desktop master; see
|
||||
pipeline.md). Falls back to the desktop `clip` if no mobile variant is given.
|
||||
- uses `stillMobile` as the scene poster when provided (pair it with native 9:16
|
||||
clipMobile renders so the poster matches the portrait video's first frame instead
|
||||
of flashing from a landscape crop). Chosen once at mount; a desktop resize into
|
||||
phone width keeps the desktop poster (clips still switch via isMobile()).
|
||||
- coalesces seeks (never issues a new currentTime while the decoder is still
|
||||
`seeking`) so fast flicks can't pile up and freeze the video.
|
||||
- keeps the still as a live poster until the clip actually paints its first frame,
|
||||
@@ -82,8 +86,8 @@ function mountScrollWorld(container, config) {
|
||||
// ---- build the interleaved segment chain: dive0, conn0, dive1, … diveN-1 ----
|
||||
const SEGMENTS = [];
|
||||
SECTIONS.forEach((s, i) => {
|
||||
const dive = { kind: 'dive', si: i, clip: s.clip, clipM: s.clipMobile, still: s.still, accent: s.accent,
|
||||
w: s.scroll || DIVE_W, linger: s.linger || 0 };
|
||||
const dive = { kind: 'dive', si: i, clip: s.clip, clipM: s.clipMobile, still: s.still, stillM: s.stillMobile,
|
||||
accent: s.accent, w: s.scroll || DIVE_W, linger: s.linger || 0 };
|
||||
SEGMENTS.push(dive);
|
||||
s._seg = dive;
|
||||
// A connector is optional: if connectors[i] is falsy, the two dives simply
|
||||
@@ -91,7 +95,8 @@ function mountScrollWorld(container, config) {
|
||||
// connector can't be generated (e.g. a content-filter false-positive).
|
||||
if (i < N - 1 && CONNECTORS[i]) {
|
||||
SEGMENTS.push({ kind: 'conn', si: i, clip: CONNECTORS[i], clipM: CONNECTORS_M[i],
|
||||
still: SECTIONS[i + 1].still, accent: SECTIONS[i + 1].accent, w: CONN_W });
|
||||
still: SECTIONS[i + 1].still, stillM: SECTIONS[i + 1].stillMobile,
|
||||
accent: SECTIONS[i + 1].accent, w: CONN_W });
|
||||
}
|
||||
});
|
||||
const NSEG = SEGMENTS.length;
|
||||
@@ -134,7 +139,8 @@ function mountScrollWorld(container, config) {
|
||||
SEGMENTS.forEach(s => {
|
||||
const scene = el('div', 'sw-scene'); scene.style.setProperty('--sw-accent', s.accent || '');
|
||||
const img = el('img', 'sw-scene__still'); img.alt = ''; img.decoding = 'async'; img.loading = 'lazy';
|
||||
if (s.still) img.src = s.still;
|
||||
const poster = (isMobile() && s.stillM) ? s.stillM : s.still;
|
||||
if (poster) img.src = poster;
|
||||
scene.appendChild(img); stage.appendChild(scene);
|
||||
s.el = scene; s.img = img; s.video = null; s.hasClip = false;
|
||||
s.loading = false; s.ready = false; s.cur = 0; s.target = 0; s.visible = false;
|
||||
|
||||
Reference in New Issue
Block a user