v0.3.0: mobile is a native 9:16 portrait chain

The mobile version is now its own camera chain rendered natively in portrait
(portrait start canvases -> 9:16 dives + connectors frame-locked against their
own renders -> 720-wide encodes -> stillMobile posters), not a centre-crop of
the landscape film. The 16:9 crop encodes remain only as an explicitly-approved
stopgap. Engine: stillMobile poster support; mobile QA checklist verifies the
served clips are natively portrait. Interview: mobile is an opt-in question
with the ~2x credit cost stated. Also: ignore .gstack/ session dir.
This commit is contained in:
cyw
2026-07-15 20:09:58 -05:00
parent f941ef93ee
commit 694fa5f7ac
6 changed files with 105 additions and 46 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "scroll-world",
"description": "Build an immersive scroll-scrubbed 'fly through the world' landing page for any industry or brand using Higgsfield. As the visitor scrolls, a pre-rendered camera flies from outside each scene into its interior, then flows on to the next scene with no cuts — one continuous connected flight. Interviews the user for the topic, story beats, and brand kit, then generates cohesive scenes + seamless camera clips and wires a portable, framework-agnostic scroll-scrub engine. Use for a '3D world' / 'browse-through-the-industry' hero, a scroll cinematic, or a diorama landing.",
"version": "0.2.0",
"version": "0.3.0",
"author": {
"name": "cyw",
"email": "cyw@cywang.me"
+1
View File
@@ -1,2 +1,3 @@
.DS_Store
node_modules/
.gstack/
+45 -27
View File
@@ -91,15 +91,21 @@ default. Cover:
section is usually the hero product + the CTA.
5. **Mobile version (beta) — ALWAYS ask this; never silently generate both.** Ask as a
two-option choice (`AskUserQuestion` in Claude Code; a plain question elsewhere):
*"Want a mobile-optimized version too? Mobile support is in
**beta** — the scroll-scrub mechanic is desktop-native; on phones you get lighter
encodes and engine hardening, but portrait crops the 16:9 frame and low-end devices
may still stutter."* Options: "Desktop only" / "Desktop + mobile (beta)". The beta
disclaimer must be stated to the user, not just implied. What the answer gates:
- **Yes** → produce the `-m.mp4` mobile encodes (Step 6) and wire
`clipMobile`/`connectorsMobile` (Step 7); run the full mobile QA (Step 8). If any
scene's focal subject sits off-centre, offer the 9:16 hero-variant escape hatch
(extra Higgsfield credits — say so).
*"Want a mobile-optimized version too? Mobile support is in **beta** — the scroll-scrub
mechanic is desktop-native. The mobile version is a second camera chain rendered
natively in **9:16 portrait** — composed for phones, not a crop of the landscape film —
which roughly doubles the Higgsfield credit spend (state the estimated number)."*
Options: "Desktop only" / "Desktop + mobile (beta, native 9:16 — ~2× credits)". The
beta disclaimer and the credit cost must be stated to the user, not just implied.
What the answer gates:
- **Yes** → render the parallel 9:16 portrait chain and ship it as the mobile variants
(Step 6 / pipeline.md §6b): portrait start canvases → 9:16 dives + connectors
frame-locked against their own renders → 720-wide `-m.mp4` encodes → `stillMobile`
portrait posters. Wire `clipMobile`/`connectorsMobile`/`stillMobile` (Step 7); run
the full mobile QA (Step 8). Budget ~2N-1 extra video gens + NSFW re-rolls.
**Never ship the centre-crop as the mobile version by default** — if credits can't
cover the portrait chain, say so and offer the crop encodes (pipeline.md §6) as an
explicitly-labelled stopgap the user must approve.
- **No** → skip the mobile encodes and wiring entirely. The engine's phone hardening
(seek-coalescing, iOS priming, safe-area CSS) is always on regardless — that's not
a "mobile version," it's just the page not breaking when a phone visits — so a
@@ -364,15 +370,17 @@ ffmpeg -i src.mp4 -an -vf "unsharp=5:5:0.8:5:5:0.0" \
Encode all 2N-1 clips (dives + connectors) with the same settings for uniform quality.
**Mobile encodes (beta — only if the user opted in at Step 1.5).** Phone video decoders seek
far slower than a laptop's, and seek cost scales with GOP length, so the 1080p `-g 8` master
that scrubs smoothly on desktop can stutter on a phone. Produce a lighter `-m.mp4` sibling for
every clip — **720p, `-g 4`** (more keyframes = cheaper seeks), crf 23 — and wire them as
`clipMobile` / `connectorsMobile` (Step 7). The engine serves them automatically on phones and
falls back to the desktop clip when absent. The exact `encm()` script is in
`references/pipeline.md` §6. If the user chose desktop-only, skip this — the engine still
hardens phone scrubbing regardless (seek-coalescing, iOS priming), so the page degrades
gracefully rather than breaking.
**Mobile encodes (beta — only if the user opted in at Step 1.5).** The mobile version is
the **native 9:16 portrait chain** (pipeline.md §6b): portrait renders of every dive and
connector, encoded **720 wide (`scale=720:-2`), `-g 4`** (more keyframes = cheaper seeks —
phone decoders' seek cost scales with GOP length), crf 23 — wired as `clipMobile` /
`connectorsMobile`, with each portrait dive's first frame extracted as the section's
`stillMobile` poster (Step 7). The engine serves them automatically on phones and falls
back to the desktop clip when absent. The 16:9 centre-crop `encm()` encodes
(pipeline.md §6) are a **fallback only** — for when credits can't cover the portrait
chain — and shipping them must be called out to the user, never silent. If the user chose
desktop-only, skip this — the engine still hardens phone scrubbing regardless
(seek-coalescing, iOS priming), so the page degrades gracefully rather than breaking.
---
@@ -388,7 +396,9 @@ mountScrollWorld(document.getElementById('world'), {
diveScroll: 1.3, connScroll: 0.9, // viewport-heights of scroll per clip
sections: [
{ id:'farm', label:'The Farms', still:'assets/farm.webp',
clip:'assets/vid/farm.mp4', clipMobile:'assets/vid/farm-m.mp4', // mobile beta only
clip:'assets/vid/farm.mp4',
clipMobile:'assets/vid/farm-m.mp4', // mobile beta only: native 9:16 render
stillMobile:'assets/farm-m.webp', // its first frame as the portrait poster
scroll: 1.6, linger: 0.45, // optional pacing: longer dwell + camera settles mid-scene
accent:'#8FB98A', eyebrow:'From leaf to last sip', title:'It starts in the hills.',
body:'…', tags:['Single-origin','Hand-picked'] },
@@ -446,12 +456,15 @@ is the thing most likely to be wrong:
over the instant you scroll — no blank/black scene (the iOS priming fix). Test iOS Safari
specifically; it's the one that goes blank if this regresses.
- Verify the `-m.mp4` variant is actually served on mobile (Network panel), and the
heavy 1080p master on desktop.
heavy 1080p master on desktop. The mobile clips must be **natively portrait**
(`videoWidth < videoHeight` — not a downscaled 16:9 file), and the `stillMobile`
posters must be served and match each portrait clip's first frame (no
landscape→portrait flash when the video paints).
- Slowly scroll so the URL bar collapses — the page must **not jump** (height-only resizes
are ignored on touch). Rotate the device — layout should recompose cleanly.
- Portrait crops a 16:9 clip to its centre; confirm the focal subject still reads. If a
hero scene's subject sits off-centre and gets cut, recompose it (prompts.md) or generate
a 9:16 variant for that scene.
- Only if the crop **fallback** shipped (no credits for the portrait chain): portrait
crops a 16:9 clip to its centre — confirm the focal subject still reads, and remind
the user this is the stopgap, not the mobile version.
- Check reduced-motion (should fall back to the stills, no video, no particles).
---
@@ -506,10 +519,11 @@ is the thing most likely to be wrong:
- **Copy hidden behind the URL bar / notch on mobile** → use the engine's safe-area-aware
bottom offset (`env(safe-area-inset-bottom)` + `dvh`); make sure the page's
`<meta viewport>` includes `viewport-fit=cover` (the template does).
- **Portrait crops the scene** → a 16:9 clip on a tall phone shows only its centre. Keep each
scene's focal subject centred with a little headroom (prompts.md), or generate a 9:16 hero
for the scenes that matter most. The engine centre-crops (`object-fit:cover`); it can't
un-crop a widescreen composition.
- **Portrait crops the scene** → a 16:9 clip on a tall phone shows only its centre — which
is why the mobile version is the native 9:16 chain (§6b), never the crop. If you're seeing
this on a mobile build, either the crop fallback shipped (call it out to the user) or the
9:16 encodes aren't actually being served (check `videoWidth < videoHeight`). Keeping each
scene's focal subject centred (prompts.md) still matters for the desktop film itself.
- **`--generate-audio` errors on seedance** → omit it; mute in HTML and `-an` on encode.
- **Kling rejects your flags** → `kling3_0` has **no `--resolution` param** (don't pass
one; encode at whatever native res ffprobe reports) and **sound defaults on** — pass
@@ -521,6 +535,10 @@ is the thing most likely to be wrong:
- **White-box scenes** → `gpt_image_2` returns a solid bg; either match the page bg to it
or knock it out (Step 3).
- **bash 3.2** on macOS → no associative arrays in scripts.
- **Connector grabs the wrong scene's frames** (or errors on a frame that doesn't exist
yet) → the array loop ran in **zsh** (macOS default interactive shell), where arrays are
1-indexed, not bash's 0-indexed. Keep every array-driven chain step in a `#!/bin/bash`
script run via `bash script.sh` — never inline array loops in the interactive shell.
## References
+37 -6
View File
@@ -119,13 +119,17 @@ i=0; for f in "$WORK"/conn_*.mp4; do i=$((i+1)); enc "$f" "$ASSETS/vid/conn$i.mp
Now the engine config's `sections[k].clip = assets/vid/<name>.mp4` and
`connectors = [assets/vid/conn1.mp4, …]` (length N-1, in order).
## 6. Mobile encodes (Step 6) — mobile beta, only if the user opted in
## 6. Centre-crop mobile encodes — FALLBACK ONLY, not the mobile version
**Skip this section unless the user chose the mobile (beta) version in the Step 1
interview.** Scrubbing sets `currentTime` every frame, and a phone decoder's **seek cost scales with
how many frames it must decode from the nearest keyframe** — so a 1080p `-g 8` master
that scrubs fine on a laptop stutters on a phone. A **smaller frame + tighter GOP** fixes
that (and halves the bytes on cellular). Produce a `-m.mp4` sibling for every clip:
**The mobile version is the native 9:16 portrait chain (§6b).** This section's crop
encodes exist for one case: the user opted into mobile but credits can't cover the
portrait chain — and shipping them must be called out and approved, never silent
(portrait phones will see the landscape film's centre ~26%). The encode mechanics
matter either way: scrubbing sets `currentTime` every frame, and a phone decoder's
**seek cost scales with how many frames it must decode from the nearest keyframe** — so
a 1080p `-g 8` master that scrubs fine on a laptop stutters on a phone. A **smaller
frame + tighter GOP** fixes that (and halves the bytes on cellular). The crop `-m.mp4`
sibling per clip:
```bash
# 720p, GOP 4 (twice the keyframes = ~half the seek-decode work), crf 23, same sharpen/faststart.
@@ -151,6 +155,33 @@ If phone scrubbing still stutters, tighten the GOP further (`-g 2`, or `-g 1` fo
the mobile encode still pays off — the tighter GOP is what makes phone seeks cheap. All-mobile encodes stay 16:9 — the engine
centre-crops them; see the portrait note in SKILL Step 8 / prompts.md.
## 6b. Native 9:16 portrait chain — THE mobile version (Step 1.5 opt-in)
When the user opts into mobile, this is what they get: a **parallel 9:16 chain** rendered
natively for phones and shipped as the mobile variants — never the §6 crops (those are the
no-credits stopgap). Same seam laws as the main chain — the portrait chain frame-locks
against its own rendered frames, never the landscape ones. Budget ~2N-1 video gens +
re-rolls (interiors trip the NSFW filter in portrait too); state the credit cost at the
Step 1.5 interview.
1. **Portrait start canvases.** Don't hand the video model a 3:2 still and hope: composite
each scene onto a 1080×1920 canvas in the page bg colour (island at ~94% width, visual
centre at ~45% height). The render then opens exactly on what the portrait poster shows.
For knocked-out stills, composite the RGBA over the bg colour first.
2. **Dives/legs**: same prompt templates with a portrait clause up front ("Vertical
portrait composition, the diorama centered with generous [bg] space above and below"),
`--aspect_ratio 9:16`, same model/params as the main chain. Review each last frame
before chaining, as ever.
3. **Connectors**: extract first/last frames **from the 9:16 renders** and generate 9:16
connectors between them. A native 9:16 scene mixed into cropped-16:9 neighbours pops at
both seams — the portrait chain must be complete, not partial.
4. **Encode** with the §6 settings but portrait-oriented scale: `scale=720:-2` (720 wide),
`-g 4`, crf 23 → these ARE the `-m.mp4` mobile files (and they replace any §6 crop
stopgaps that shipped earlier).
5. **Posters**: extract each 9:16 dive's first frame → webp → wire as the section's
`stillMobile` so the poster matches the portrait video's frame 0 (no landscape→portrait
flash when the clip paints). Engine support: `sections[k].stillMobile`.
## Notes
- `.[0].result_url` is the field on the `--wait --json` job object. `.min_result_url` is
+10 -7
View File
@@ -13,7 +13,10 @@ Collect and write down:
- `TONE` — a word or two (cozy/premium, playful, industrial…).
- `STYLE` — the art direction (default below).
- `SECTIONS[]` — ordered list; for each: `id`, `label`, `subject` (what's in the diorama), `eyebrow`, `title`, `body` (≤ 1 sentence), `tags[]` (03). Last section = hero product + CTA.
- `MOBILE` — yes/no. **Always asked** (SKILL Step 1.5), presented to the user as **beta**. Gates the `-m.mp4` encodes (pipeline §6) + `clipMobile`/`connectorsMobile` wiring + the full mobile QA.
- `MOBILE` — yes/no. **Always asked** (SKILL Step 1.5), presented to the user as **beta**
with the ~2× credit cost stated. Yes = the **native 9:16 portrait chain** (pipeline §6b):
portrait renders of every dive/connector + `clipMobile`/`connectorsMobile`/`stillMobile`
wiring + the full mobile QA. The §6 crop encodes are a no-credits stopgap only.
## Style preamble (default: clay diorama)
@@ -47,12 +50,12 @@ Tips:
- For the final "hero product" section, drop the diorama-island framing and prompt a
single oversized product centerpiece floating on the same background with a few small
orbiting props.
- **Compose for the centre.** The page renders every clip `object-fit:cover`, and a portrait
phone crops a 16:9 frame to roughly its centre half. Keep the focal subject horizontally
centred with a little headroom, and don't park anything essential at the far left/right
edges — it will be cut off on phones. This also keeps the dive's focal point (which the
camera flies toward) inside the mobile crop. For a scene that absolutely must show its full
width on mobile, generate a separate 9:16 variant for it.
- **Compose for the centre.** The page renders every clip `object-fit:cover`. Keep the
focal subject horizontally centred with a little headroom, and don't park anything
essential at the far left/right edges. Mobile ships its own native 9:16 chain
(pipeline §6b), so this is not about surviving a crop — but a centred composition makes
the portrait renders open cleanly from the same still, and it keeps the dive's focal
point where the camera actually flies.
- Aspect `3:2`, `--resolution 2k --quality high`.
## Leg prompt — architecture A, continuous forward take (Step 4)
+11 -5
View File
@@ -15,7 +15,7 @@
nav: true, // show the top section nav
atmosphere: true, // subtle gradient + drifting particles behind the clips
sections: [
{ id, label, still, clip, clipMobile, accent,
{ id, label, still, stillMobile, clip, clipMobile, accent,
scroll: 1.6, // optional per-section override of diveScroll — more scroll
// distance = a slower, longer dwell in this scene
linger: 0.5, // optional 0..1 — remaps time so the camera settles mid-scene
@@ -35,6 +35,10 @@
tighter-GOP — seek cost on a phone decoder is dominated by frames-from-keyframe,
so a 720p, -g 4 file scrubs far smoother than the 1080p desktop master; see
pipeline.md). Falls back to the desktop `clip` if no mobile variant is given.
- uses `stillMobile` as the scene poster when provided (pair it with native 9:16
clipMobile renders so the poster matches the portrait video's first frame instead
of flashing from a landscape crop). Chosen once at mount; a desktop resize into
phone width keeps the desktop poster (clips still switch via isMobile()).
- coalesces seeks (never issues a new currentTime while the decoder is still
`seeking`) so fast flicks can't pile up and freeze the video.
- keeps the still as a live poster until the clip actually paints its first frame,
@@ -82,8 +86,8 @@ function mountScrollWorld(container, config) {
// ---- build the interleaved segment chain: dive0, conn0, dive1, … diveN-1 ----
const SEGMENTS = [];
SECTIONS.forEach((s, i) => {
const dive = { kind: 'dive', si: i, clip: s.clip, clipM: s.clipMobile, still: s.still, accent: s.accent,
w: s.scroll || DIVE_W, linger: s.linger || 0 };
const dive = { kind: 'dive', si: i, clip: s.clip, clipM: s.clipMobile, still: s.still, stillM: s.stillMobile,
accent: s.accent, w: s.scroll || DIVE_W, linger: s.linger || 0 };
SEGMENTS.push(dive);
s._seg = dive;
// A connector is optional: if connectors[i] is falsy, the two dives simply
@@ -91,7 +95,8 @@ function mountScrollWorld(container, config) {
// connector can't be generated (e.g. a content-filter false-positive).
if (i < N - 1 && CONNECTORS[i]) {
SEGMENTS.push({ kind: 'conn', si: i, clip: CONNECTORS[i], clipM: CONNECTORS_M[i],
still: SECTIONS[i + 1].still, accent: SECTIONS[i + 1].accent, w: CONN_W });
still: SECTIONS[i + 1].still, stillM: SECTIONS[i + 1].stillMobile,
accent: SECTIONS[i + 1].accent, w: CONN_W });
}
});
const NSEG = SEGMENTS.length;
@@ -134,7 +139,8 @@ function mountScrollWorld(container, config) {
SEGMENTS.forEach(s => {
const scene = el('div', 'sw-scene'); scene.style.setProperty('--sw-accent', s.accent || '');
const img = el('img', 'sw-scene__still'); img.alt = ''; img.decoding = 'async'; img.loading = 'lazy';
if (s.still) img.src = s.still;
const poster = (isMobile() && s.stillM) ? s.stillM : s.still;
if (poster) img.src = poster;
scene.appendChild(img); stage.appendChild(scene);
s.el = scene; s.img = img; s.video = null; s.hasClip = false;
s.loading = false; s.ready = false; s.cur = 0; s.target = 0; s.visible = false;