skills/jackwener__opencli
1
0
Fork 0
mirror of https://github.com/jackwener/OpenCLI.git synced 2026-09-14 18:25:42 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(chatgpt): use data-turn to detect upload previews vs generated images (#2292)

Browse Source 
`chatgpt image` with 2+ --image attachments could return the just-uploaded
reference thumbnails instead of the actual generated image.

isUserUploadPreview() classified an <img> as a user upload (to exclude it
from waitForChatGPTImages' before/after diff) using two signals, both
broken against ChatGPT's current DOM:

- turn.querySelector('h4')?.innerText: the heading is visually hidden, so
  real Chrome's innerText resolves to '' (layout-dependent) even though
  .textContent correctly reads "You said:" / "ChatGPT said:". jsdom's
  innerText is always undefined, so the test suite never exercised this
  path either - it happened to pass via the aria-label/alt fallback below.
- button[aria-label^="Open image:"]: ChatGPT's current label for a
  multi-file attachment reads "Open image N of M: <name>", which no
  longer starts with "Open image:", so this selector stopped matching.

With both signals dead, classification fell through to alt-text sniffing.
Right after upload, an attachment thumbnail's alt/aria-label haven't
populated yet, so for a poll or two every uploaded image is misclassified
as "new". waitForChatGPTImages returns as soon as two consecutive polls
agree on a URL set - long enough for that transient window to win when
multiple attachments are involved, so it can return the uploads instead of
the real result.

Fix: check the turn <section>'s own data-turn="user"|"assistant"
attribute first. It's set structurally as soon as the turn mounts, not
tied to the attachment's async metadata, so it isn't subject to the race.
Keep the heading/aria-label checks as a fallback (now using textContent
and a substring aria-label match) for markup that lacks data-turn.

Verified live against chatgpt.com: reproduced the bug with 3 reference
images, then confirmed the patched build returns exactly the one real
generated image instead of the 3 uploaded thumbnails.

Adds regression tests for both the data-turn race and the aria-label
format change; confirmed both fail against the pre-fix code.


Claude-Session: https://claude.ai/code/session_01L29nrhaeQ4W5rjNr27z47h

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Mai Hoàng Anh Vũ
2026-08-23 02:21:56 +07:00
committed by GitHub
parent da333c99a7
commit 083f78acf5
2 changed files with 71 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clis/chatgpt/utils.js
+19 -3
View File
@@ -2560,12 +2560,28 @@ export async function getChatGPTVisibleImageUrls(page) {
return /avatar|profile|logo|icon/.test(text);
};
const isUserUploadPreview = (img) => {
const alt = (img.getAttribute('alt') || '').toLowerCase();
const turn = img.closest('section[data-testid^="conversation-turn"]');
const heading = (turn?.querySelector('h4')?.innerText || '').toLowerCase();
// Authoritative signal first: ChatGPT stamps data-turn directly on the
// turn section as soon as the turn mounts, well before an attached
// image's alt text/aria-label finish populating. Racing multiple
// uploads against that async metadata (the old behaviour here) let
// still-generic upload-preview thumbnails pass as "new" images for a
// poll or two, which is long enough to satisfy the stability check in
// waitForChatGPTImages and return the wrong (uploaded, not generated)
// images when 2+ files were attached.
const turnRole = turn?.getAttribute('data-turn') || '';
if (turnRole === 'user') return true;
if (turnRole === 'assistant') return false;
// Fallback for markup without data-turn. innerText reads as empty
// on a visually-hidden heading in real Chrome (jsdom has no layout and
// never exposed this gap) - use textContent instead.
const heading = (turn?.querySelector('h4')?.textContent || '').toLowerCase();
if (/you said|你说/.test(heading)) return true;
if (/chatgpt|assistant|助手/.test(heading)) return false;
const openButtonLabel = (img.closest('button[aria-label^="Open image:"], button[aria-label^="打开图片："]')?.getAttribute('aria-label') || '').toLowerCase();
const alt = (img.getAttribute('alt') || '').toLowerCase();
// ChatGPT's multi-image "Open image" button label now reads
// "Open image N of M: name", not the older "Open image: name".
const openButtonLabel = (img.closest('button[aria-label*="Open image"], button[aria-label*="打开图片"]')?.getAttribute('aria-label') || '').toLowerCase();
const previewText = [alt, openButtonLabel].join(' ');
return /\.(png|jpe?g|webp|gif|heic|heif)(?:\b|$)/i.test(previewText)
|| /ref-|reference|参考|上传|upload|uploaded|attachment/.test(previewText);
clis/chatgpt/utils.test.js
+52 -1
View File
@@ -1551,7 +1551,7 @@ describe('chatgpt generated image detection', () => {
it('ignores user-uploaded previews labeled by the Chinese UI', async () => {
const page = createDomPage(`
<!doctype html>
<button aria-label="打开图片：用户上传的图片">
<button aria-label="打开图片 1 / 2 用户上传的图片">
<img alt="" src="https://chatgpt.com/backend-api/files/reference">
</button>
<section data-testid="conversation-turn-2">
@@ -1571,6 +1571,39 @@ describe('chatgpt generated image detection', () => {
]);
});
it('ignores multiple upload-preview thumbnails via data-turn before their alt/aria-label metadata populate', async () => {
// Reproduces a real regression: uploading 2+ reference images made
// waitForChatGPTImages return the just-uploaded thumbnails instead of
// the actual generated image. Right after upload, a thumbnail's alt
// text and "Open image N of M: <name>" aria-label haven't populated
// yet, so the old alt/aria-label-only fallback couldn't tell them
// apart from a real result during that window. `data-turn` on the
// turn <section> is set immediately and must be checked first.
const page = createDomPage(`
<!doctype html>
<section data-testid="conversation-turn-1" data-turn="user">
<h4>You said:</h4>
<img alt="" src="https://chatgpt.com/backend-api/uploaded/ref-1.png">
<img alt="" src="https://chatgpt.com/backend-api/uploaded/ref-2.png">
<img alt="" src="https://chatgpt.com/backend-api/uploaded/ref-3.png">
</section>
<section data-testid="conversation-turn-2" data-turn="assistant">
<h4>ChatGPT said:</h4>
<img alt="Generated image: result" src="https://chatgpt.com/backend-api/generated/foo.webp">
</section>
`, (window) => {
for (const img of window.document.querySelectorAll('img')) {
Object.defineProperty(img, 'naturalWidth', { configurable: true, value: 512 });
Object.defineProperty(img, 'naturalHeight', { configurable: true, value: 512 });
img.getBoundingClientRect = () => ({ width: 512, height: 512 });
}
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
'https://chatgpt.com/backend-api/generated/foo.webp',
]);
});
it('keeps assistant generated images even when they are inside an open-image button', async () => {
const page = createDomPage(`
<!doctype html>
@@ -1592,6 +1625,24 @@ describe('chatgpt generated image detection', () => {
]);
});
it('recognizes the "Open image N of M: name" aria-label ChatGPT uses for multi-attachment uploads', async () => {
const page = createDomPage(`
<!doctype html>
<section data-testid="conversation-turn-1">
<button aria-label="Open image 1 of 3: reference.png">
<img alt="" src="https://chatgpt.com/backend-api/uploaded/reference.png">
</button>
</section>
`, (window) => {
const img = window.document.querySelector('img');
Object.defineProperty(img, 'naturalWidth', { configurable: true, value: 512 });
Object.defineProperty(img, 'naturalHeight', { configurable: true, value: 512 });
img.getBoundingClientRect = () => ({ width: 512, height: 512 });
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([]);
});
it('exports assets for generated CSS background images', async () => {
const imageUrl = 'https://chatgpt.com/backend-api/generated/foo.webp';
const page = createDomPage(`