burpsuite-project-parser: classify the whole stream, not the first line

Three defects in the first-line check, all in the same place.

The pattern accepted `{` or `[`, and `[main] INFO burp.StartBurp - ...`
is very common Java stdout. With the extension unloaded that first line
passed as JSON, awk exited 0, and the banner went to stdout as a
verified clean result -- the exact state this branch exists to prevent,
through a one-character widening.

The opposite direction was worse: a working install that prints a
licence or startup line before the JSON set not_json on line 1 and
exited 4, telling the agent the extension is missing and to install it
before trusting any result. Every documented workflow would have failed
on a setup that worked before this branch. A blank first line did the
same, with an empty string in the diagnostic.

Tightening to `{` trades one for the other, so the check is now over the
whole stream: count lines that are JSON objects, fail only when that
count is zero. A preamble is tolerated, a log line is not mistaken for
data, and only JSON objects reach stdout -- everything else goes to
stderr, so a downstream grep or jq cannot match a banner.

SKILL.md also now says the exit code is invisible through a pipe, since
nearly every documented example ends in | jq or | head and reports that
command's status. stderr is the reliable signal; pipefail and
PIPESTATUS are shown for reading the code itself.

Verified against the stub: JSON 0, empty 3, banner 4, [main] INFO log 4,
licence preamble then JSON 0, blank first line 0, Burp exit 7
propagated, SIGPIPE clean, banner text absent from stdout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kz-tob
2026-08-28 16:36:23 -04:00
parent a39a28cf07
commit e3dd90c374
2 changed files with 35 additions and 12 deletions
@@ -53,6 +53,23 @@ matched nothing.
Anything other than 0 means the search result is unverified, and saying "no matching traffic" on the strength
of it is a false negative reported as a clean finding.
**Through a pipe the exit code is not yours to read.** A pipeline reports the status of its *last* command, and
nearly every example here ends in `| jq`, `| head` or `| wc -cl` — so `$?` is `head`'s 0, not the script's 3.
Two reliable signals:
- **stderr**, which reaches you regardless of piping. `Error: the parser produced no output.` or
`Error: Burp produced output, but not one JSON object` is the answer; no such block means the run was fine.
- **`set -o pipefail`** when you want the code itself, or read `${PIPESTATUS[0]}`:
```bash
set -o pipefail
{baseDir}/scripts/burp-search.sh project.burp auditItems | jq -c 'select(.severity == "High")'
echo "exit: $?"
```
Non-JSON output never reaches stdout, so a downstream `grep` or `jq` cannot match a Burp startup banner and
mistake it for data.
See [Platform Configuration](#platform-configuration) for setup instructions.
## Sub-Component Filters (USE THESE)
@@ -61,8 +61,11 @@ Output: JSON objects, one per line
Exit codes:
0 output produced
1 bad usage, or a missing file, Java or JAR
3 no output -- an empty result set and a missing parser extension look the same
4 output was not JSON -- Burp ignored the query flags, extension not loaded
3 no output at all -- an empty result set and a missing parser extension look the same
4 output, but not one JSON object -- Burp ignored the query flags, extension not loaded
Only JSON objects reach stdout; any other line is reported on stderr. Through a pipe the exit
code is invisible, so stderr is the signal to read -- or set -o pipefail and check PIPESTATUS.
EOF
exit 1
}
@@ -104,21 +107,24 @@ fi
# Burp silently ignores flags it does not recognise, so with the parser extension missing it starts
# normally and drops the query -- producing either its own non-JSON startup output or nothing at all.
# Both look like a successful search that found nothing. Stream the output through awk so the common
# case still pipes to jq/head unbuffered by a temp file, and classify what went past:
# exit 3 no output at all -- an empty result set and a missing extension are indistinguishable
# exit 4 output that is not JSON -- the flags were dropped; the extension is not loaded
# case still pipes to jq/head unbuffered by a temp file, and classify the WHOLE stream rather than
# just its first line. Judging line 1 alone breaks both ways: a working install may print a licence
# or startup line before the JSON, and a Java log line like `[main] INFO ...` would pass a check
# that accepts a leading `[`.
#
# Only JSON objects reach stdout. Anything else goes to stderr rather than being dropped, so a
# downstream `grep` or `jq` can never match a startup banner.
# exit 3 nothing at all -- an empty result set and a missing extension are indistinguishable
# exit 4 output, but no JSON -- the flags were dropped; the extension is not loaded
# `set +e` rather than `|| true`: `true` is a command of its own and would reset PIPESTATUS before it
# could be read.
set +e
"$JAVA_PATH" -jar -Djava.awt.headless=true "$BURP_JAR" \
--project-file="$PROJECT_FILE" \
"$@" | awk '
NR == 1 && $0 !~ /^[[:space:]]*[{[]/ {
not_json = 1
print "burp-search.sh: first line of output was not JSON: " $0 > "/dev/stderr"
}
{ print; lines++ }
END { if (lines == 0) exit 3; if (not_json) exit 4 }
/^[[:space:]]*\{/ { print; json++; next }
{ print "burp-search.sh: ignored non-JSON output: " $0 > "/dev/stderr"; other++ }
END { if (json == 0 && other == 0) exit 3; if (json == 0) exit 4 }
'
pipe_status=("${PIPESTATUS[@]}")
set -e
@@ -143,7 +149,7 @@ case "$awk_status" in
exit 3
;;
4)
echo "Error: Burp produced output that is not JSON, so it ignored the query flags." >&2
echo "Error: Burp produced output, but not one JSON object, so it ignored the query flags." >&2
echo "This is what a missing burpsuite-project-file-parser extension looks like: Burp starts" >&2
echo "normally and drops flags it does not recognise." >&2
echo "Install it from https://github.com/BuffaloWill/burpsuite-project-file-parser and add the" >&2