Skip to content

fix: do not echo RemoteTestRunner control frames to test output#1881

Draft
wenytang-ms wants to merge 13 commits into
mainfrom
fix/junit-output-control-frames
Draft

fix: do not echo RemoteTestRunner control frames to test output#1881
wenytang-ms wants to merge 13 commits into
mainfrom
fix/junit-output-control-frames

Conversation

@wenytang-ms

@wenytang-ms wenytang-ms commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Problem

When running JUnit tests, the Test Results output channel is filled with the Eclipse RemoteTestRunner machine protocol instead of readable output, e.g.:

%TSTTREE2,com.example.FooTest,true,1,false,-1,FooTest,,
%TESTS1,fooBar(com.example.FooTest)
%FAILED1,fooBar(com.example.FooTest)
%TRACES
java.lang.AssertionError: expected:<1> but was:<2>
%TRACEE
%TESTE1,fooBar(com.example.FooTest)
%RUNTIME20

These %-prefixed lines are the wire protocol between the test runner and the extension. They are already parsed by processData() to build the test tree, status, diff messages, and failure details, so echoing them verbatim to the user-facing output is pure noise and breaks parity with the CLI (mvn test / gradle test).

Root cause

JUnitRunnerResultAnalyzer.analyzeData() forwarded every received line to testRun.appendOutput(), without distinguishing protocol frames/payload from user-visible output:

for (const line of lines) {
    this.processData(line);
    this.testContext.testRun.appendOutput(line + '\r\n'); // echoes protocol frames and payload too
}

This was introduced when the analyzer was migrated to the VS Code test API.

Fix

Filter known Eclipse RemoteTestRunner protocol frame prefixes and their protocol payload blocks before appending to the Test Results output channel. This intentionally uses a known-prefix list instead of a broad %[A-Z] regex so legitimate user output such as %OK, %ABC, or %STATUS 200 remains visible.

The filtered prefixes include frames handled by processData() plus runner markers not represented in MessageId, such as %TESTC, %RUNTIME, %MENTER, and %MEXIT.

Expected/actual and trace payloads between %EXPECTS/%EXPECTE, %ACTUALS/%ACTUALE, and %TRACES/%TRACEE are also filtered from raw output because processData() already uses them to populate VS Code's test failure messages, diffs, and stack traces.

Genuine program output (System.out/System.err) is still forwarded if it is present in the runner stream. The analyzer also skips only the final empty split segment produced by a newline-terminated runner chunk, avoiding spurious blank lines while preserving real blank output lines inside a chunk.

Test

Added a regression test asserting that:

  • protocol frames do not reach appendOutput, including bare delimiters (%TRACES, %TRACEE, %EXPECTS, %ACTUALS) and numeric frames (%RUNTIME20);
  • expected/actual and trace protocol payloads are consumed for UI failure messages but not echoed as raw output;
  • real program output (Hello from System.out) is still forwarded;
  • percent-prefixed program output (%OK, %ABC, %STATUS 200) is still forwarded;
  • a trailing newline on a runner chunk does not append an extra blank output line.

Verified locally with:

npm run compile
npm run lint
npm test

JUnitRunnerResultAnalyzer.analyzeData() forwarded every line received
from the Eclipse RemoteTestRunner socket to the Test Results output
channel, including the machine-readable control frames (%TSTTREE,
%TESTS, %TESTE, %FAILED, %TRACES, %TESTC, %RUNTIME, ...). These frames
are already consumed by processData() to build the test tree and
failure messages, so echoing them verbatim produced noisy, non
CLI-parity output.

Only forward lines that are not Eclipse control frames (identified by a
leading '%' followed by an upper-case message id). Genuine program
output and stack-trace content are still forwarded. Also covers control
markers that are emitted by the runner but not modelled in the
MessageId enum.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces noise in the VS Code Test Results output channel by preventing Eclipse RemoteTestRunner protocol/control frames from being echoed as user-facing output, while still allowing real program output and relevant payload lines (e.g., stack traces) through.

Changes:

  • Filter %... RemoteTestRunner control frames out of testRun.appendOutput() in JUnitRunnerResultAnalyzer.analyzeData().
  • Add a regression test ensuring control frames are not forwarded, but program output and stack-trace content are.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
test/suite/JUnitAnalyzer.test.ts Adds a regression test to ensure RemoteTestRunner control frames don’t leak into the output channel.
src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts Stops echoing RemoteTestRunner control frames to the output channel by filtering them during analysis.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts Outdated
Comment thread src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts Outdated
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread test/suite/JUnitAnalyzer.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread test/suite/JUnitAnalyzer.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread test/suite/JUnitAnalyzer.test.ts Outdated
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated
Comment thread test/suite/JUnitAnalyzer.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@wenytang-ms wenytang-ms closed this Jul 8, 2026
@wenytang-ms wenytang-ms reopened this Jul 8, 2026
@wenytang-ms wenytang-ms marked this pull request as draft July 8, 2026 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants