fix: do not echo RemoteTestRunner control frames to test output#1881
Draft
wenytang-ms wants to merge 13 commits into
Draft
fix: do not echo RemoteTestRunner control frames to test output#1881wenytang-ms wants to merge 13 commits into
wenytang-ms wants to merge 13 commits into
Conversation
JUnitRunnerResultAnalyzer.analyzeData() forwarded every line received from the Eclipse RemoteTestRunner socket to the Test Results output channel, including the machine-readable control frames (%TSTTREE, %TESTS, %TESTE, %FAILED, %TRACES, %TESTC, %RUNTIME, ...). These frames are already consumed by processData() to build the test tree and failure messages, so echoing them verbatim produced noisy, non CLI-parity output. Only forward lines that are not Eclipse control frames (identified by a leading '%' followed by an upper-case message id). Genuine program output and stack-trace content are still forwarded. Also covers control markers that are emitted by the runner but not modelled in the MessageId enum.
There was a problem hiding this comment.
Pull request overview
This PR reduces noise in the VS Code Test Results output channel by preventing Eclipse RemoteTestRunner protocol/control frames from being echoed as user-facing output, while still allowing real program output and relevant payload lines (e.g., stack traces) through.
Changes:
- Filter
%...RemoteTestRunner control frames out oftestRun.appendOutput()inJUnitRunnerResultAnalyzer.analyzeData(). - Add a regression test ensuring control frames are not forwarded, but program output and stack-trace content are.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test/suite/JUnitAnalyzer.test.ts | Adds a regression test to ensure RemoteTestRunner control frames don’t leak into the output channel. |
| src/runners/junitRunner/JUnitRunnerResultAnalyzer.ts | Stops echoing RemoteTestRunner control frames to the output channel by filtering them during analysis. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When running JUnit tests, the Test Results output channel is filled with the Eclipse
RemoteTestRunnermachine protocol instead of readable output, e.g.:These
%-prefixed lines are the wire protocol between the test runner and the extension. They are already parsed byprocessData()to build the test tree, status, diff messages, and failure details, so echoing them verbatim to the user-facing output is pure noise and breaks parity with the CLI (mvn test/gradle test).Root cause
JUnitRunnerResultAnalyzer.analyzeData()forwarded every received line totestRun.appendOutput(), without distinguishing protocol frames/payload from user-visible output:This was introduced when the analyzer was migrated to the VS Code test API.
Fix
Filter known Eclipse
RemoteTestRunnerprotocol frame prefixes and their protocol payload blocks before appending to the Test Results output channel. This intentionally uses a known-prefix list instead of a broad%[A-Z]regex so legitimate user output such as%OK,%ABC, or%STATUS 200remains visible.The filtered prefixes include frames handled by
processData()plus runner markers not represented inMessageId, such as%TESTC,%RUNTIME,%MENTER, and%MEXIT.Expected/actual and trace payloads between
%EXPECTS/%EXPECTE,%ACTUALS/%ACTUALE, and%TRACES/%TRACEEare also filtered from raw output becauseprocessData()already uses them to populate VS Code's test failure messages, diffs, and stack traces.Genuine program output (
System.out/System.err) is still forwarded if it is present in the runner stream. The analyzer also skips only the final empty split segment produced by a newline-terminated runner chunk, avoiding spurious blank lines while preserving real blank output lines inside a chunk.Test
Added a regression test asserting that:
appendOutput, including bare delimiters (%TRACES,%TRACEE,%EXPECTS,%ACTUALS) and numeric frames (%RUNTIME20);Hello from System.out) is still forwarded;%OK,%ABC,%STATUS 200) is still forwarded;Verified locally with: