Skip to content

TEZ-4725: Fix flaky tests in TestAMRecoveryAggregationBroadcast#520

Open
maheshrajus wants to merge 1 commit into
apache:masterfrom
maheshrajus:TEZ-4725_flaky
Open

TEZ-4725: Fix flaky tests in TestAMRecoveryAggregationBroadcast#520
maheshrajus wants to merge 1 commit into
apache:masterfrom
maheshrajus:TEZ-4725_flaky

Conversation

@maheshrajus

Copy link
Copy Markdown
Contributor

Root causes and fixes:

  1. TestAMRecoveryAggregationBroadcast.testMapJoinTemporalFailure (race condition)
    Replace fixed Thread.sleep(10s) before AM kill with a deterministic
    waitForVertexSucceeded() helper that polls DAGClient.getVertexStatus()
    every 500ms (up to 60s) until the target vertices reach SUCCEEDED state.
    Each test now waits for only the vertices it logically depends on before
    killing the AM, ensuring the recovery log assertions always see the
    expected counts.
    Make OUT_PATH unique per test run (random suffix) to eliminate cross-test.

  2. DAGClientRPCImpl / TezClientUtils: port out of range:-1 (YARN-808 gap)
    YARN sets rpcPort=-1 when an AM container is allocated (state=RUNNING)
    but the AM has not yet bound its RPC listener. The existing guard only
    checked rpcPort==0 (protobuf default), so rpcPort==-1 reached
    NetUtils.createSocketAddrForHost(), which threw
    IllegalArgumentException: port out of range:-1.
    Fix DAGClientRPCImpl.createAMProxyIfNeeded(): rpcPort == 0 → rpcPort <= 0.
    Fix TezClientUtils.getAMProxy(FrameworkClient,...): add the same
    rpcPort <= 0 guard

@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 4m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 3 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 0m 35s Maven dependency ordering for branch
+1 💚 mvninstall 4m 33s master passed
+1 💚 compile 3m 59s master passed
+1 💚 checkstyle 1m 4s master passed
+1 💚 javadoc 1m 11s master passed
+0 🆗 spotbugs 1m 23s tez-api in master has 92 extant spotbugs warnings.
+0 🆗 spotbugs 0m 53s tez-tests in master has 6 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 9s Maven dependency ordering for patch
+1 💚 mvninstall 3m 51s the patch passed
+1 💚 codespell 1m 37s No new issues.
+1 💚 compile 4m 5s the patch passed
+1 💚 javac 4m 5s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 30s /results-checkstyle-tez-api.txt tez-api: The patch generated 2 new + 62 unchanged - 0 fixed = 64 total (was 62)
+1 💚 javadoc 1m 10s the patch passed
+1 💚 spotbugs 2m 33s the patch passed
_ Other Tests _
+1 💚 unit 75m 58s root in the patch passed.
+1 💚 asflicense 1m 0s The patch does not generate ASF License warnings.
111m 9s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-520/1/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux c15aef73b5a9 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 4bccf3a
Default Java Eclipse Adoptium-21.0.11+10-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-520/1/testReport/
Max. process+thread count 1525 (vs. ulimit of 5500)
modules C: tez-api tez-tests U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-520/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants