Optimize runs for Auto Validate Examples#1786
Draft
siri-varma wants to merge 7 commits into
Draft
Conversation
Introduce a standard and tooling to run the mm.py example-validation READMEs concurrently against a single shared 'dapr init' runtime: - examples/validate/PARALLEL_STANDARD.md: the isolation standard (unique slug-prefixed app-ids -> free Redis state isolation, omit dapr ports so the CLI auto-assigns, unique app-ports for server apps, namespaced topics/actor types). - examples/validate/run-parallel.sh: bounded-concurrency mm.py runner with per-README logs, per-README retry, and exit-code aggregation. - examples/validate/check-parallel-safe.sh: static checker enforcing the rules. - examples/validate/readmes.txt: canonical README list. Migrate a 3-README pilot (state, invoke/http, pubsub) to the standard and add a TEMPORARY push-triggered probe workflow that runs them sequentially then in parallel on one shared runtime to confirm auto-ports, isolation, and speedup. The probe workflow is not intended for master. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
Dapr only exposes a scoped component to the app-ids in its 'scopes:' list. The pilot renamed the pubsub app-ids but not the scopes in components/pubsub/redis_messagebus.yaml, so daprd reported 'pubsub messagebus is not found'. Update scopes to the renamed app-ids and document the component-scopes rule in PARALLEL_STANDARD.md. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
Complete the parallel-safe migration across every example validated by mm.py: - Unique slug-prefixed app-ids (Rule 1) -> free Redis state-key isolation. - Unique app-ports from the port registry for server apps; drop hardcoded default dapr ports so the CLI auto-assigns (Rule 2). - Namespaced pub/sub topic for pubsub/stream so it no longer collides with the pubsub example on the shared broker (Rule 3). - Component scopes in components/pubsub/redis_messagebus.yaml updated to the renamed pubsub and pubsub/stream app-ids. - Java cascades updated in lockstep: invoke/http + invoke/grpc + tracing service-invocation target app-ids, tracing Zipkin span-name assertions, workflows multiapp child-workflow app-ids, jobs Spring --server.port. - checker gains Rule 4 (component-scope app-ids must be slug-prefixed). check-parallel-safe.sh now reports 0 violations across all 17 READMEs. Probe workflow updated to run all 17 sequentially then in parallel on one dapr init. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
- validate.yml now runs the example READMEs concurrently on one shared dapr runtime via examples/validate/run-parallel.sh -j 4 (spring-boot-examples runs unchanged). - check-parallel-safe.sh: add Rule 4 (component-scope app-ids must be slug-prefixed) so app-id renames can't silently break Dapr component scopes. - Remove the temporary parallel-examples-probe.yml workflow (was only for validating this change on the branch; not intended for master). Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
crypto and jobs pin the Dapr sidecar ports in Java via withPropertyOverrides (CryptoExample/StreamingCryptoExample -> 3500/50001; DemoJobsClient -> 3500/51439; the jobs client runs without its own sidecar so it must hardcode). Dropping the explicit --dapr-*-port flags moved the sidecar to auto-assigned ports while the apps still dialed the old ones -> gRPC connection failure. Give each a unique fixed port in its registry range and match the Java overrides: crypto -> --dapr-http-port 3180 --dapr-grpc-port 3181 jobs -> --dapr-http-port 3261 --dapr-grpc-port 3262 Auto-assigned ports are high ephemeral, so these fixed low ports don't collide. Also remove examples/validate/PARALLEL_STANDARD.md (rules stay documented in the checker header) and fix the dangling references. Probe workflow re-added temporarily to validate this fix + the full parallel run; will be removed once green. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
The jobs example schedules a recurring job 'dapr-job-1' (cron '* * * * * *') and never deletes it, so the Dapr scheduler retains it across runs. Re-running (or retrying) the example then fails with ALREADY_EXISTS. Delete any existing instance before scheduling so the example is re-runnable and safe under the run-parallel per-README retry and validate.yml's retry wrapper. Concurrency itself was fine: the -j 4 parallel run passed 16/17, only jobs collided. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
The -j 4 parallel run passed 16/17 but the shared runner (JVM-heavy examples) shows intermittent timeout/kill flakiness (~1 example/run). Lower concurrency to -j 2 and raise per-README retries to 2 (3 attempts; safe now that jobs is idempotent) for a reliable run; still ~1.7-2x faster than sequential. Probe now emits parallel failures as GitHub error annotations for diagnosis. Signed-off-by: Siri Varma Vegiraju <siri.varma@outlook.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1786 +/- ##
============================================
- Coverage 77.06% 77.02% -0.05%
Complexity 2324 2324
============================================
Files 245 245
Lines 7198 7198
Branches 760 760
============================================
- Hits 5547 5544 -3
- Misses 1285 1288 +3
Partials 366 366 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Please explain the changes you've made
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: