Skip to content

experimental/air: lazily page older runs in air list#5811

Merged
riddhibhagwat-db merged 2 commits into
air-clifrom
air-list-pagination
Jul 6, 2026
Merged

experimental/air: lazily page older runs in air list#5811
riddhibhagwat-db merged 2 commits into
air-clifrom
air-list-pagination

Conversation

@riddhibhagwat-db

Copy link
Copy Markdown

The interactive air list table only ever held the newest --limit (20) runs, so its ←/→ paging just scrolled that fixed window — older runs (e.g. anything before a recent date) were never fetched from the server and could never be reached.

Fix

Replace the one-shot listAirRuns with a stateful runFetcher that pages Jobs runs/list on demand:

  • It buffers a page's leftover runs and keeps its page-token cursor, so successive next() calls resume where the last stopped.
  • The interactive table (list_tui.go) holds the fetcher and calls next(listPageRows) in the background as the cursor nears the end of the loaded rows (maybeFetchfetchCmdmoreRowsMsg). New rows and their MLflow links are appended and column widths recomputed; only one fetch runs at a time.
  • The hint line shows row N/M plus (loading…) / (load failed).
  • maxListScan (2000) remains the safety ceiling on total runs scanned.

One-shot output paths (JSON, piped, and explicit --limit) are unchanged: they call fetcher.next(limit) once, so acceptance output is identical.

Tests

  • Updated the existing list/model tests to the new runFetcher/newListModel signatures.
  • Added TestRunFetcherResumesAcrossCalls: a next() that stops mid-page buffers the rest, hands it back on the next call, then reports exhaustion — without re-fetching.
  • air list / air get acceptance tests unchanged and green; build + vet clean.

This pull request and its description were written by Isaac.

The interactive table only ever held the newest `--limit` (20) runs, so
its ←/→ paging scrolled the loaded window and could never reach older
runs. Replace the one-shot listAirRuns with a stateful runFetcher that
pages runs/list on demand and buffers a page's leftover runs, so the
table fetches the next batch (and its MLflow links) as the cursor nears
the end. One-shot JSON and piped/`--limit` output is unchanged.

Also tighten a few comments that had drifted after MLflow enrichment was
added.

Co-authored-by: Isaac
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: ff6a3b7

Run: 28805012267

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🔄​ aws linux 3 4 268 1025 8:02
💚​ aws windows 1 4 272 1023 10:00
💚​ aws-ucws linux 1 4 366 939 8:30
💚​ aws-ucws windows 1 4 368 937 10:45
💚​ azure linux 1 4 267 1025 6:45
💚​ azure windows 1 4 269 1023 12:49
💚​ azure-ucws linux 1 4 365 937 8:27
💚​ azure-ucws windows 1 4 367 935 11:02
💚​ gcp linux 1 4 263 1028 7:51
🔄​ gcp windows 1 1 4 264 1026 10:59
8 interesting tests: 4 flaky, 4 SKIP
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🔄​ TestAccept 🔄​f 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/generate/auto-bind 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/generate/auto-bind/DATABRICKS_BUNDLE_ENGINE=terraform 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestSyncEnsureRemotePathIsUsableIfRepoExists ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
Top 29 slowest tests (at least 2 minutes):
duration env testname
6:47 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
5:01 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:59 aws windows TestAccept
4:57 azure-ucws windows TestAccept
4:56 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:55 azure windows TestAccept
4:53 gcp windows TestAccept
4:46 aws-ucws windows TestAccept
4:13 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:00 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:47 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:14 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:12 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:08 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:06 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:01 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:55 azure linux TestAccept
2:54 gcp linux TestAccept
2:52 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:51 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:50 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:43 aws-ucws linux TestAccept
2:42 azure-ucws linux TestAccept
2:42 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:37 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:24 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Comment thread experimental/air/cmd/list_tui.go Outdated
if url := m.rows[m.cursor].MLflowURL; url != "" && url != "-" {
return m, openURL(url)
}
if url := m.rows[m.cursor].MLflowURL; url != "" && url != "-" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This drops the previous len(m.rows) > 0 guard, so m.rows[m.cursor] will panic if the model is ever created/updated with zero rows.
Is it intended?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, this path is not currently reachable from the production path through renderListText (since it early returns on empty), but added back the guard since it is reachable via direct model construction (through tests etc)

m.offset = m.clampedOffset()
return m, m.maybeFetch()

case moreRowsMsg:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lazy-paging logic added here has no model-level test — the list_tui_test.go cases all pass a nil fetcher. A test that drives Update with a moreRowsMsg (asserting rows are appended, loading is cleared, and the error branch sets loadErr) would lock in this behavior.

Restore the len(m.rows) > 0 guard on the enter handler that the paging
change dropped, so opening MLflow can't panic on m.rows[m.cursor] when
the model holds zero rows (e.g. cursor at -1 after end/G).

Make fetchCmd/maybeFetch value receivers returning the updated model,
matching Update/View. They were the only pointer-receiver methods on
listModel, which tripped the recvcheck linter (mixed receiver types).

Add model-level tests for the moreRowsMsg handler: rows are appended and
loading cleared on a batch, loadErr is set on the error branch, and an
empty page keeps paging until the fetcher is exhausted.

Co-authored-by: Isaac
@riddhibhagwat-db riddhibhagwat-db merged commit f9f70e2 into air-cli Jul 6, 2026
22 checks passed
@riddhibhagwat-db riddhibhagwat-db deleted the air-list-pagination branch July 6, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants