Skip to content

benchmark: add metrics evaluator benchmark#1

Closed
timothyw553 wants to merge 13 commits into
tw/benchmark-infrafrom
tw/metrics-evaluator-benchmark
Closed

benchmark: add metrics evaluator benchmark#1
timothyw553 wants to merge 13 commits into
tw/benchmark-infrafrom
tw/metrics-evaluator-benchmark

Conversation

@timothyw553

@timothyw553 timothyw553 commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

Adds the first real benchmark on top of the benchmark build scaffolding from apache#825. The benchmark focuses on metrics evaluator cost because this path is central to file pruning and is a good starting point for tracking C++ Iceberg performance over time.

  • Covers both InclusiveMetricsEvaluator and StrictMetricsEvaluator.
  • Separates evaluator construction from data-file evaluation so binding/setup cost does not get mixed with per-file evaluation cost.
  • Exercises equality, range, starts-with, small IN, medium IN, and null-or-NaN predicate shapes.
  • Scales evaluation across 1, 100, and 10,000 synthetic DataFile objects.

Stack

This is PR 2 of 2 and depends on apache#825.

This PR is opened in the fork with base tw/benchmark-infra because the Apache repository is read-only for this account, so the intermediate stack base branch cannot be pushed to apache/iceberg-cpp. After apache#825 merges, this branch can be retargeted or replayed onto apache/main for upstream review.

Test Plan

  • Built this branch's benchmark target after applying the implementation on top of the scaffolding branch.
  • Ran the metrics evaluator benchmark with --benchmark_min_time=0.01s.
  • Re-ran the default build and test suite on the top of the stack: 18/18 tests passed.
  • Meson wiring is inherited from the base PR and was not locally executed because meson is not installed in this shell.

Verification Commands

cmake --build build-bench-stack --target metrics_evaluator_benchmark
build-bench-stack/src/iceberg/benchmark/metrics_evaluator_benchmark --benchmark_min_time=0.01s
cmake --build build
ctest --test-dir build --output-on-failure

@timothyw553 timothyw553 mentioned this pull request Jul 8, 2026
Merged
5 tasks
@timothyw553
timothyw553 force-pushed the tw/metrics-evaluator-benchmark branch from 027d958 to f3b4c97 Compare July 8, 2026 22:25
@timothyw553
timothyw553 force-pushed the tw/benchmark-infra branch from eaff169 to e922f8f Compare July 8, 2026 22:25
@timothyw553
timothyw553 force-pushed the tw/metrics-evaluator-benchmark branch from f3b4c97 to 348b763 Compare July 8, 2026 22:34
@timothyw553
timothyw553 force-pushed the tw/benchmark-infra branch from e922f8f to 0717dac Compare July 8, 2026 22:34
@timothyw553
timothyw553 force-pushed the tw/metrics-evaluator-benchmark branch from 348b763 to bc4b05f Compare July 10, 2026 21:19
ebyhr and others added 8 commits July 11, 2026 10:24
@ebyhr
Infra: Group github/codeql-action bumps into a single dependabot PR (a…
ebc0e07 
…pache#826)

Thus, dependabot can update these two jobs in a single PR:


https://github.com/apache/iceberg-cpp/blob/f4f6359fa9a95bde4bb2627bd8a5d45273644294/.github/workflows/codeql.yml#L56-L62

- apache/infrastructure-actions#1028
@goel-skd
feat: make ToLower Unicode-aware via utf8proc (apache#760)
4c906d1 
Replaces the ASCII-only `StringUtils::ToLower` with a Unicode-aware
implementation backed by
[utf8proc](https://github.com/JuliaStrings/utf8proc), aligning
case-insensitive name handling with Iceberg Java's
`toLowerCase(Locale.ROOT)` for simple (1:1) case mappings. The few code
points where simple and full case mapping differ (e.g. `İ` U+0130) are
tracked in apache#808.

- `ToLower` lower-cases UTF-8 using utf8proc simple (1:1) case mapping
(e.g. `CAFÉ` → `café`, `GROẞE` → `große`). Pure-ASCII input takes a
byte-wise fast path and never touches utf8proc.
- Invalid UTF-8 is handled as a correctness guarantee (raised in
review): `ToLower` is total — a byte that does not begin a valid UTF-8
sequence is passed through unchanged and decoding resumes at the next
byte, so the valid code points around it are still lower-cased. The
comparisons below therefore compare invalid bytes verbatim.
- `EqualsIgnoreCase` and `StartsWithIgnoreCase` fold through `ToLower`,
so they are case-insensitive for non-ASCII letters too. Both take an
allocation-free ASCII fast path and fall back to `ToLower` only when a
non-ASCII byte is present.
- `StartsWithIgnoreCase` no longer assumes case mapping preserves byte
length — the old byte-slice guard mishandled length-changing maps (e.g.
`İ` U+0130 → `i`). Now `"İx"` starts with `"i"`, and `"i"` starts with
`"İ"`.
- `ToUpper` is intentionally left ASCII-only — it only normalizes ASCII
enum/codec strings, and simple case mapping would be wrong for some
letters (e.g. `ß` would stay unchanged instead of becoming `SS`).
- utf8proc is pinned to 2.10.0 and wired into both the CMake (vendored
via FetchContent / system package) and Meson
(`subprojects/utf8proc.wrap`) builds, with matching source hashes and
correct static-vs-shared linkage on Windows.

## Testing

- `string_util_test.cc`: `ToLowerUnicode` (incl. invalid / truncated /
stray-continuation bytes), `ToUpperAsciiOnly`, and Unicode +
invalid-UTF-8 cases for `EqualsIgnoreCase` / `StartsWithIgnoreCase`.
- `IgnoreCaseAgreesWithToLowerOracle` exhaustively checks both ASCII
fast paths against the `ToLower` oracle over all short strings drawn
from an alphabet with length-changing maps (`İ`→`i`, `K`→`k`) and
invalid UTF-8.

## Remaining work (tracked in apache#808)

- Full case-mapping parity with Java for the code points where simple ≠
full mapping (e.g. `İ` → `i̇`).
- A streaming (allocation-free) non-ASCII path for `EqualsIgnoreCase` /
`StartsWithIgnoreCase`.

Part of apache#613 — does not fully fix it; apache#808 holds the design and
remaining plan.
Follow-up to apache#748.
@zhjwpku
fix: generate public API docs correctly (apache#804)
b4737df
@dependabot
chore(deps): bump the codeql-action group with 2 updates (apache#827)
a45bce6 
Bumps the codeql-action group with 2 updates:
[github/codeql-action/init](https://github.com/github/codeql-action) and
[github/codeql-action/analyze](https://github.com/github/codeql-action).

Updates `github/codeql-action/init` from 4.36.2 to 4.36.3
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/releases">github/codeql-action/init's
releases</a>.</em></p>
<blockquote>
<h2>v4.36.3</h2>
<p>No user facing changes.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action/init's
changelog</a>.</em></p>
<blockquote>
<h1>CodeQL Action Changelog</h1>
<p>See the <a
href="https://github.com/github/codeql-action/releases">releases
page</a> for the relevant changes to the CodeQL CLI and language
packs.</p>
<h2>[UNRELEASED]</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.20.6 and earlier. These versions of
CodeQL were discontinued on 1 July 2026 alongside GitHub Enterprise
Server 3.16, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3956">#3956</a></li>
</ul>
<h2>4.37.0 - 08 Jul 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.0">2.26.0</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3995">#3995</a></li>
<li>In addition to the existing input format, the
<code>config-file</code> input for the <code>codeql-action/init</code>
step will soon support a new <code>[owner/]repo[@ref][:path]</code>
format. All components except the repository name are optional. If
omitted, <code>owner</code> defaults to the same owner as the repository
the analysis is running for, <code>ref</code> to <code>main</code>, and
<code>path</code> to <code>.github/codeql-action.yaml</code>. Support
for this format ships in this version of the CodeQL Action, but will
only be enabled over the coming weeks. <a
href="https://redirect.github.com/github/codeql-action/pull/3973">#3973</a></li>
</ul>
<h2>4.36.3 - 01 Jul 2026</h2>
<p>No user facing changes.</p>
<h2>4.36.2 - 04 Jun 2026</h2>
<ul>
<li>Cache CodeQL CLI version information across Actions steps. <a
href="https://redirect.github.com/github/codeql-action/pull/3943">#3943</a></li>
<li>Reduce requests while waiting for analysis processing by using
exponential backoff when polling SARIF processing status. <a
href="https://redirect.github.com/github/codeql-action/pull/3937">#3937</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.6">2.25.6</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3948">#3948</a></li>
</ul>
<h2>4.36.1 - 02 Jun 2026</h2>
<p>No user facing changes.</p>
<h2>4.36.0 - 22 May 2026</h2>
<ul>
<li><em>Breaking change</em>: Bump the minimum required CodeQL bundle
version to 2.19.4. <a
href="https://redirect.github.com/github/codeql-action/pull/3894">#3894</a></li>
<li>Add support for SHA-256 Git object IDs. <a
href="https://redirect.github.com/github/codeql-action/pull/3893">#3893</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.5">2.25.5</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3926">#3926</a></li>
</ul>
<h2>4.35.5 - 15 May 2026</h2>
<ul>
<li>We have improved how the JavaScript bundles for the CodeQL Action
are generated to avoid duplication across bundles and reduce the size of
the repository by around 70%. This should have no effect on the runtime
behaviour of the CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3899">#3899</a></li>
<li>For performance and accuracy reasons, <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> will now only be enabled on a pull request when
diff-informed analysis is also enabled for that run. If diff-informed
analysis is unavailable (for example, because the PR diff ranges could
not be computed), the action will fall back to a full analysis. <a
href="https://redirect.github.com/github/codeql-action/pull/3791">#3791</a></li>
<li>If multiple inputs are provided for the GitHub-internal
<code>analysis-kinds</code> input, only <code>code-scanning</code> will
be enabled. The <code>analysis-kinds</code> input is experimental, for
GitHub-internal use only, and may change without notice at any time. <a
href="https://redirect.github.com/github/codeql-action/pull/3892">#3892</a></li>
<li>Added an experimental change which, when running a Code Scanning
analysis for a PR with <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> enabled, prefers CodeQL CLI versions that have
a cached overlay-base database for the configured languages. This speeds
up analysis for a repository when there is not yet a cached overlay-base
database for the latest CLI version. We expect to roll this change out
to everyone in May. <a
href="https://redirect.github.com/github/codeql-action/pull/3880">#3880</a></li>
</ul>
<h2>4.35.4 - 07 May 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4">2.25.4</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3881">#3881</a></li>
</ul>
<h2>4.35.3 - 01 May 2026</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.19.3 and earlier. These versions of
CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise
Server 3.15, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3837">#3837</a></li>
<li>Configurations for private registries that use Cloudsmith or GCP
OIDC are now accepted. <a
href="https://redirect.github.com/github/codeql-action/pull/3850">#3850</a></li>
<li>Best-effort connection tests for private registries now use
<code>GET</code> requests instead of <code>HEAD</code> for better
compatibility with various registry implementations. For NuGet feeds,
the test is now always performed against the service index. <a
href="https://redirect.github.com/github/codeql-action/pull/3853">#3853</a></li>
<li>Fixed a bug where two diagnostics produced within the same
millisecond could overwrite each other on disk, causing one of them to
be lost. <a
href="https://redirect.github.com/github/codeql-action/pull/3852">#3852</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/github/codeql-action/commit/54f647b7e1bb85c95cddabcd46b0c578ec92bc1a"><code>54f647b</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3984">#3984</a>
from github/update-v4.36.3-1f34ec164</li>
<li><a
href="https://github.com/github/codeql-action/commit/e78819e05527766c3c5919e3177647e280c6cb83"><code>e78819e</code></a>
Trigger checks</li>
<li><a
href="https://github.com/github/codeql-action/commit/2c9d3d63eb4941734e2d29468953529a56f5ff1c"><code>2c9d3d6</code></a>
Update changelog for v4.36.3</li>
<li><a
href="https://github.com/github/codeql-action/commit/1f34ec16430d82636d18716acc7aaa6d843b35a9"><code>1f34ec1</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3983">#3983</a>
from github/mbg/repo-props/ff-for-config-file-prop</li>
<li><a
href="https://github.com/github/codeql-action/commit/d5f0145480025b49d8b08c3f6b36e6ad41a68c90"><code>d5f0145</code></a>
Log when repository property has a value but is ignored</li>
<li><a
href="https://github.com/github/codeql-action/commit/f27f56386a3c745af8d7bbfb806098c714a5e32a"><code>f27f563</code></a>
Add test for when the FF is off</li>
<li><a
href="https://github.com/github/codeql-action/commit/0025d0f2b5676fde748a0be9725dcce18dd9f986"><code>0025d0f</code></a>
Use FF</li>
<li><a
href="https://github.com/github/codeql-action/commit/f7fa18f05d107ff6735857c3510fbff190c9a1eb"><code>f7fa18f</code></a>
Add FF for config file repo property</li>
<li><a
href="https://github.com/github/codeql-action/commit/628fc3f124e68b0151f0d2a5d81e864ee1e42335"><code>628fc3f</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3979">#3979</a>
from github/henrymercer/overlay-db-cleanup-size-tele...</li>
<li><a
href="https://github.com/github/codeql-action/commit/9cfb67bab9b32441237f92d4ba29a7f3ccff259f"><code>9cfb67b</code></a>
Add clarifying comments</li>
<li>Additional commits viewable in <a
href="https://github.com/github/codeql-action/compare/8aad20d150bbac5944a9f9d289da16a4b0d87c1e...54f647b7e1bb85c95cddabcd46b0c578ec92bc1a">compare
view</a></li>
</ul>
</details>
<br />

Updates `github/codeql-action/analyze` from 4.36.2 to 4.36.3
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/releases">github/codeql-action/analyze's
releases</a>.</em></p>
<blockquote>
<h2>v4.36.3</h2>
<p>No user facing changes.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action/analyze's
changelog</a>.</em></p>
<blockquote>
<h1>CodeQL Action Changelog</h1>
<p>See the <a
href="https://github.com/github/codeql-action/releases">releases
page</a> for the relevant changes to the CodeQL CLI and language
packs.</p>
<h2>[UNRELEASED]</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.20.6 and earlier. These versions of
CodeQL were discontinued on 1 July 2026 alongside GitHub Enterprise
Server 3.16, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3956">#3956</a></li>
</ul>
<h2>4.37.0 - 08 Jul 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.26.0">2.26.0</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3995">#3995</a></li>
<li>In addition to the existing input format, the
<code>config-file</code> input for the <code>codeql-action/init</code>
step will soon support a new <code>[owner/]repo[@ref][:path]</code>
format. All components except the repository name are optional. If
omitted, <code>owner</code> defaults to the same owner as the repository
the analysis is running for, <code>ref</code> to <code>main</code>, and
<code>path</code> to <code>.github/codeql-action.yaml</code>. Support
for this format ships in this version of the CodeQL Action, but will
only be enabled over the coming weeks. <a
href="https://redirect.github.com/github/codeql-action/pull/3973">#3973</a></li>
</ul>
<h2>4.36.3 - 01 Jul 2026</h2>
<p>No user facing changes.</p>
<h2>4.36.2 - 04 Jun 2026</h2>
<ul>
<li>Cache CodeQL CLI version information across Actions steps. <a
href="https://redirect.github.com/github/codeql-action/pull/3943">#3943</a></li>
<li>Reduce requests while waiting for analysis processing by using
exponential backoff when polling SARIF processing status. <a
href="https://redirect.github.com/github/codeql-action/pull/3937">#3937</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.6">2.25.6</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3948">#3948</a></li>
</ul>
<h2>4.36.1 - 02 Jun 2026</h2>
<p>No user facing changes.</p>
<h2>4.36.0 - 22 May 2026</h2>
<ul>
<li><em>Breaking change</em>: Bump the minimum required CodeQL bundle
version to 2.19.4. <a
href="https://redirect.github.com/github/codeql-action/pull/3894">#3894</a></li>
<li>Add support for SHA-256 Git object IDs. <a
href="https://redirect.github.com/github/codeql-action/pull/3893">#3893</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.5">2.25.5</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3926">#3926</a></li>
</ul>
<h2>4.35.5 - 15 May 2026</h2>
<ul>
<li>We have improved how the JavaScript bundles for the CodeQL Action
are generated to avoid duplication across bundles and reduce the size of
the repository by around 70%. This should have no effect on the runtime
behaviour of the CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3899">#3899</a></li>
<li>For performance and accuracy reasons, <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> will now only be enabled on a pull request when
diff-informed analysis is also enabled for that run. If diff-informed
analysis is unavailable (for example, because the PR diff ranges could
not be computed), the action will fall back to a full analysis. <a
href="https://redirect.github.com/github/codeql-action/pull/3791">#3791</a></li>
<li>If multiple inputs are provided for the GitHub-internal
<code>analysis-kinds</code> input, only <code>code-scanning</code> will
be enabled. The <code>analysis-kinds</code> input is experimental, for
GitHub-internal use only, and may change without notice at any time. <a
href="https://redirect.github.com/github/codeql-action/pull/3892">#3892</a></li>
<li>Added an experimental change which, when running a Code Scanning
analysis for a PR with <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> enabled, prefers CodeQL CLI versions that have
a cached overlay-base database for the configured languages. This speeds
up analysis for a repository when there is not yet a cached overlay-base
database for the latest CLI version. We expect to roll this change out
to everyone in May. <a
href="https://redirect.github.com/github/codeql-action/pull/3880">#3880</a></li>
</ul>
<h2>4.35.4 - 07 May 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4">2.25.4</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3881">#3881</a></li>
</ul>
<h2>4.35.3 - 01 May 2026</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.19.3 and earlier. These versions of
CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise
Server 3.15, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3837">#3837</a></li>
<li>Configurations for private registries that use Cloudsmith or GCP
OIDC are now accepted. <a
href="https://redirect.github.com/github/codeql-action/pull/3850">#3850</a></li>
<li>Best-effort connection tests for private registries now use
<code>GET</code> requests instead of <code>HEAD</code> for better
compatibility with various registry implementations. For NuGet feeds,
the test is now always performed against the service index. <a
href="https://redirect.github.com/github/codeql-action/pull/3853">#3853</a></li>
<li>Fixed a bug where two diagnostics produced within the same
millisecond could overwrite each other on disk, causing one of them to
be lost. <a
href="https://redirect.github.com/github/codeql-action/pull/3852">#3852</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/github/codeql-action/commit/54f647b7e1bb85c95cddabcd46b0c578ec92bc1a"><code>54f647b</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3984">#3984</a>
from github/update-v4.36.3-1f34ec164</li>
<li><a
href="https://github.com/github/codeql-action/commit/e78819e05527766c3c5919e3177647e280c6cb83"><code>e78819e</code></a>
Trigger checks</li>
<li><a
href="https://github.com/github/codeql-action/commit/2c9d3d63eb4941734e2d29468953529a56f5ff1c"><code>2c9d3d6</code></a>
Update changelog for v4.36.3</li>
<li><a
href="https://github.com/github/codeql-action/commit/1f34ec16430d82636d18716acc7aaa6d843b35a9"><code>1f34ec1</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3983">#3983</a>
from github/mbg/repo-props/ff-for-config-file-prop</li>
<li><a
href="https://github.com/github/codeql-action/commit/d5f0145480025b49d8b08c3f6b36e6ad41a68c90"><code>d5f0145</code></a>
Log when repository property has a value but is ignored</li>
<li><a
href="https://github.com/github/codeql-action/commit/f27f56386a3c745af8d7bbfb806098c714a5e32a"><code>f27f563</code></a>
Add test for when the FF is off</li>
<li><a
href="https://github.com/github/codeql-action/commit/0025d0f2b5676fde748a0be9725dcce18dd9f986"><code>0025d0f</code></a>
Use FF</li>
<li><a
href="https://github.com/github/codeql-action/commit/f7fa18f05d107ff6735857c3510fbff190c9a1eb"><code>f7fa18f</code></a>
Add FF for config file repo property</li>
<li><a
href="https://github.com/github/codeql-action/commit/628fc3f124e68b0151f0d2a5d81e864ee1e42335"><code>628fc3f</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3979">#3979</a>
from github/henrymercer/overlay-db-cleanup-size-tele...</li>
<li><a
href="https://github.com/github/codeql-action/commit/9cfb67bab9b32441237f92d4ba29a7f3ccff259f"><code>9cfb67b</code></a>
Add clarifying comments</li>
<li>Additional commits viewable in <a
href="https://github.com/github/codeql-action/compare/8aad20d150bbac5944a9f9d289da16a4b0d87c1e...54f647b7e1bb85c95cddabcd46b0c578ec92bc1a">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@WZhuo
fix: return error for reserved property keys in UpdateProperties::Set…
3d5af47 
…() (apache#817)

Previously, reserved table properties (except `format-version`) were
silently ignored when set via `UpdateProperties::Set()`. This caused
undetected configuration drift compared to Java, which throws exceptions
for such attempts.

Now attempts to set reserved properties accumulate a validation error
via `ICEBERGER_BUILDER_CHECK`, surfaced at `Apply()` time. This matches
Java's loud-failure behavior.
@huan233usc
feat: support v3 column default values in UpdateSchema (apache#793)
62e1bd8 
## What

Part 3 of 4 of Iceberg v3 column default-value support (POC apache#731), built
on the
schema-layer support merged in apache#746. Independent of the read-path PRs
(apache#792 and
the Avro follow-up).

Adds default-value handling to `UpdateSchema` (schema evolution).

## Changes

- **`AddColumn` / `AddRequiredColumn`** take an optional
`default_value`. When
provided it is set as both the column's `initial-default` and
`write-default`.
  A non-null default also lets a required column be added without
`AllowIncompatibleChanges()` — rows written before the change read the
default
  instead of null.
- **`UpdateColumnDefault(name, default)`** (new) sets, or clears with
`std::nullopt`, a column's `write-default`; the `initial-default` is
fixed when
  the column is added.
- Defaults are **cast to the column type** (rejecting uncastable or
out-of-range
values) and **preserved** across rename / doc / type-promotion updates
and
  nested field-id reassignment.
- `RequireColumn` may now mark a column that was added with a default
required.

The `SchemaField` constructor stores defaults verbatim (it does not
coerce
them), so the cast/promotion is performed explicitly at each evolution
site —
the same effect as Java, where `NestedField`'s constructor runs
`castDefault`.
Same-scale decimal precision widening is handled directly (the unscaled
value is
unchanged), since `Literal::CastTo` does not cast between decimal types.

## Tests

13 cases in `update_schema_test.cc`: add optional/required/nested column
with a
default, mismatched/narrowing rejection, `UpdateColumnDefault`
(set / clear / cast-to-type / pre-existing column),
require-after-default, and
preservation across doc updates and type promotion — including
same-scale
decimal precision promotion.

## Stack

1. apache#746 — schema: represent / serialize / validate (merged)
2. apache#792 — read path: Parquet
3. read path: Avro (follows)
4. **this PR** — schema evolution: `addColumn` / `updateColumnDefault`
@zhjwpku
ci: add RelWithDebInfo build and fix free-nonheap-object error (apach…
aa36d35 
…e#812)

Adding a RelWithDebInfo build revealed a build error. This PR fixes the
issue.

```
error: ‘void operator delete(void*, std::size_t)’ called on a pointer to an unallocated object ‘1’ [-Werror=free-nonheap-object]
```
@zhjwpku
refactor: use decimal high and low for Arrow scalar (apache#803)
755799d
@timothyw553
timothyw553 force-pushed the tw/metrics-evaluator-benchmark branch from bc4b05f to e08f726 Compare July 12, 2026 00:08
timothyw553 and others added 5 commits July 12, 2026 09:39
@timothyw553
build: add benchmark build scaffolding (apache#825)
263795e 
## Summary

Adds an opt-in benchmark build path so future C++ microbenchmarks can
live in-tree without affecting normal builds. Benchmarks stay disabled
by default, and this PR keeps the executable intentionally small so
reviewers can focus on CMake/Meson wiring and Google Benchmark
integration.

- Adds `ICEBERG_BUILD_BENCHMARKS` for CMake and a disabled-by-default
`benchmarks` Meson feature.
- Wires `src/iceberg/benchmark` into both build systems.
- Adds Google Benchmark dependency setup for CMake and Meson.
- Adds a tiny smoke benchmark so the new target is buildable in
isolation.

Part of apache#690.

## Stack

This is PR 1 of 2.

PR 2 adds the actual metrics evaluator benchmark cases on top of this
scaffolding: #1.

## Test Plan

- [x] Configured benchmark-enabled CMake build with
`-DICEBERG_BUILD_BENCHMARKS=ON`.
- [x] Built `metrics_evaluator_benchmark` on this branch.
- [x] Ran the smoke benchmark with `--benchmark_min_time=0.01s`.
- [x] Re-ran the default build and test suite on the top of the stack:
18/18 tests passed.
- [ ] Meson wiring is syntax-reviewed but not locally executed because
`meson` is not installed in this shell.

## Verification Commands

```bash
cmake -S . -B build-bench-stack -G Ninja \
  -DFETCHCONTENT_TRY_FIND_PACKAGE_MODE=NEVER \
  -DCMAKE_IGNORE_PREFIX_PATH=/opt/homebrew/anaconda3 \
  -DCMAKE_PREFIX_PATH='/opt/homebrew/opt/snappy;/opt/homebrew/opt/zstd' \
  -DICEBERG_BUILD_BENCHMARKS=ON
cmake --build build-bench-stack --target metrics_evaluator_benchmark
build-bench-stack/src/iceberg/benchmark/metrics_evaluator_benchmark --benchmark_min_time=0.01s
cmake --build build
ctest --test-dir build --output-on-failure
```
@yadavay-amzn
build: enable clang-tidy misc-include-cleaner for include hygiene (ap…
2191844 
…ache#820)
@abnobdoss
ci: retry build dir cleanup on Windows (apache#807)
0825f56 
## What
Retry `rm -rf` cleanup in the CMake CI helper scripts. Windows can
briefly keep a just-built executable or DLL open, which can make cleanup
fail even after the build and tests have passed.

`build_example.sh` retries before configuring and still lets `mkdir`
fail if the directory remains, so it will not build in a half-deleted
tree. `build_iceberg.sh` retries after the build and does not fail an
otherwise successful job if final cleanup is still blocked.

---------

Co-authored-by: Abanoub Doss <abanoub.doss@gmail.com>
@abnobdoss
ci: build Windows test leg in Debug (apache#806)
fee1884 
## What
Let the CMake CI scripts use `ICEBERG_BUILD_TYPE` and default to
`Debug`, including on Windows. MSVC builds with debug info now use
embedded `/Z7` debug info through `CMP0141` and
`CMAKE_MSVC_DEBUG_INFORMATION_FORMAT` so Debug objects remain cacheable
by sccache.

The build scripts also use the same single-config Ninja build path on
every platform. Since these jobs pass `-G Ninja`, `CMAKE_BUILD_TYPE`
controls the build and `--config Release` / `ctest -C Release` were not
needed.

## Why
Windows was the only main `Test` workflow leg still forcing the helper
scripts through `Release`. After this change, the helper-script CMake
test jobs default to `Debug` on pull requests and `main`;
`ICEBERG_BUILD_TYPE` remains available if the project wants an override.
Release verification and the SQL Catalog workflow's explicit Windows
`Release` matrix entry are unchanged.

The original Windows Debug blocker in apache#39 was an `IMPORTED_LOCATION`
failure from a multi-config CMake path. That specific failure no longer
applies because the helper scripts now use Ninja single-config builds.
The remaining Debug-only blocker was MSVC rejecting a `constexpr static
PartitionValues` test helper: `PartitionValues` owns a `std::vector`,
and MSVC Debug's checked-iterator mode (`_ITERATOR_DEBUG_LEVEL != 0`)
makes that construction fail constant evaluation. This PR constructs the
empty `PartitionValues` at the call site instead.

This follows the CI direction discussed in apache#799.

## Validation
Fork validation passed:

- `Test / AMD64 Windows 2025` passed in 23m48s on the pull-request run:
https://github.com/abnobdoss/iceberg-cpp/actions/runs/28638474339/job/84929642907
- `SQL Catalog / AMD64 Windows 2025` passed in 4m13s:
https://github.com/abnobdoss/iceberg-cpp/actions/runs/28638474303/job/84929643407
- `Meson / AMD64 Windows 2025` passed in 3m21s:
https://github.com/abnobdoss/iceberg-cpp/actions/runs/28638474339/job/84929642897

The Windows Debug test job reported 45% sccache hits, 385 hits, 477
misses, and 0 errors. The first post-merge Apache `main` run will reseed
Windows cache entries because the previous entries were built from
Release objects.

Closes apache#39.

---------

Co-authored-by: Abanoub Doss <abanoub.doss@gmail.com>
@timothyw553
benchmark: add metrics evaluator benchmark
2b0d474
@timothyw553
timothyw553 force-pushed the tw/metrics-evaluator-benchmark branch from e08f726 to 2b0d474 Compare July 14, 2026 08:31
@timothyw553

timothyw553 commented Jul 14, 2026

Copy link
Copy Markdown
Owner Author

Superseded by apache#831 now that apache#825 has merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@timothyw553 @ebyhr @goel-skd @zhjwpku @WZhuo @huan233usc @yadavay-amzn @abnobdoss