Skip to content

feat: Enforce toUpper for objectids - BED-8944#202

Merged
ktstrader merged 8 commits into
mainfrom
BED-8944-enforce-toUpper
Jul 17, 2026
Merged

feat: Enforce toUpper for objectids - BED-8944#202
ktstrader merged 8 commits into
mainfrom
BED-8944-enforce-toUpper

Conversation

@ktstrader

@ktstrader ktstrader commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Description

AzureHound now emits uppercased identifier fields (object IDs, tenant IDs, principal IDs, ARM resource scopes, etc.) directly in its JSON output. Previously BloodHound Enterprise normalized these identifiers to uppercase internally (toUpper) during ingest. This change moves that normalization to the collector so BHE can trust the collector's casing and eventually retire its internal string-normalization pass (gated behind the use_raw_object_id flag on the BHE side).

The uppercasing is implemented at the JSON-serialization boundary via MarshalJSON on the wrapper models, so the collector's in-memory data and API interactions are unchanged — only the emitted output is normalized. All conversions are non-mutating (they operate on copies/aliases).

Motivation and Context

BHE historically uppercased Azure identifiers during ingest to make graph node lookups case-insensitive. Doing this at collection time instead:

  • Produces a self-consistent output artifact (identifiers already match how they'll be stored).
  • Lets BHE drop redundant per-ingest normalization.
  • Eliminates a class of "case-split" graph anomalies where the same logical entity ended up as two nodes differing only by casing.

Resolves: BED-8944

What changed

  • models/utils.go — shared, non-mutating helpers:
    • UpperRoleAssignment — uppercases Properties.PrincipalId and Properties.Scope.
    • UpperManagedIdentity — uppercases system- and user-assigned identity principal IDs (rebuilds the map rather than mutating it).
    • OmitEmptyUpper(raw, keys...) — strips empty entries and uppercases the given top-level string keys.
    • A helper to uppercase string slices.
  • ~59 models/*.go wrapper types gained (or had extended) MarshalJSON methods that uppercase their identifier fields (object IDs, tenant IDs, resource IDs, scopes, on-prem SIDs, role-assignment endpoints, FIC IDs, etc.). UUID-typed fields (e.g. PrincipalId uuid.UUID) are uppercased via a map[string]any override since the typed value can't hold an uppercased string.
  • models/azure/descendant-info.go — management-group descendant IDs uppercased.
  • New tests: models/marshal_test.go (comprehensive MarshalJSON coverage across all model types) and additions to models/utils_test.go.

Explicitly not uppercased (deliberate)

  • Display-only fields: DisplayName, Name, TenantName, Issuer, Subject — preserved as-is for readability.
  • Constant-matched fields: roleDefinitionId, roleTemplateId, appRoleId — left untouched because BHE ingest matches these against lowercase built-in constants. Uppercasing them would break edge/permission resolution. (Note: role-assignments.go / role-eligibility-schedule-instance.go / role-management-policy-assignment.go uppercase RoleDefinitionId only where it functions as a scope/identifier, not a constant-match key — covered by tests.)

Validation

The output was validated end-to-end against BHE by comparing three independently-produced graphs of the same tenant:

Path Collector BHE use_raw_object_id Nodes / Edges
Baseline (main) v2.12.2 BHE main off 24,411 / 2,798,941
Baseline (branch) v2.12.2 feature branch off 24,411 / 2,798,941
New path this branch (uppercased) feature branch on 24,411 / 2,798,941

Case-insensitive per-node and per-edge diffs across all three pairs were identical (0 additions, 0 removals), with matching per-node-kind and all 47 per-rel-kind counts, and 0 case-split groups / 0 orphan stub nodes. The uppercased-collector path produces a graph equivalent (modulo uniform casing) to the current production behavior.

Compatibility / rollout

  • This output is only safe to ingest into a BHE build that either (a) still runs its internal toUpper (older BHE — casing is idempotent), or (b) has use_raw_object_id enabled.
  • Pending (follow-up, not in this PR): bump the file-sink ingest Version (sinks/file.go, currently 5 → 6) so BHE can reject pre-uppercasing collector output when use_raw_object_id is enabled.

Testing

go test ./models/...
All MarshalJSON tests pass, including assertions that display/constant fields are preserved and that conversions are non-mutating.

Summary by CodeRabbit

  • Bug Fixes
    • Standardized JSON serialization to uppercase key resource, tenant, subscription, group, application, and identifier fields across supported Azure-related models.
    • Normalized nested payloads (owners, contributors, access administrators, managed identities, and role-assignment endpoints) for consistent casing.
    • Improved handling of empty/nil embedded objects to emit the correct JSON (null vs omitted/normalized), while preserving intended display-oriented fields and non-mutating source values where required.
  • Tests
    • Expanded the JSON marshaling test suite to validate identifier normalization, nested normalization, non-mutating behavior, and the new empty-and-uppercase helper.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: df9d0e0e-81e4-436b-b9e0-406bf13dbffe

📥 Commits

Reviewing files that changed from the base of the PR and between e48f4c1 and e0ce998.

📒 Files selected for processing (3)
  • models/app-fic.go
  • models/device-owner.go
  • models/group-member.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • models/group-member.go
  • models/app-fic.go
  • models/device-owner.go

Walkthrough

Custom MarshalJSON implementations now uppercase Azure identifiers across core models, relationship payloads, and role-assignment wrappers. Shared helpers provide non-mutating normalization, while tests cover serialized casing, preserved fields, errors, and source immutability.

Changes

JSON identifier normalization

Layer / File(s) Summary
Normalization helpers and core models
models/utils.go, models/*
Adds reusable uppercase-copy helpers and custom marshaling for core application, identity, directory, subscription, tenant, and Azure resource models.
Owner, member, contributor, and access payloads
models/*owner.go, models/*member.go, models/*contributor.go, models/*user-access-admin.go
Normalizes relationship endpoint IDs and nested role-assignment payloads across application, group, service principal, Key Vault, management group, resource group, subscription, and virtual machine models.
Role assignment and endpoint serialization
models/*role-assignment.go, models/role-assignments.go, models/role-eligibility-schedule-instance.go, models/role-management-policy-assignment.go, models/azure/descendant-info.go
Uppercases resource endpoints, principals, scopes, tenants, role definitions, and approver lists while preserving explicitly excluded fields.
Marshaling behavior tests
models/marshal_test.go, models/utils_test.go
Validates normalized JSON output, preserved fields, error handling, and non-mutating serialization.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Model
  participant NormalizationHelpers
  participant JSONEncoder
  Model->>NormalizationHelpers: copy and normalize identifiers
  NormalizationHelpers->>JSONEncoder: marshal alias payload
  JSONEncoder-->>Model: normalized JSON
Loading

Poem

I’m a rabbit with uppercase ears,
Hopping through JSON fields and tiers.
IDs shine bright, copies stay tame,
Nested roles join the casing game.
Tests thump softly: “No source changed!”

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: uppercasing object IDs in JSON serialization.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch BED-8944-enforce-toUpper

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
models/app-fic.go (1)

36-50: 🩺 Stability & Availability | 🔴 Critical | ⚡ Quick win

Missing safe handling for empty json.RawMessage fields during serialization.

The MarshalJSON implementations for AppFIC, GroupMember, and DeviceOwner crash when their raw JSON fields (FIC, Member, Owner) are empty or nil.

  • In AppFIC.MarshalJSON, returning nil, nil directly violates the json.Marshaler contract and causes json.Marshal on the parent object to fail with unexpected end of JSON input.
  • In GroupMember.MarshalJSON and DeviceOwner.MarshalJSON, passing an empty json.RawMessage to OmitEmptyUpper without a length check causes its internal json.Unmarshal call to fail with unexpected end of JSON input.

To resolve this safely, check the length of the json.RawMessage before processing it. Additionally, these methods can be refactored to use the same elegant Alias pattern adopted elsewhere in this PR (e.g., App, VirtualMachineOwner), which avoids unnecessary map allocations and preserves strict struct field ordering.

  • models/app-fic.go#L36-L50: replace the method with the Alias pattern and an empty-length guard.
  • models/group-member.go#L30-L40: replace the method with the Alias pattern and an empty-length guard.
  • models/device-owner.go#L30-L40: replace the method with the Alias pattern and an empty-length guard.
🛠️ Proposed fixes

models/app-fic.go

-func (s *AppFIC) MarshalJSON() ([]byte, error) {
-	output := make(map[string]any)
-	output["appId"] = strings.ToUpper(s.AppId)
-
-	if s.FIC == nil {
-		return nil, nil
-	}
-
-	if fic, err := OmitEmptyUpper(s.FIC, "id"); err != nil {
-		return nil, err
-	} else {
-		output["fic"] = fic
-		return json.Marshal(output)
-	}
-}
+func (s *AppFIC) MarshalJSON() ([]byte, error) {
+	type Alias AppFIC
+	a := Alias(*s)
+	a.AppId = strings.ToUpper(a.AppId)
+
+	if len(a.FIC) > 0 {
+		fic, err := OmitEmptyUpper(a.FIC, "id")
+		if err != nil {
+			return nil, err
+		}
+		a.FIC = fic
+	} else {
+		a.FIC = nil
+	}
+	return json.Marshal(a)
+}

models/group-member.go

-func (s GroupMember) MarshalJSON() ([]byte, error) {
-	output := make(map[string]any)
-	output["groupId"] = strings.ToUpper(s.GroupId)
-
-	if member, err := OmitEmptyUpper(s.Member, "id"); err != nil {
-		return nil, err
-	} else {
-		output["member"] = member
-		return json.Marshal(output)
-	}
-}
+func (s GroupMember) MarshalJSON() ([]byte, error) {
+	type Alias GroupMember
+	a := Alias(s)
+	a.GroupId = strings.ToUpper(a.GroupId)
+
+	if len(a.Member) > 0 {
+		member, err := OmitEmptyUpper(a.Member, "id")
+		if err != nil {
+			return nil, err
+		}
+		a.Member = member
+	} else {
+		a.Member = nil
+	}
+	return json.Marshal(a)
+}

models/device-owner.go

-func (s DeviceOwner) MarshalJSON() ([]byte, error) {
-	output := make(map[string]any)
-	output["deviceId"] = strings.ToUpper(s.DeviceId)
-
-	if owner, err := OmitEmptyUpper(s.Owner, "id"); err != nil {
-		return nil, err
-	} else {
-		output["owner"] = owner
-		return json.Marshal(output)
-	}
-}
+func (s DeviceOwner) MarshalJSON() ([]byte, error) {
+	type Alias DeviceOwner
+	a := Alias(s)
+	a.DeviceId = strings.ToUpper(a.DeviceId)
+
+	if len(a.Owner) > 0 {
+		owner, err := OmitEmptyUpper(a.Owner, "id")
+		if err != nil {
+			return nil, err
+		}
+		a.Owner = owner
+	} else {
+		a.Owner = nil
+	}
+	return json.Marshal(a)
+}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/app-fic.go` around lines 36 - 50, Update MarshalJSON in
models/app-fic.go:36-50, models/group-member.go:30-40, and
models/device-owner.go:30-40 to use the established Alias pattern, checking each
raw JSON field (FIC, Member, or Owner) for zero length before calling
OmitEmptyUpper. Preserve normal serialization for populated fields and return
valid JSON for empty or nil fields instead of returning nil bytes or
unmarshalling empty data.
🧹 Nitpick comments (1)
models/utils.go (1)

66-79: 🗄️ Data Integrity & Integration | 🔵 Trivial | 💤 Low value

Avoid else after return and prevent float64 precision loss.

The else block is unnecessary since the if block returns early. Additionally, json.Unmarshal into any parses all numbers as float64, which can silently truncate precision for large integers. Consider using json.NewDecoder with UseNumber() to prevent potential data loss.

🛠️ Proposed refactor
-	if err := json.Unmarshal(raw, &data); err != nil {
-		return nil, err
-	} else {
-		StripEmptyEntries(data)
-		for _, key := range keys {
-			if value, ok := data[key].(string); ok {
-				data[key] = strings.ToUpper(value)
-			}
-		}
-		return json.Marshal(data)
-	}
+	dec := json.NewDecoder(bytes.NewReader(raw))
+	dec.UseNumber()
+	if err := dec.Decode(&data); err != nil {
+		return nil, err
+	}
+
+	StripEmptyEntries(data)
+	for _, key := range keys {
+		if value, ok := data[key].(string); ok {
+			data[key] = strings.ToUpper(value)
+		}
+	}
+	return json.Marshal(data)

(Note: You will need to import "bytes" if you apply this fix.)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/utils.go` around lines 66 - 79, Update OmitEmptyUpper to remove the
unnecessary else after the early unmarshal error return, and decode through
json.Decoder configured with UseNumber so numeric values retain their original
precision instead of becoming float64. Preserve the existing StripEmptyEntries,
uppercase-key processing, and JSON marshaling behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@models/app-role-assignments.go`:
- Around line 46-52: Update the JSON patching logic around the output map to use
map[string]json.RawMessage instead of map[string]any, preserving embedded
numeric values without float64 conversion. Continue replacing the principalId
entry with the uppercase s.PrincipalId.String() value, encoding it as the
appropriate raw JSON value before returning.

In `@models/automation-account.go`:
- Around line 34-43: Update AutomationAccount.MarshalJSON to uppercase-normalize
the SubscriptionId field alongside Id, ResourceGroupId, and TenantId before
marshaling, matching the normalization used by ResourceGroup.

In `@models/container-registry.go`:
- Around line 35-43: Update MarshalJSON in models/container-registry.go (lines
35-43), models/function-app.go (lines 35-43), models/logic-app.go (lines 35-43),
models/vm-scale-set.go (lines 34-42), and models/web-app.go (lines 35-43) to
uppercase the alias’s SubscriptionId before marshaling, alongside the existing
identifier normalization. Review similar core wrappers such as AutomationAccount
for the same omission and apply the normalization there if applicable.

In `@models/key-vault-access-policy.go`:
- Around line 32-38: Update KeyVaultAccessPolicy.MarshalJSON to also uppercase
the embedded AccessPolicyEntry identifiers ApplicationId and TenantId before
marshaling, alongside the existing ObjectId and KeyVaultId normalization.

In `@models/key-vault.go`:
- Around line 34-41: Update the MarshalJSON methods for KeyVault in
models/key-vault.go, AutomationAccount in models/automation-account.go, and
ManagedCluster in models/managed-cluster.go to uppercase SubscriptionId
alongside the existing identifier fields before JSON marshaling.

In `@models/role-eligibility-schedule-instance.go`:
- Around line 34-41: Update RoleEligibilityScheduleInstance.MarshalJSON to
uppercase the Id and DirectoryScopeId fields alongside RoleDefinitionId,
PrincipalId, and TenantId before marshaling. Preserve the existing alias-based
serialization flow.

In `@models/virtual-machine.go`:
- Around line 34-41: Complete identifier normalization in
VirtualMachine.MarshalJSON by uppercasing SubscriptionId and update its
serialization test to assert “SUB-1”. In
RoleManagementPolicyAssignment.MarshalJSON, uppercase Id and add serialization
coverage verifying the normalized value; apply these changes in
models/virtual-machine.go lines 34-41 and
models/role-management-policy-assignment.go lines 42-50.

---

Outside diff comments:
In `@models/app-fic.go`:
- Around line 36-50: Update MarshalJSON in models/app-fic.go:36-50,
models/group-member.go:30-40, and models/device-owner.go:30-40 to use the
established Alias pattern, checking each raw JSON field (FIC, Member, or Owner)
for zero length before calling OmitEmptyUpper. Preserve normal serialization for
populated fields and return valid JSON for empty or nil fields instead of
returning nil bytes or unmarshalling empty data.

---

Nitpick comments:
In `@models/utils.go`:
- Around line 66-79: Update OmitEmptyUpper to remove the unnecessary else after
the early unmarshal error return, and decode through json.Decoder configured
with UseNumber so numeric values retain their original precision instead of
becoming float64. Preserve the existing StripEmptyEntries, uppercase-key
processing, and JSON marshaling behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1171a94a-4312-43b8-ad04-770de45339ba

📥 Commits

Reviewing files that changed from the base of the PR and between 195e96f and 3d151ac.

📒 Files selected for processing (60)
  • models/app-fic.go
  • models/app-member.go
  • models/app-owner.go
  • models/app-role-assignments.go
  • models/app.go
  • models/automation-account.go
  • models/azure-role-assignment.go
  • models/azure/descendant-info.go
  • models/container-registry.go
  • models/device-owner.go
  • models/device.go
  • models/function-app.go
  • models/group-member.go
  • models/group-owner.go
  • models/group.go
  • models/key-vault-access-policy.go
  • models/key-vault-contributor.go
  • models/key-vault-kvcontributor.go
  • models/key-vault-owner.go
  • models/key-vault-role-assignment.go
  • models/key-vault-user-access-admin.go
  • models/key-vault.go
  • models/logic-app.go
  • models/managed-cluster.go
  • models/management-group-contributor.go
  • models/management-group-owner.go
  • models/management-group-role-assignment.go
  • models/management-group-user-access-admin.go
  • models/marshal_test.go
  • models/mgmt-group.go
  • models/resource-group-contributor.go
  • models/resource-group-owner.go
  • models/resource-group-role-assignment.go
  • models/resource-group-user-access-admin.go
  • models/resource-group.go
  • models/role-assignments.go
  • models/role-eligibility-schedule-instance.go
  • models/role-management-policy-assignment.go
  • models/role.go
  • models/service-principal-owner.go
  • models/service-principal.go
  • models/subscription-contributor.go
  • models/subscription-owner.go
  • models/subscription-role-assignment.go
  • models/subscription-user-access-admin.go
  • models/subscription.go
  • models/tenant.go
  • models/user.go
  • models/utils.go
  • models/utils_test.go
  • models/virtual-machine-admin-login.go
  • models/virtual-machine-avere-contributor.go
  • models/virtual-machine-contributor.go
  • models/virtual-machine-owner.go
  • models/virtual-machine-role-assignment.go
  • models/virtual-machine-user-access-admin.go
  • models/virtual-machine-vmcontributor.go
  • models/virtual-machine.go
  • models/vm-scale-set.go
  • models/web-app.go

Comment thread models/app-role-assignments.go Outdated
Comment thread models/automation-account.go
Comment thread models/container-registry.go
Comment thread models/key-vault-access-policy.go
Comment thread models/key-vault.go
Comment thread models/role-eligibility-schedule-instance.go
Comment thread models/virtual-machine.go
@ktstrader ktstrader self-assigned this Jul 17, 2026
@ktstrader
ktstrader merged commit 7092bc6 into main Jul 17, 2026
10 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants