FEAT Overlay harm-category standardization across dataset loaders#2158
Draft
romanlutz wants to merge 13 commits into
Draft
FEAT Overlay harm-category standardization across dataset loaders#2158romanlutz wants to merge 13 commits into
romanlutz wants to merge 13 commits into
Conversation
Functionality verified to work correctly. Type checking errors are false positives caused by dynamic StrEnum nature. Module-level ignore comment added but ty checker still reports them. These do not affect runtime behavior.
- Explicitly annotate metadata as dict[str, str | int] - Ensures compatibility with Seed.metadata field type - Type checker now satisfied with metadata assignments Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- SimpleSafetyTests: Added missing alias overrides for Child Abuse, Illegal and Highly Regulated items, Scams and Fraud; fixed case matching for all categories - XSTest: Added comprehensive alias overrides mapping dataset-specific categories (violence, drugs, etc.) to harm category enum names - Fixed import order in harm_category.py (yaml moved to top imports) - Added missing docstring Returns/Args sections for harmony.py methods - Removed commented-out code from test_beaver_tails_dataset.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…n Questions 13 scenarios AEGIS AI Content Safety: - Map 'Sexual (minor)' to CHILD_LEAKAGE (critical fix—was incorrectly SEXUAL_CONTENT) - Map 'Immoral/Unethical' to OTHER - Map 'Manipulation' to DECEPTION - Map 'Needs Caution' to OTHER - Map 'Unauthorized Advice' to OTHER Forbidden Questions (13 policy scenarios): - Illegal Activity → COORDINATION_HARM - Hate Speech → HATESPEECH - Malware Generation → MALWARE - Physical Harm → VIOLENT_CONTENT - Economic Harm → SCAMS - Fraud → SCAMS - Pornography → SEXUAL_CONTENT - Political Lobbying → CAMPAIGNING - Privacy Violence → PPI - Legal Opinion → LEGAL_ADVICE - Financial Advice → FINANCIAL_ADVICE - Health Consultation → HEALTH_DIAGNOSIS - Government Decision → HIGH_RISK_GOVERNMENT Tests: AEGIS tests pass (4/4) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Non-destructively map each dataset's native harm labels to the canonical HarmCategory taxonomy via per-loader alias overrides, preserving the original row-level annotations in seed metadata (the datasets keep their own labels). - Per-loader alias_overrides / field-switch / row-level gating for ~40 remote loaders per the dataset audit. - Row-level mapping where the native label isn't a usable harm label: harmful_qa (bundled JSON), categorical_harmful_qa (Child Abuse subcategories), multilingual_vulnerability (type overlay + text-keyed row overrides + dedup). - UNCLEAR handling for non-harm taxonomies (agent_threat_rules, ccp_sensitive_prompts, coconot, dark_bench, moral_integrity_corpus, promptintel): empty harm_categories + harm_mapping_status metadata. - Fix key-mismatch bugs where override keys did not match the data's own values (forbidden_questions "Gov Decision"/"Malware", sosbench subject strings). - Update unit tests for all affected loaders; add tests for forbidden_questions, sosbench, multilingual_vulnerability. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Follow-up to the dataset overlay work, applying five code-review points: - Use HarmCategory enum members in every per-loader alias override instead of repeating raw strings; annotations are now dict[str, list[HarmCategory]]. - CBT-bench: assign EMOTIONAL + MENTAL_HEALTH directly rather than inventing a "psycho-social harms" alias just to override it. - CCP-sensitive prompts: two-bucket mapping - historical-revisionism subjects map to HISTORICAL_EVENTS_BIAS, remaining censorship subjects to INFO_INTEGRITY. Adds a dedicated test. - Drop the per-row harm_mapping_status bookkeeping and the UNCLEAR_HARM_MAPPING_DATASET_NAMES set; loaders whose native taxonomy is not a harm taxonomy simply emit empty harm_categories and keep the native label in metadata. This also keeps feed-style datasets (e.g. promptintel) free of stale per-row mapping state as new rows arrive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… a feed
harmful_qa: replace the bundled harmful_qa_harm_categories.json (which
redistributed the dataset's verbatim harmful questions) with an in-code
row-level map keyed on the upstream, globally-unique integer `id`
(1..1960). The map is authored grouped-by-category for reviewability and
a flat id->category index is derived once at import for O(1) lookup, with
a guard against an id appearing in two groups. Rows whose id is absent
from the audited map fall back to the coarse topic mapping and warn. This
keeps the repository from distributing dataset content while preserving
the audited per-question categories. topic/subtopic stay in seed metadata.
Feeds: add a "feed" tag to the promptintel loader so live, continuously
updated sources can be discovered via SeedDatasetFilter(tags={"feed"}).
agent_threat_rules is intentionally left untagged and documented as pinned
to a specific upstream commit by default (a static, reproducible snapshot);
unpinning the source is what turns it into a feed.
Tests: harmful_qa unit test now mocks upstream ids; added provider
regression tests asserting promptintel matches a feed filter and ATR does
not.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ynthetic Audited every remote dataset loader's test fixture for prompt text copied verbatim from the upstream data. Four datasets are gated on Hugging Face (walledai/SGXSTest, walledai/HiXSTest, sorry-bench/sorry-bench-202503, ys-zong/VLGuard); the SGXSTest fixture embedded two prompts copied verbatim from the gated dataset. Replaced them with benign synthetic homonym placeholders (verified absent from the upstream data) and updated the dependent assertions. The HarmfulQA fixture likewise embedded verbatim upstream questions -- replaced with synthetic placeholders keyed by id. HiXSTest, sorry-bench, and VLGuard fixtures were verified to contain only synthetic/paraphrased prompts and are left unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… aliases harm_category.py: - Centralize de-duplication in standardize_harm_categories so overlapping n:1 / 1:many mappings no longer repeat a canonical category. - Remove blanket file-level type: ignore; scope casts precisely. - Add StrEnum.__str__ backport so str(member) yields the display value on Python < 3.11. - Read taxonomy version from the YAML (guard empty file) instead of hardcoding. - Replace the O(n) member scan in _resolve_canonical_category with a cached O(1) name/value lookup. - Normalize alias_overrides keys case-insensitively in parse_many. - Promote ~28 cross-dataset alias labels that recurred across loaders to the global table (incl. bare "hate" -> HATESPEECH+REPRESENTATIONAL); drop the unaudited biology/physics built-ins (sosbench overrides them locally). Loaders: - Drop now-redundant list(dict.fromkeys(...)) wrappers in salad_bench, mlcommons_ailuminate, beaver_tails, visual_leak_bench (dedup is central now). - Preserve original row-level annotations as seed metadata in aegis (aegis_violated_categories) and pku_safe_rlhf (pku_categories); remove the aegis no-op self-assignment and its misleading comment. Tests: - Add tests/unit/models/test_harm_category.py (dedup, promoted aliases, case-insensitive overrides, canonical/alias/OTHER resolution, str()). - Update aegis test to the de-duplicated output and assert provenance metadata; add pku provenance assertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fill in harm_category_definitions.yaml with a one-line definition for every one of the 63 HarmCategory members (previously only 3 were defined). Add completeness tests asserting every enum member has a non-empty definition and that no definition key references an unknown category name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Introduce a coarse 22-pillar grouping layered over the fine-grained HarmCategory taxonomy. Pillars are a many-to-many rollup: a pillar expands to its member categories, and a category can belong to multiple pillars (e.g. HARASSMENT is in both CHILD_SAFETY and HARMFUL_CONTENT). - HarmCategoryPillar StrEnum (22 members) with an authored pillar->categories map and a derived category->pillars inverse. - HarmCategory.pillars() and HarmCategoryPillar.categories() accessors. - Every HarmCategory except OTHER is covered by at least one pillar, enforced by tests (count, coverage, inverse consistency, many-to-many). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve conflicts in 35 dataset loaders + 4 test files: keep both main's async fetch (_fetch_from_huggingface_async) and metadata-coverage infrastructure, and the overlay branch's harm-category standardization (alias_overrides, standardized categories, provenance metadata). Unify duplicate promptintel tags field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ty resolves imports against the minimum supported Python version (3.10), where enum.StrEnum is unavailable. Use a sys.version_info guard so ty statically selects the backport branch on 3.10, and remove now-redundant HarmCategory casts exposed by that narrowing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Supersedes #998. That PR introduced a shared
HarmCategorytaxonomy but did it by forcing every dataset onto standardized labels, which discards each dataset's own annotations. This PR keeps the shared taxonomy but treats it as an overlay: datasets are mapped onto the canonical categories for consistent cross-dataset querying, while their original row-level annotations are preserved so nothing is lost.How it works:
Canonical taxonomy. New
pyrit/models/harm_category.pydefinesHarmCategory(aStrEnumof 63 categories) plus a one-line definition for each inharm_category_definitions.yaml. The taxonomy version is a module constant read from the YAML (not a bogus enum member as in [DRAFT] [BREAKING] FEAT Refactor Harm Category as StrEnum #998). Relative to [DRAFT] [BREAKING] FEAT Refactor Harm Category as StrEnum #998 this adds 4 child-safety categories (SEXUALIZATION,GROOMING,CHILD_LEAKAGE,DANGEROUS_SITUATIONS) and drops the catch-allILLEGAL(now aliased toCOORDINATION_HARM).Central standardization with n:1 aliasing.
standardize_harm_categories(...)maps raw dataset labels to canonical enum names via a shared alias table (e.g. "violence"/"violent"/"physical harm" ->VIOLENT_CONTENT), supports one-to-many mappings, dedups centrally, and warns-then-falls-back-to-OTHERfor anything unknown. ~30 labels that recurred across multiple loaders are promoted into the shared table so loaders don't each redefine them.Provenance preserved. When a dataset supplies its own per-row category annotations, the loader keeps the original wording in seed
metadata(e.g.aegis_violated_categories,pku_categories) so the source labels stay searchable. Datasets whose native labels are not harm categories (e.g. attack-technique feeds, censorship topics) are tagged as unclear rather than force-fit.Pillar rollup layer. Adds
HarmCategoryPillar(22 coarse pillars) as a third rollup tier:pillar -> standardized categories -> original labels. It is intentionally many-to-many (6 child-safety categories belong to two pillars). This layer is not consumed by any query path yet; it is taxonomy scaffolding for later.Scope note: this PR is the dataset + taxonomy layer only. Plumbing harm categories through to attack results (and the standardized-vs-original query flag) is a separate stacked PR.
Behavioral change worth a careful look: loaders that previously emitted raw harm-category strings now emit standardized enum names on
Seed.harm_categories. Original values remain available in seed metadata, but anyone filtering seeds by the old raw strings should read the migration note here. Flagging for reviewers in case you want a[BREAKING]tag.Tests and Documentation
tests/unit/models/test_harm_category.py(43 tests): standardization/dedup/alias resolution, canonical vs override precedence, unknown ->OTHER+ warning, every category has a definition and no stray definition keys, and the full pillar layer (22-pillar count, coverage of all non-OTHERcategories, inverse round-trip consistency, many-to-many spot checks).test_forbidden_questions_dataset.py,test_sosbench_dataset.py,test_ccp_sensitive_prompts_dataset.py,test_multilingual_vulnerability_dataset.py) asserting standardized outputs and preserved provenance metadata. Test fixtures use synthetic prompts only (no gated/source prompt text).tests/unit/datasetsandtests/unit/modelssuites pass; ruff format, ruff check, and ty are clean on the changed scope.