Feat: SKOS import support alongside OWL (design proposal — feedback wanted)

[libei]

Status: Design proposal. Not yet implemented. Comments welcome — especially on the "Open questions" section at the bottom.

Goal

Many real-world ontologies mix OWL (formal classes and properties) with SKOS (human-readable labels, taxonomies, cross-vocabulary mappings). The current gm import-owl importer recognizes SKOS only partially: skos:altLabel and skos:prefLabel populate synonyms, and everything else (skos:definition, skos:broader, skos:scopeNote, skos:notation, skos:exactMatch, standalone skos:Concept resources, etc.) is silently dropped.

This issue proposes first-class SKOS support in the same gm import-owl command, so users importing FIBO, SNOMED excerpts, library vocabularies, and other hybrid OWL+SKOS sources don't lose descriptive metadata or taxonomy structure.

Audit of current behavior

On main (src/bigquery_ontology/owl_importer.py):

• skos:prefLabel, skos:altLabel → synonyms (lines 213–218)
• All other SKOS predicates → silently dropped (no code path reads them)
• Standalone skos:Concept (no owl:Class typing) → not imported (entity iteration is g.subjects(RDF.type, OWL.Class) only, line 350)
• Design doc docs/ontology/owl-import.md:168-170 claims literal annotations are preserved as annotations: {iri: value}, but the implementation only covers a fixed OWL allowlist — SKOS literals fall through unhandled
• No tests or fixtures cover SKOS input

Guiding principle: informational by default

SKOS and OWL make different kinds of claims, and the importer should preserve the distinction:

• OWL makes formal claims. rdfs:subClassOf means every instance of the child is an instance of the parent. Drives inheritance, keys, substitutability.
• SKOS makes informational claims. skos:broader means "this is a narrower topic" — explicitly not subsumption per the W3C SKOS Primer. Drives documentation, browsing, search.

Consequences for the importer:

1. OWL constructs flow into structural fields (extends, from/to, keys, properties).
2. SKOS graph-shaped predicates (skos:broader, skos:related, skos:*Match) flow into abstract relationships — edges declared in the ontology but not expected to be bound to BigQuery tables.
3. SKOS literal predicates (skos:definition, skos:notation, skos:scopeNote, etc.) flow into annotations.
4. Nothing from SKOS flows into extends or description. The ontology never claims inheritance or human-readable descriptions that the source author didn't assert via OWL/RDFS core vocabulary.

Proposed model extensions

Two small, symmetric additions:

1. abstract: bool = False on Entity and Relationship

An abstract entity is declared but not bound to a BigQuery table. Primary key not required. gm validate does not demand one; gm bind rejects bindings that target it.

An abstract relationship is declared but not bound. It describes the graph without pretending to be backed by rows.

Rule: a concrete relationship cannot have an abstract endpoint (nothing to bind). An abstract relationship may have any combination of endpoints.

2. Relaxed name uniqueness for abstract relationships

Today, relationship names must be globally unique in the ontology (ontology_loader.py:80, rule 1 in ontology.md). This prevents emitting multiple skos_broader edges with different endpoints.

Proposed change, scoped narrowly:

• Concrete relationships: (name,) must be unique. Unchanged.
• Abstract relationships: (name, from, to) must be unique. A single predicate like skos_broader can repeat across endpoint pairs.

Why this scoping is safe:

• compile_graph only emits DDL for elements that have bindings. Abstract relationships have no bindings, so they never reach the DDL emitter. Edge-table naming in CREATE PROPERTY GRAPH is unaffected.
• Binding resolution targets concrete relationships, whose names are still globally unique.
• GQL queries over the generated property graph reference concrete relationship names, still unique.

In short, the relaxation only affects the informational/documentation surface of the ontology. Structural, bindable, and DDL-facing surfaces are untouched.

Naming convention

Provenance is encoded in names so that a reader can tell at a glance which elements are SKOS-derived (informational) versus OWL-derived (structural):

• Pure SKOS entity (resource typed only as skos:Concept): name prefixed skos_ → skos_Banking, skos_RetailBanking.
• Mixed OWL+SKOS entity (same resource typed as both): name unprefixed → Account. OWL won the structure, so it wins the name.
• SKOS relationship (always abstract, derived from skos:broader, skos:related, skos:*Match): name prefixed skos_ → skos_broader, skos_related, skos_exactMatch.
• OWL relationship: unprefixed. Unchanged.
• Annotation keys: keep the skos: colon-style prefix (e.g., skos:definition, skos:notation). This matches the existing owl: convention in annotations and preserves RDF provenance for potential round-trip tooling.

The rule reduces to: skos_ name prefix = "this element is purely SKOS-sourced and informational." Unprefixed = has OWL structure (or is a core ontology element) and can be trusted as formal.

A note on the dual prefix form. SKOS provenance appears as skos_ in names and skos: in annotation keys. The split isn't stylistic — it reflects the identifier/metadata boundary: names flow into YAML parse keys, BigQuery labels, GQL syntax, and Python/SQL code, where colons are unsafe or already syntactic. Annotation keys are free-form map keys where colons parse cleanly and where owl: already sets the convention. Any tool bridging RDF and code hits the same boundary (JSON-LD contexts, rdflib's CURIE-to-attribute translation, protobuf-to-Python name generation). Learn the split once and it stays out of the way.

Mapping table (SKOS additions)

SKOS construct	Ontology equivalent	Notes
skos:Concept (no owl:Class)	Abstract entity with skos_ name prefix	Keys not required
owl:Class + skos:Concept	Concrete entity, unprefixed	OWL provides structure
skos:ConceptScheme	Ontology-level annotation
skos:prefLabel	synonyms (when ≠ name)	Label data
skos:altLabel, skos:hiddenLabel	synonyms	Label data
skos:definition	Annotation skos:definition	Not a label; preserved with provenance
skos:broader, skos:narrower	Abstract relationship skos_broader (normalized: narrower → broader)
skos:related	Abstract relationship skos_related
skos:exactMatch, skos:closeMatch, skos:broadMatch, skos:narrowMatch, skos:relatedMatch	Abstract relationship skos_exactMatch etc. if target is an imported entity; annotation if external IRI
skos:notation, skos:scopeNote, skos:example, skos:historyNote, skos:editorialNote, skos:changeNote	Annotation (prefix preserved)
skos:inScheme, skos:topConceptOf	Annotation (prefix preserved)

Rule of thumb: the predicate's object is another imported entity → relationship; otherwise → annotation.

Label and description handling

SKOS never populates description directly. The rules are one-to-one per predicate, no fallback chain:

• rdfs:label → description (unchanged from today)
• rdfs:comment → description (appended) (unchanged)
• skos:prefLabel → synonyms if different from the entity name; otherwise elided
• skos:altLabel, skos:hiddenLabel → synonyms
• skos:definition, skos:scopeNote, etc. → annotations with skos: prefix

Consequence: a pure-SKOS file produces entities with empty description. The definition is available in annotations.skos:definition if anyone needs it. This is intentional — promoting a skos:definition into description is the same kind of structural smuggling we rejected for skos:broader → extends.

Multilingual labels are selected by --language (default en). Non-selected languages are preserved as language-suffixed annotations (e.g., skos:prefLabel@fr: Chat).

Examples

Example 1 — Mixed OWL + SKOS

Input:

:Account a owl:Class, skos:Concept ;
    rdfs:label "Account" ;
    skos:altLabel "Acct"@en ;
    skos:definition "A record of financial transactions."@en ;
    skos:related :Ledger ;
    skos:exactMatch <http://fibo.org/ontology/FBC/Account> ;
    owl:hasKey ( :account_id ) .

:Ledger a owl:Class ;
    rdfs:label "Ledger" ;
    owl:hasKey ( :ledger_id ) .

Output:

entities:
  - name: Account
    description: Account
    synonyms: [Acct]
    keys:
      primary: [account_id]
    properties:
      - name: account_id
        type: string
    annotations:
      skos:definition: A record of financial transactions.
      skos:exactMatch: http://fibo.org/ontology/FBC/Account

  - name: Ledger
    description: Ledger
    keys:
      primary: [ledger_id]
    properties:
      - name: ledger_id
        type: string

relationships:
  - name: skos_related
    abstract: true
    from: Account
    to: Ledger

Concrete entities (OWL built them, names unprefixed), abstract skos_related relationship (SKOS is informational), literal annotations for the definition and cross-vocab match.

Example 2 — Pure SKOS taxonomy

Input:

:Banking a skos:Concept ;
    skos:prefLabel "Banking"@en ;
    skos:definition "Activities of financial institutions."@en .

:RetailBanking a skos:Concept ;
    skos:prefLabel "Retail Banking"@en ;
    skos:altLabel "Consumer Banking"@en ;
    skos:broader :Banking ;
    skos:notation "RB" .

:InvestmentBanking a skos:Concept ;
    skos:prefLabel "Investment Banking"@en ;
    skos:broader :Banking .

Output:

entities:
  - name: skos_Banking
    abstract: true
    annotations:
      skos:definition: Activities of financial institutions.

  - name: skos_InvestmentBanking
    abstract: true

  - name: skos_RetailBanking
    abstract: true
    synonyms: [Consumer Banking]
    annotations:
      skos:notation: RB

relationships:
  - name: skos_broader
    abstract: true
    from: skos_InvestmentBanking
    to: skos_Banking

  - name: skos_broader
    abstract: true
    from: skos_RetailBanking
    to: skos_Banking

All entities are prefixed (pure SKOS, informational), all relationships are skos_broader with different endpoints (legal under the scoped uniqueness relaxation), and description is empty on every entity — the SKOS source offered no rdfs:label or rdfs:comment. A stderr hint suggests the user consider representing the taxonomy as a dimension column instead of entity types, but the import proceeds regardless.

Example 3 — Cross-kind reference

A concrete OWL class pointing to a pure-SKOS concept via skos:broader:

:Account a owl:Class ;
    rdfs:label "Account" ;
    skos:broader :FinancialProduct ;
    owl:hasKey ( :account_id ) .

:FinancialProduct a skos:Concept ;
    skos:prefLabel "Financial Product"@en .

Output:

entities:
  - name: Account
    description: Account
    keys:
      primary: [account_id]
    properties:
      - name: account_id
        type: string

  - name: skos_FinancialProduct
    abstract: true

relationships:
  - name: skos_broader
    abstract: true
    from: Account
    to: skos_FinancialProduct

Concrete entity, abstract entity, abstract relationship. The name prefix travels with the element wherever it's referenced, so provenance is visible at every site.

CLI additions

Only one new flag:

• --language <tag> — language tag used to select labels and notes (default en).

Everything else is default behavior. Pure SKOS concepts are imported as abstract entities, SKOS graph predicates become abstract relationships, SKOS literals become annotations. No flag gates SKOS support on or off, and no flag converts SKOS claims into structural claims — consistent with the "informational by default" principle. If you want formal subsumption, write rdfs:subClassOf in your source.

The command itself stays gm import-owl — same input pipeline, same namespace filter.

Drop summary additions

The existing stderr drop summary gets new lines for:

• SKOS predicates seen and mapped to annotations
• Labels and notes discarded by --language selection
• skos:*Match targets outside included namespaces, preserved as annotations
• Taxonomy-shaped imports (all-abstract entities) with a hint to consider dimension columns

Fix for an existing bug

docs/ontology/owl-import.md:168-170 states that unknown literal-valued predicates should be preserved as annotations: {iri: value}. The code does not implement this generically — only a fixed OWL allowlist is handled. As part of this work, a generic literal-annotation pass will be added, closing the silent-drop gap for any RDF predicate (SKOS, Dublin Core, custom).

Open questions — feedback wanted

1. Is "informational by default" the right stance? Or should skos:broader map to extends by default, since hybrid ontology authors sometimes conflate the two? (Proposal leans conservative: SKOS never maps to structural fields.)

2. Is abstract: true the right name and shape? Alternatives considered: bindable: false, informational: true, splitting entities into two types. The abstract name parallels OO semantics most closely.

3. Relaxing relationship uniqueness to (name, from, to) for abstract relationships only is required to emit multiple skos_broader edges without synthetic naming. Analysis above argues this doesn't affect DDL, binding, or GQL. Anyone see a downstream assumption this would break?

4. Is the skos_ name prefix the right choice? Alternative: no prefix, trust abstract: true alone to signal informational status. Pro-prefix: greppability, collision safety, provenance visible at every reference site. Pro-no-prefix: cleaner names in GQL queries and bindings, less visual noise, redundant with abstract: true.

   Why there isn't a --preserve-skos-prefix flag. A flag feels cheap but fragments the tool's output: teams on the same project end up with different conventions in committed YAML, and downstream tooling (diffs, audit scripts, code review) has to handle both. We'd rather pick one default based on feedback here than punt the decision to every user. If this thread surfaces a roughly even split between prefix and no-prefix camps, a flag becomes justified and we'll add it; until then, post-import renaming via sd/sed covers the escape hatch.

5. Should skos:prefLabel ever populate description when rdfs:label is absent? Current proposal says no: pure-SKOS entities get empty description and the label lives in synonyms. A one-level fallback would be simpler than the earlier four-way proposal and would prevent awkward-looking empty fields. Open to either.

6. Multilingual handling currently proposes one selected language populates standard fields and others go to language-suffixed annotations (e.g., skos:prefLabel@fr). Alternative: all languages in synonyms with @lang tags preserved. Preference?

7. Scope: should Dublin Core (dc:title, dc:description, dcterms:creator, etc.) be handled in the same pass, since it shows up in many of the same files? Or deferred to a follow-up?

8. Naming of CLI flag. --language — open to bikeshedding.

9. Should we ever support an opt-in flag to promote SKOS to structural semantics? The current proposal says no: SKOS is informational, full stop. A future --promote-skos-broader flag was considered and dropped for simplicity. Revisit if real users request it.

10. Should there be a flag to opt out of SKOS import entirely? For users who want OWL-only behavior, --no-skos could suppress all SKOS processing. Dropped from the initial proposal to keep surface minimal.

Please comment if you have opinions or real-world SKOS+OWL ontologies you'd like to see handled.

[caohy1988]

Good design. The informational-by-default stance and the (abstract, name-prefix) symmetry are the right call. Most of my comments are on the integration risk between the two packages — the proposal is self-consistent inside bigquery_ontology, but landing it without a matching change in the SDK consumer layer creates a few sharp edges.

Split: what goes where

Clean rule: bigquery_ontology knows what SKOS means. bigquery_agent_analytics knows which elements are runnable. The ontology package describes, the SDK materializes.

bigquery_ontology (the knowledge layer)

Change	File	Notes
Add abstract: bool = Field(default=False) on Entity and Relationship	ontology_models.py	Default False = no break for existing ontologies
Keys not required on abstract entities	ontology_models.py + ontology_loader.py	Current "primary required on entities" rule needs an if not entity.abstract guard
Relax uniqueness: concrete = (name,), abstract = (name, from, to)	ontology_loader.py:79-80, 119-130	Split _check_unique_names or add a sibling. Keep cross-kind (entity-vs-relationship) collision check
Reject bindings targeting abstract entities/relationships	binding_loader.py:219-252	New check in _check_binding_targets_exist with clear error
Skip abstract elements in DDL emission	graph_ddl_compiler.py	Already excluded by binding-iteration, add explicit guard as defense-in-depth
Skip abstract elements in scaffold output	scaffold.py:93, 99	No binding stubs for things that can't be bound
SKOS import logic (the major work)	owl_importer.py	Everything in the mapping table, plus the generic literal-annotation pass to fix the existing docs/ontology/owl-import.md:168-170 bug
--language <tag> CLI flag	cli.py:630 in import_owl_command	Wire through to import_owl()
Tests + docs	tests/ and docs/ontology/owl-import.md	Fixtures for pure SKOS, mixed OWL+SKOS, cross-kind, multilingual, duplicate-name abstract relationships

bigquery_agent_analytics (the runtime layer)

The SDK doesn't need to know what SKOS is. It needs to know: abstract elements have no physical backing — skip them in every runtime code path.

Change	File(s)	Notes
Single choke-point filter in the Ontology → RuntimeGraphSpec adapter	runtime_spec.py:~229	Filter ontology.entities / ontology.relationships to exclude abstract=True once, at the adapter boundary. Every downstream consumer then sees a concrete-only view.
GraphSpec (SDK's runtime model) rejects abstract=True	ontology_models.py (SDK's copy) validator	Invariant: every element in GraphSpec is bindable and has data. Even hand-constructed specs refuse abstract elements.
Clean error for pure-SKOS-only ontology	adapter	If, after filtering, concrete entity count is 0: "this ontology has no concrete entities; the SDK needs at least one bindable entity." Different from the existing min_length=1 — entities existed, none were concrete.
spec.relationships[0].name default	cli.py:884	Free once filter runs upstream
ttl_importer.py pass-through	ttl_importer.py:813	Surface --language if the SDK wraps import_owl with its own flags

Why the choke-point filter is worth the discipline

There are 10+ sites in the SDK that iterate spec.entities / spec.relationships — ontology_graph.py:195, 753; ontology_materializer.py:156, 465, 471, 486, 498, 678, 679; ontology_orchestrator.py:118, 121, 130, 132, 214, 228, 337, 338, 369, 370; ontology_property_graph.py:162, 272, 281; plus validators in the SDK's own ontology_models.py.

If filtering is done per-site, every one of these is a potential bug where an abstract element leaks into a runtime artifact (DDL, JSON schema for AI.GENERATE, materialization view SQL). If filtering is done once in the Ontology→GraphSpec adapter, every downstream site is provably correct by construction.

This is the single biggest integration risk and it maps directly to Q3. The uniqueness relaxation is safe only if consumers that build {r.name: r for r in ...} maps filter to concrete first. Without the choke-point filter, abstract skos_broader with multiple endpoints will silently collapse under rel_map[name] in any consumer that hasn't been updated. With the choke-point filter, it's impossible — the map only ever sees concrete elements.

Recommended sequence

1. Ontology model changes — abstract field, relaxed uniqueness, binding rejection, scaffold skip. Self-contained, merges without touching SKOS or the SDK.
2. SDK choke-point filter + GraphSpec abstract-rejection before any SKOS import code lands. This protects the SDK against the uniqueness relaxation before abstract data can reach it.
3. SKOS import implementation in owl_importer.py with tests.
4. Generic literal-annotation pass (fixes the existing OWL allowlist bug) — can ship with Revise README for clarity and updated link #3.
5. Multilingual + drop-summary + taxonomy hint — polish.

Step 2 is the gate. Without it, step 3 can silently break step 1's uniqueness assumptions inside consumer code.

Answers to the open questions

• Q1 Informational default: correct. Promoting skos:broader → extends is exactly the silent-semantics drift SKOS users rely on being absent. Keep it out.
• Q2 abstract: true name: fine. bindable: false is more literal but abstract reads well in YAML and parallels OO semantics.
• Q3 Uniqueness relaxation: see above. Safe iff the SDK filters to concrete at the adapter boundary. Mandate that to avoid silent regressions.
• Q4 skos_ name prefix: keep it. Greppability, provenance at every reference site, zero collision risk. abstract: true alone is invisible at call sites in GQL and YAML references.
• Q5 skos:prefLabel → description fallback: no fallback. Empty description is honest. Pure-SKOS entities are documentation-layer elements; let the annotation carry the definition.
• Q6 Multilingual: --language selects one, others as language-suffixed annotations. Cleaner for downstream tooling than @lang-tagged synonyms.
• Q7 Dublin Core: defer. The generic literal-annotation pass (the existing-bug fix) handles DC mechanically without designing for it. If users need structured DC handling later, that's a separate issue.
• Q8 --language name: fine.
• Q9 Promote flag, Q10 opt-out flag: don't ship. Ontology authors wanting formal subsumption write rdfs:subClassOf; opt-out is grep -v skos: upstream. Either can be added later if real users ask.

Risk items not covered in the proposal

• Pure-SKOS ontologies + the SDK: an ontology that's all-abstract is legal at the ontology layer but useless to the SDK. Need an intentional clean error, not a cryptic "GraphSpec has no entities" from deep in the adapter. Surface it at the adapter boundary with a message that names the cause.
• Abstract element collision with concrete name: the cross-kind collision check already covers entity-vs-relationship. Worth an explicit test that a concrete Account and an abstract skos_Account are both allowed (they should be — different names by construction), and that a concrete Account and an abstract Account are rejected (same name).
• Round-trip: if someone imports SKOS, edits the YAML, and re-exports, do annotations with skos: colon keys round-trip through YAML cleanly? Worth a test — YAML allows colons in keys but some YAML tools quote aggressively, and the skos_ name vs skos: annotation split means two different key shapes. Not a blocker, just a "verify once" item.

Happy to review PRs on this. Most excited about step 2 landing cleanly — the adapter filter is the piece that makes the whole thing sound.

[caohy1988]

One additional integration-risk note to pin explicitly:

The SDK “choke-point filter” needs to run before any name-index maps are built, not just before emitting GraphSpec. On current main, both packages build {name: obj} maps very early:

• bigquery_ontology.binding_loader._validate_binding() builds entity_map = {e.name: e ...} / rel_map = {r.name: r ...} up front.
• bigquery_agent_analytics.runtime_spec.graph_spec_from_ontology_binding() does the same with ont_entity_map / ont_rel_map.

So the sequencing guard should be slightly stricter than “land the SDK filter before SKOS import code”:

1. In bigquery_ontology, uniqueness relaxation and “bindings may not target abstract elements” should land together.
2. In bigquery_agent_analytics, the abstract-element filter must happen before ont_entity_map / ont_rel_map are constructed in the adapter, otherwise duplicate abstract names can still collapse by last-write-wins before the runtime boundary has a chance to protect anything.

Same core safety argument as the choke-point filter, just one level earlier in the call graph.