Cache manifest schema¶

A feature cache records what it was derived from in a dependency manifest (medh5.cache/1), stored as canonical JSON in the cache file's manifest dataset. This schema validates it. It is part of the separately versioned task and cache contract, not of the format.

import json
import medh5.cache

schema = json.loads(medh5.cache.schema_text())
schema["properties"]["level"]

A manifest that fails this schema is T401; the checksums and source pins it records are checked by:

medh5 cache validate reports.medh5cache

The tables below are generated from the schema itself at build time.

The document¶

Property	Type	Required	Constraints	Description
`schema`		yes	`medh5.cache/1`
`level`		yes	`event`, `patient`	event: one feature per source event version (or its document). patient: one per task row, pinned to its cutoff and selected versions.
`encoder`	`object`	yes	no other keys
`preprocessing`	`object`	yes		The configuration that, with the encoder, determines every output.
`output`	`object`	yes	no other keys
`task_fingerprint`	`digest` or `null`	when `level` is `patient`
`selection`			`strict_prospective`, `latest_provable`, `null`
`fitted_on`	`null` or `object`			For learned preprocessing (normalisation, code bins, vocabularies): the training split it was fitted on.
`entries`	array of `entry`	yes

The document itself is closed: additionalProperties is false, so a key the schema does not name is a schema failure rather than a silently ignored extension.

preprocessing is an open map: the keys are yours. Where a value schema is given it still applies; where additionalProperties is omitted entirely, the values are unconstrained too.

Definitions¶

6 shared definitions, referenced by $ref above and by each other.

digest¶

Type	Constraints
`string`	pattern `^(sha256\\|sha512\\|blake2b):[0-9a-f]+$`

int64¶

Type	Constraints
`integer`	minimum `-9223372036854775808`; maximum `9223372036854775807`

patient_entry¶

Property	Type	Required	Constraints
`row_id`	`string`	yes	minLength `1`
`row_fingerprint`	`digest`	yes
`cutoff_us`	`int64`	yes
`event_versions`	array of `string`	yes

event_entry¶

Property	Type	Required	Constraints
`event_id`	`string`	yes	minLength `1`
`row_id`	`null`
`row_fingerprint`	`null`
`cutoff_us`	`null`
`event_versions`	`null`

source¶

Property	Type	Required	Constraints
`uri`	`string`	yes	minLength `1`
`sample_key`	`string` or `null`
`content_id`	`digest`	yes

entry¶

Property	Type	Required	Constraints	Description
`entry_id`	`string`	yes	pattern `^[A-Za-z0-9_.@:-]{1,128}$`	The task id syntax, so a patient-level entry may be named by its row id.
`sources`	array of `source`	yes	minItems `1`
`event_id`	`string` or `null`
`document_id`	`string` or `null`
`row_id`	`string` or `null`
`cutoff_us`	`int64` or `null`
`event_versions`	`array` or `null`
`row_fingerprint`	`digest` or `null`			Patient level: the fingerprint of the row the feature was built for (task-cache-1 §3.2).
`digest`	`digest`	yes

The schema itself¶

Published verbatim as medh5-cache-1.schema.json at https://medh5.readthedocs.io/en/stable/reference/schemas/medh5-cache-1.schema.json. Its $id, https://github.com/XwK-P/medh5/schemas/medh5-cache-1.schema.json, identifies the schema; it is not where the file is served. The format engine embeds the same file, and medh5.cache.schema_text() returns it.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://github.com/XwK-P/medh5/schemas/medh5-cache-1.schema.json",
  "title": "MEDH5 feature cache manifest, version 1 (medh5.cache/1)",
  "description": "The dependency manifest of a derived feature cache, stored as canonical JSON in the `manifest` dataset of a cache file whose root carries `medh5_companion = \"medh5.cache/1\"` and `manifest_digest`. Each entry's payload is the dataset `entries/<entry_id>`, carrying its MEDH5 §13.1 digest. A cache is derived: it never redefines or invalidates a source. See docs/spec/task-cache-1.md.",
  "type": "object",
  "required": ["schema", "level", "encoder", "preprocessing", "output", "entries"],
  "additionalProperties": false,
  "properties": {
    "schema": { "const": "medh5.cache/1" },
    "level": { "enum": ["event", "patient"],
      "description": "event: one feature per source event version (or its document). patient: one per task row, pinned to its cutoff and selected versions." },
    "encoder": {
      "type": "object",
      "required": ["name", "revision"],
      "additionalProperties": false,
      "properties": {
        "name": { "type": "string", "minLength": 1 },
        "revision": { "type": "string", "minLength": 1,
          "description": "An immutable revision: a content hash or a commit, never a moving tag." },
        "tokenizer": {
          "anyOf": [
            { "type": "null" },
            { "type": "object", "required": ["name", "revision"], "additionalProperties": false,
              "properties": { "name": { "type": "string", "minLength": 1 }, "revision": { "type": "string", "minLength": 1 } } }
          ]
        }
      }
    },
    "preprocessing": { "type": "object", "description": "The configuration that, with the encoder, determines every output." },
    "output": {
      "type": "object",
      "required": ["dtype", "shape"],
      "additionalProperties": false,
      "properties": {
        "dtype": { "enum": ["float16", "float32", "float64", "int32", "int64", "uint8"] },
        "shape": { "type": "array", "items": { "type": "integer", "minimum": 0 } },
        "pooling": { "type": ["string", "null"] },
        "chunking": { "type": ["string", "null"] }
      }
    },
    "task_fingerprint": { "anyOf": [{ "$ref": "#/$defs/digest" }, { "type": "null" }] },
    "selection": { "enum": ["strict_prospective", "latest_provable", null] },
    "fitted_on": {
      "description": "For learned preprocessing (normalisation, code bins, vocabularies): the training split it was fitted on.",
      "anyOf": [
        { "type": "null" },
        { "type": "object", "required": ["task_fingerprint", "partition", "subjects_digest"], "additionalProperties": false,
          "properties": {
            "task_fingerprint": { "$ref": "#/$defs/digest" },
            "set_id": { "type": ["string", "null"] },
            "partition": { "type": "string", "minLength": 1 },
            "subjects_digest": { "$ref": "#/$defs/digest" }
          } }
      ]
    },
    "entries": { "type": "array", "items": { "$ref": "#/$defs/entry" } }
  },
  "allOf": [
    {
      "description": "A patient-level cache names the task it was built for, and every entry pins its row (by id and fingerprint), the row's cutoff and the versions it selected.",
      "if": { "properties": { "level": { "const": "patient" } }, "required": ["level"] },
      "then": {
        "required": ["task_fingerprint"],
        "properties": {
          "task_fingerprint": { "$ref": "#/$defs/digest" },
          "entries": { "items": { "$ref": "#/$defs/patient_entry" } }
        }
      }
    },
    {
      "description": "An event-level entry names the event version it encodes, and no row: it is free of any cutoff.",
      "if": { "properties": { "level": { "const": "event" } }, "required": ["level"] },
      "then": { "properties": { "entries": { "items": { "$ref": "#/$defs/event_entry" } } } }
    }
  ],

  "$defs": {
    "digest": { "type": "string", "pattern": "^(sha256|sha512|blake2b):[0-9a-f]+$" },
    "int64": { "type": "integer", "minimum": -9223372036854775808, "maximum": 9223372036854775807 },
    "patient_entry": {
      "required": ["row_id", "row_fingerprint", "cutoff_us", "event_versions"],
      "properties": {
        "row_id": { "type": "string", "minLength": 1 },
        "row_fingerprint": { "$ref": "#/$defs/digest" },
        "cutoff_us": { "$ref": "#/$defs/int64" },
        "event_versions": { "type": "array", "items": { "type": "string" } }
      }
    },
    "event_entry": {
      "required": ["event_id"],
      "properties": {
        "event_id": { "type": "string", "minLength": 1 },
        "row_id": { "type": "null" },
        "row_fingerprint": { "type": "null" },
        "cutoff_us": { "type": "null" },
        "event_versions": { "type": "null" }
      }
    },
    "source": {
      "type": "object",
      "required": ["uri", "content_id"],
      "additionalProperties": false,
      "properties": {
        "uri": { "type": "string", "minLength": 1 },
        "sample_key": { "type": ["string", "null"] },
        "content_id": { "$ref": "#/$defs/digest" }
      }
    },
    "entry": {
      "type": "object",
      "required": ["entry_id", "sources", "digest"],
      "additionalProperties": false,
      "properties": {
        "entry_id": { "type": "string", "pattern": "^[A-Za-z0-9_.@:-]{1,128}$",
          "description": "The task id syntax, so a patient-level entry may be named by its row id." },
        "sources": { "type": "array", "items": { "$ref": "#/$defs/source" }, "minItems": 1 },
        "event_id": { "type": ["string", "null"] },
        "document_id": { "type": ["string", "null"] },
        "row_id": { "type": ["string", "null"] },
        "cutoff_us": { "anyOf": [{ "$ref": "#/$defs/int64" }, { "type": "null" }] },
        "event_versions": { "type": ["array", "null"], "items": { "type": "string" } },
        "row_fingerprint": { "anyOf": [{ "$ref": "#/$defs/digest" }, { "type": "null" }],
          "description": "Patient level: the fingerprint of the row the feature was built for (task-cache-1 §3.2)." },
        "digest": { "$ref": "#/$defs/digest" }
      }
    }
  }
}

Task and cache contract §7 — caches, normatively.
Task manifest schema — the other schema of the contract.
Train on clinical tasks — building and validating a cache from Python.