Record

record ¶

Build the machine-readable provenance record of a population catalogue.

A catalogue without a record of how it was produced cannot be published: the seed, the configuration and the code that made it are not recoverable from the samples. Every record this package writes is assembled here, so the two persistence paths cannot describe the same run in two different ways.

Record layout, version :data:PROVENANCE_SCHEMA_VERSION::

schema_version  the version of this layout
created_utc     when the file was written, ISO 8601 in UTC
tool            the package name, version, checkout state and writer
catalogue       source type, row count, column names in output order, format
run             what the run actually did: name, seed, how the seed was chosen
origin          where the samples came from, keyed by ``kind``

run is the authority on the seed and catalogue on the row count; origin.configuration is the configuration those values were resolved from, with defaults filled in. Keeping the two roles apart is deliberate: a declared configuration and an effective run are different facts, and a reader that cannot tell them apart cannot tell an override from a default.

Classes¶

EngineDescription `dataclass` ¶

EngineDescription(name: str, version: str | None = None, config_hash: str | None = None, run_record: str | None = None)

Identity of the engine that produced a catalogue.

Attributes:

Name	Type	Description
`name`	`str`	Name of the engine.
`version`	`str \| None`	Version of the engine, when it reports one.
`config_hash`	`str \| None`	Digest of the engine's own configuration.
`run_record`	`str \| None`	Pointer to the engine's own run record, such as a URL or path.

Methods:¶

payload ¶

payload() -> dict[str, Any]

Return the engine block as it appears in a record.

Returns:

Type	Description
`dict[str, Any]`	Mapping with the engine name, version, config hash and run-record pointer.

GraphConfigResolution `dataclass` ¶

GraphConfigResolution(config: dict[str, Any], unresolved_nodes: tuple[str, ...])

A parameter graph with its unstated defaults made explicit.

Attributes:

Name	Type	Description
`config`	`dict[str, Any]`	The graph config with callable defaults filled into `arguments`.
`unresolved_nodes`	`tuple[str, ...]`	Names of nodes whose defaults could not be resolved.

Functions:¶

git_source_state `cached` ¶

git_source_state() -> dict[str, Any] | None

Return the commit and dirty flag of the checkout the package runs from.

The state is read next to the installed package rather than from the working directory, so an unrelated repository the process happens to be started in is never reported as the source of the code.

Returns:

Type	Description
`dict[str, Any] \| None`	Mapping with the `commit` hash and a `dirty` flag, or `None` when
`dict[str, Any] \| None`	the package is not running from a git checkout.

source_code_provenance ¶

source_code_provenance() -> dict[str, Any]

Return the block identifying the code that produced a catalogue.

Returns:

Type	Description
`dict[str, Any]`	Mapping with the tool name, the package version and the checkout state.

configuration_hash ¶

configuration_hash(payload: Any) -> str

Return a digest of a configuration payload.

The digest is order-sensitive. Graph node order fixes both the output column order and the order in which the samplers consume the random stream, so two configurations differing only in node order are different runs.

Parameters:

Name	Type	Description	Default
`payload`	`Any`	JSON-ready configuration payload.	required

Returns:

Type	Description
`str`	The digest, prefixed with the name of the algorithm.

resolve_graph_config_defaults ¶

resolve_graph_config_defaults(config: Mapping[str, Any]) -> GraphConfigResolution

Return a parameter graph with every unstated argument default filled in.

A record of only the values the caller wrote cannot reconstruct the run: the rest of the sampler's arguments come from its signature, and a later version of the package may choose them differently.

Parameters:

Name	Type	Description	Default
`config`	`Mapping[str, Any]`	Parameter graph mapping node names to their specifications.	required

Returns:

Type	Description
`GraphConfigResolution`	The resolved graph and the names of any nodes whose defaults could not
`GraphConfigResolution`	be filled in. The input is not modified.

run_metadata ¶

run_metadata(*, name: str | None, seed: int, seed_source: str) -> dict[str, Any]

Return the block describing what a sampling run actually did.

Parameters:

Name	Type	Description	Default
`name`	`str \| None`	Run identifier, or `None` when the caller has no name for it.	required
`seed`	`int`	The seed the run's generator was actually initialized with, which is the drawn value when the caller asked for no particular seed.	required
`seed_source`	`str`	One of :data:`SEED_SOURCES`.	required

Returns:

Type	Description
`dict[str, Any]`	Mapping with the run name, seed and the origin of the seed.

Raises:

Type	Description
`ValueError`	If `seed_source` is not one of :data:`SEED_SOURCES`.

graph_simulation_origin ¶

graph_simulation_origin(*, graph_config: Mapping[str, Any], configuration: Mapping[str, Any], preset: str | None = None, config_path: str | None = None) -> dict[str, Any]

Return the origin block for a catalogue sampled from a parameter graph.

Parameters:

Name	Type	Description	Default
`graph_config`	`Mapping[str, Any]`	Parameter graph the simulator was built from. Its unstated defaults are filled in and the result becomes the `parameters` block of the recorded configuration, so the graph is stored once.	required
`configuration`	`Mapping[str, Any]`	Resolved simulation configuration, JSON-ready, with every default filled in.	required
`preset`	`str \| None`	Name of the packaged preset the run came from, when it did.	`None`
`config_path`	`str \| None`	Path of the configuration file the run came from, when it did.	`None`

Returns:

Type	Description
`dict[str, Any]`	The origin block.

simulator_origin ¶

simulator_origin(*, component: str, configuration: Mapping[str, Any] | None = None) -> dict[str, Any]

Return the origin block for a simulator with no recoverable graph config.

Parameters:

Name	Type	Description	Default
`component`	`str`	Import path of the simulator class that produced the samples.	required
`configuration`	`Mapping[str, Any] \| None`	Whatever configuration the simulator can report, if any.	`None`

Returns:

Type	Description
`dict[str, Any]`	The origin block. A record with this kind names the code that produced
`dict[str, Any]`	the catalogue but does not describe a replayable run.

external_engine_origin ¶

external_engine_origin(*, engine: EngineDescription, input_path: str | None = None, fetch: Mapping[str, Any] | None = None, upstream: Mapping[str, Any] | None = None) -> dict[str, Any]

Return the origin block for a catalogue produced by another engine.

Parameters:

Name	Type	Description	Default
`engine`	`EngineDescription`	Identity of the producing engine, including a pointer to its own run record.	required
`input_path`	`str \| None`	Path or URL the catalogue was read from.	`None`
`fetch`	`Mapping[str, Any] \| None`	Fetch details for a remotely retrieved catalogue, as reported by :class:`~gwmock_pop.loaders.FilePopulationLoader`.	`None`
`upstream`	`Mapping[str, Any] \| None`	Provenance record found alongside the input catalogue.	`None`

Returns:

Type	Description
`dict[str, Any]`	The origin block.

converted_catalogue_origin ¶

converted_catalogue_origin(*, input_path: str, column_map: Mapping[str, str] | None = None, fetch: Mapping[str, Any] | None = None, upstream: Mapping[str, Any] | None = None) -> dict[str, Any]

Return the origin block for a catalogue derived from another catalogue.

Parameters:

Name	Type	Description	Default
`input_path`	`str`	Path or URL of the input catalogue.	required
`column_map`	`Mapping[str, str] \| None`	Column renaming applied on the way through.	`None`
`fetch`	`Mapping[str, Any] \| None`	Fetch details for a remotely retrieved input catalogue.	`None`
`upstream`	`Mapping[str, Any] \| None`	Provenance record found alongside the input catalogue, so a conversion extends the chain rather than truncating it.	`None`

Returns:

Type	Description
`dict[str, Any]`	The origin block.

build_provenance_record ¶

build_provenance_record(*, origin: Mapping[str, Any], source_type: str | None, parameter_names: Sequence[str], n_samples: int, file_format: str, writer: str, run: Mapping[str, Any] | None = None, created: datetime | None = None) -> dict[str, Any]

Assemble the provenance record of one catalogue.

Parameters:

Name	Type	Description	Default
`origin`	`Mapping[str, Any]`	Origin block from one of the `*_origin` builders in this module.	required
`source_type`	`str \| None`	Routing key of the catalogue, such as `"bbh"`.	required
`parameter_names`	`Sequence[str]`	Column names in the order they are written.	required
`n_samples`	`int`	Number of rows written.	required
`file_format`	`str`	Format the catalogue was written in.	required
`writer`	`str`	Import path of the function that wrote the file.	required
`run`	`Mapping[str, Any] \| None`	Block from :func:`run_metadata`, for catalogues this package sampled.	`None`
`created`	`datetime \| None`	Creation time. Defaults to now, in UTC.	`None`

Returns:

Type	Description
`dict[str, Any]`	The complete record, ready to be encoded and stored.

Record

record ¶

Classes¶

EngineDescription dataclass ¶

Methods:¶

payload ¶

GraphConfigResolution dataclass ¶

Functions:¶

git_source_state cached ¶

source_code_provenance ¶

configuration_hash ¶

resolve_graph_config_defaults ¶

run_metadata ¶

graph_simulation_origin ¶

simulator_origin ¶

external_engine_origin ¶

converted_catalogue_origin ¶

build_provenance_record ¶

EngineDescription `dataclass` ¶

GraphConfigResolution `dataclass` ¶

git_source_state `cached` ¶