SimulationPointResults

class axtreme.evaluation.SimulationPointResults(metric_names: list[str], means: ndarray[tuple[int], dtype[float64]], cov: ndarray[tuple[int, int], dtype[float64]] | None)

Bases: object

For a point that has been simulation, stores the mean(s) and covariance(s).

Parameters:

• means (NDArray[np.float64]) – array of the mean paramter/metric estimates at a X point.

• cov (NDArray[np.float64]|None) – covarianc matrix with uncertainty distibution of the metric estimates. - This can be None if the error is unknown

• parameter_names – list of names. Gives the index where relevant data is stored

Design rational:

• The primary purpose of this is to define the interface between the Runner which generate simulation results, and Metric which reports the required parts of the results for AX to then use.

• The intent is to :

  • Explicitally define the interface information between the two (rather than use a dict)

  • Prevent Runner needing to know about specific structure of metric.

  • Prevent Metric needing to know the stucture of Runner

  • Esentially it mean the translatioin logic between these two components is contained in one discrete unit/object

This is generated (in the Runner) when the simulation is evaluated for a specific Trial. It is attached to the Trail.metadata, and later read by Metric.fetch_trial_data.

Parameters:

• metric_name (-) – The name of the metric that these results are relevant to

• mean (-) – The mean estimate of the metric for this particular trial

• sem (-) – Standard Error Measure, as defined here

Background: For documenation on what can do into this object

• Ax tutorial on defining custom metrics

• Ax definition

• For the possible things might want to contruct in the end

Todo

Revist if there is a better abstraction for this. Detail in Github issue #31.

__init__(metric_names: list[str], means: ndarray[tuple[int], dtype[float64]], cov: ndarray[tuple[int, int], dtype[float64]] | None) → None

Methods

__init__(metric_names, means, cov)
metric_data(metric_name)	Construct the 'Metric data-related columns' as defined in ax.Data.

Attributes

metric_names
means
cov

metric_data(metric_name: str) → dict[str, float | None]

Construct the ‘Metric data-related columns’ as defined in ax.Data.

This consists of:

• “mean”: mean estimate of this parameter

• “sem”: as defined here

cov: ndarray[tuple[int, int], dtype[float64]] | None

means: ndarray[tuple[int], dtype[float64]]

metric_names: list[str]