EvaluationFunction

class axtreme.evaluation.EvaluationFunction(simulator: Simulator, output_dist: rv_continuous, parameter_order: list[str], n_simulations_per_point: int = 20)

Bases: object

Class for evaluating a simulation at a point, and fits a distribution to the results.

Take ax.Parameter, runs the simulator at that point, and packages the result into the format later required for unpacking in the metrics.

__init__(simulator: Simulator, output_dist: rv_continuous, parameter_order: list[str], n_simulations_per_point: int = 20) → None

Initializes the EvaluationFunction.

Parameters:

  • simulator – Conforming to the simulator in

  • output_dist – Distribution that should be used to fit the response. This is the distribution of the noise in the simulation, at a given x,

  • n_simulations_per_point – The number of simulations to run at each point.

  • parameter_order –

    Order of features that the simulator expects.

    Note

    This is a temporary measure to check that the order isn’t ever shuffled.

    Todo

    Got through ax and check is order is always respected, so we can remove this

Methods

__init__(simulator, output_dist, parameter_order)

Initializes the EvaluationFunction.

post_process_simulation_output(y)

Post process the simulation output be fitting a the distribution to the results.

run_simulator(x)

Runs the simulator at a point and returns the results.
post_process_simulation_output(y: ndarray[tuple[int, int, int], dtype[float64]]) → SimulationPointResults

Post process the simulation output be fitting a the distribution to the results.

Parameters:

y –

The output of the simulation with shape (n_simulations_per_points,)

Note

This is the output of the simulator, at a single point and a single output dimension.

Returns:

A SimulationPointResults object with the mean and covariance of the fitted distribution.

run_simulator(x: ndarray[tuple[int, int], dtype[float64]]) → ndarray[tuple[int, int, int], dtype[float64]]

Runs the simulator at a point and returns the results.

Parameters:

x – The points at which to run the simulator with shape (n_points, n_input_dims)

Returns:

The results of the simulator with shape (n_points, n_simulations_per_point, n_output_dims)

Raises:

AssertionError – If the shape of the output of self.simulator is not as expected.