ApproximateMixture

class axtreme.distributions.mixture.ApproximateMixture(mixture_distribution: Categorical, component_distribution: Distribution, *, validate_args: bool | None = None, check_dtype: bool = True)

Bases: MixtureSameFamily

Mixture distributions where extreme caclulations are approximated.

Some distribution only support a limited range of quantiles (e.g \([.01, 99]\)) due to numerical issues (see “Details”). When calculation such as \(q=cdf(x)\) or \(x=icdf(q)\) fail outside this range the function then error. ApproximateMixture allows all x values ,and approximates the results for x values outside the supported range (see “Details” for approximation method).

Details:

Distribution Quantiles Bounds:

Some distributions (e.g TransformedDistribution) have bounds on the quantiles they can support. Calculations such as \(q=cdf(x)\) or \(x=icdf(q)\) will fail when q is outside of these bounds. Bounds exist because for sum distributions \(icdf(1)=inf\) or \(icdf(0)=-inf\). Eventually there comes a point for q value close to 0 or 1 where they lack the numerical precision to capture very small changes in q, and the very large values of x.

Approximation principles:

It is assumed we want to conservatively estimate (\(1-cdf(x)\)), the chance the a value will exceed some level \(x\). For example, x could represent the strength a structure is designed to withstand, and (\(1-cdf(x)\)) represents the chance of experiencing a force that will break the structure (e.g the risk). It is better to overestimate the risk (conservative), rather than underestimate it. In other words, if the \(cdf_est(x)\) over estimates the \(cdf_true(x)\), then the true risk of exceeding x is. e.g:

TLDR:

  • \(cdf_est(x) < cdf_true(x)\): produces conservative design (okay)

  • \(cdf_est(x) > cdf_true(x)\): BAD

Worked example:

  • \(cdf_est(x) = .5\) and \(cdf_true(x) = .4\)

  • If structure is designed to be \(x\) strong, then estimated number of failure is .5, true number of failures is .6.

  • Have underestimated the risk and designed a structure more likely to fail than we expect.

Approximate results.

ApproximateMixture provides exact results within the quantile bounds [finfo.eps, 1 - finfo.eps] (where finfo = torch.finfo(component_distribution.dtype)) For details regarding why this range is selected see _lower_bound_x and _upper_bound_x. Values smaller than finfo.eps are approximated according to _lower_bound_x. Values larger than 1 - finfo.eps are approximated according to _upper_bound_x. Note the range [finfo.eps, 1 - finfo.eps] is still used even if the component distribution supports a greater range.

Impact of Approximation:

As a result of the approximation, the cdf method can underestimate the analytical cdf value by up to finfo.eps. This values come from the following calculation, Similar behaviour occurs at the lower bound.

  • Consider one of the underlying distributions in the marginal: once x hits the upper bound

    • it will give q = 1.0-finfo.eps

    • in reality it could be as high as 1.0

    • so the q give is finfo.eps too small (at worst)

  • The marginal cdf is calcualted from the underlying distributions:

    • e.g \(q_marginal = w_1 * q_1 + ... + w_n * q_n\)

    • \(sum(w_i) = 1\)

  • In the worst case:

    • Underling distribution: q give is finfo.eps too small (at worst)

    • weight = 1

Note

It is worth ensuring that the ApproximateMixture distribution has suitable numeric precision for its intended use. See axtreme.distributions.utils.mixture_dtype for more details.

__init__(mixture_distribution: Categorical, component_distribution: Distribution, *, validate_args: bool | None = None, check_dtype: bool = True) → None

Initialise the ApproximateMixture.

Parameters:

  • mixture_distribution – torch.distributions.Categorical-like instance. Manages the probability of selecting component. The number of categories must match the rightmost batch dimension of the component_distribution. Must have either scalar batch_shape or batch_shape matching component_distribution.batch_shape[:-1]

  • component_distribution – torch.distributions.Distribution-likeinstance. Right-most batch dimension indexes component.

  • validate_args – Runs the default validation defined by torch

  • check_dtype – Check datatype of distributions match and inpput are at least as precise as the distribution.

Methods

__init__(mixture_distribution, ...[, ...])

Initialise the ApproximateMixture.

cdf(x)

Return the CDF.

entropy()

Returns entropy of distribution, batched over batch_shape.

enumerate_support([expand])

Returns tensor containing all values supported by a discrete distribution.

expand(batch_shape[, _instance])

Returns a new distribution instance (or populates an existing instance provided by a derived class) with batch dimensions expanded to batch_shape.

icdf(value)

Returns the inverse cumulative density/mass function evaluated at value.

log_prob(x)

Calculate the log prob (e.g log(pdf)).

perplexity()

Returns perplexity of distribution, batched over batch_shape.

rsample([sample_shape])

Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.

sample([sample_shape])

Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.

sample_n(n)

Generates n samples or n batches of samples if the distribution parameters are batched.

set_default_validate_args(value)

Sets whether validation is enabled or disabled.

Attributes

arg_constraints

batch_shape

Returns the shape over which parameters are batched.

component_distribution

event_shape

Returns the shape of a single sample (without batching).

has_enumerate_support

has_rsample

mean

Returns the mean of the distribution.

mixture_distribution

mode

Returns the mode of the distribution.

stddev

Returns the standard deviation of the distribution.

support

Returns a Constraint object representing this distribution's support.

variance

Returns the variance of the distribution.
cdf(x: Tensor) → Tensor

Return the CDF.

Identical to MixtureSameFamily implementation except for clamping.

Parameters:

x –

Values to calcuate the CDF for. Must be broadcastable with the ApproximateMixture.batch_shape. E.g

  • self.component_distribution.batch_shape = (2,5) (last dimension is the components that are combined to make a single Mixture distribution)

  • self.batch_shape = (2,)

  • x.shape = (10) This will fail as it is not broadcastable

  • x.shape = (10,1) This will pass as it is broadcastable

Returns:

-1], self.batch_shape)

Return type:

Tensor of shape (x.shape[

log_prob(x: Tensor) → Tensor

Calculate the log prob (e.g log(pdf)).

Parameters:

x –

Values to calcuate the log prob for. Must be broadcastable with the ApproximateMixture.batch_shape. E.g

  • self.component_distribution.batch_shape = (2,5) (last dimension is the components that are combined to make a single Mixture distribution)

  • self.batch_shape = (2,)

  • x.shape = (10) This will fail as it is not broadcastable

  • x.shape = (10,1) This will pass as it is broadcastable

Returns:

-1], self.batch_shape)

Return type:

Tensor of shape (x.shape[