Edit on GitHub

Choosing Summary Metrics

The choice of distance measure in the storm summary space is not always obvious. One way to choose is to pick some parametric distance (e.g. weighted Euclidean distance) and then choose the parameters by try to optimise for the performance of the simulated traces.

If we have a score for the similarity of two traces, we can try to minimise the expected score of the simulated traces against the true historical traces.

To do this, use

HistoricalStormTraceSimulation.find_best_distance — Function
find_best_distance(D::Type{<:Metric},x₀,history; lowerbounds, upperbounds, optim_kwargs=(), kwargs...)

Use optim to find the best distance based on score_method scoring.

Arguments:

  • D the type of distance (e.g. WeightedEuclidean). Note this is the constructor for the type, not an instance!
  • x₀ initial parameters for the distance to start optimisation.
  • history the storm history.
  • lowerbounds the lower bounds for the optimisation, defaults to fill(-Inf,length(x₀)).
  • upperbounds the upper bounds for the optimisation, defaults to fill(Inf,length(x₀)).
  • optim_kwargs: Key word arguments to be passed to Optim.
  • kwargs additional key word arguments, similar to score_method, but not including summarymetric, as this is specified by the optimisation.

Note:

To use WeightedPeriodicEuclidean, you first need to select the periods p, and then create a constructor to fix these, e.g.

julia> myperiods = [Inf,Inf,360.0]
3-element Vector{Float64}:
  Inf
  Inf
 360.0

julia> myperiodicdistance(w) = WeightedPeriodicEuclidean(myperiods,w)
myperiodicdistance (generic function with 1 method)

Then pass myperiodicdistance as the argument D to find_best_distance.

source
HistoricalStormTraceSimulation.expected_score — Function
expected_score(history::StormHistory; samplemethod=1:50, rescalemethod, summarymetric::Metric=Euclidean(), interpolation_method=LinearInterpolation, tracescore::TraceScore)

Compute the expected score.

Arguments:

  • history, samplemethod, rescalemethod, summarymetric, interpolation_method: see sampletraces.
  • tracescore: Score for comparing traces.
source
HistoricalStormTraceSimulation.conditional_expected_score — Function
conditional_expected_score(summary,trace,history,sampler,rescalemethod,interpolation_method,tracescore)

Compute the conditional expected score of a given historical trace. A different method will be used if the trace sampler uses a uniform over the closest m points method (i.e. if the sampler has a UnitRange for its samplemethod).

Arguments

  • summary: The summary of a historical trace.
  • trace: The corresponding trace.
  • sampler: A TraceSampler.
  • history, rescalemethod, interpolation_method: see sampletraces
  • tracescore: Score for comparing traces.
source

Trace Scoring

Trace scores can be specified as a subtype of TraceScore. A functor should then be defined taking in two traces, which returns a measure of similarity between the two, with 0 being identical.

Provided is MarginalTraceScore which allows the user to define weighted sums of scores (usually metrics) applied to each variable.

HistoricalStormTraceSimulation.MarginalTraceScore — Type
MarginalTraceScore(metrics::NTuple{N,Metric},weights=ones(length(metrics)))

Construct a scoring method for traces based on a weighted some of marginal metrics.

Arguments

  • metrics: Should be a tuple of p metrics, where p is the number of variables in the trace of interest (not including time).
  • weights: A vector of weights to be used (defaults to vector of ones).

Use

Given a MarginalTraceScore called mscore.

(mscore::MarginalTraceScore)(t1::StormTrace,t2::StormTrace)

will compute the marginal trace score between two traces t1 and t2.

source

Convenience functions for checking compatibility are also provided:

Implementing new trace scores

New methods can be implemented by analogy to the implementation for MarginalTraceScore:

struct MarginalTraceScore{T,S} <: TraceScore
    metrics::T
    weights::S
    function MarginalTraceScore(metrics::NTuple{N,Metric},weights=ones(length(metrics))) where {N}
        length(metrics) == length(weights) || throw(DimensionMismatch("weights and metrics should be same length."))
        new{typeof(metrics),typeof(weights)}(metrics,weights)
    end
end

function (mscore::MarginalTraceScore)(t1::StormTrace,t2::StormTrace)
    checkcompatible(t1,t2)
    checkcompatible(mscore,t1)
    score = 0.0
    for i in eachindex(mscore.metrics)
        @views score += mscore.weights[i]*mscore.metrics[i](t1.value[:,i],t2.value[:,i])
    end
    return score
end

Periodic trace scores

If one of the marginal traces should be periodic, use the following metric as opposed to PeriodicEuclidean, as that will only work for arrays of a fixed size.