Sample

../_images/sample.svg

Modeling noise variations across many runs of a single model

Background

Sampling-based wrappers compute a measure of model (i.e., epistemic) uncertainty by measuring the disagreement of the model across multiple runs on the same input. Intuitively, if a model is shown an input five times and responds with five very different answers, then this would be a strong indicator that the model is uncertain and should not be trusted.

Traditionally, neural networks are deterministic – running a deterministic neural network on the same input will result in the same output no matter how many times it is called – which makes such models incompatible with uncertainty estimation. This wrapper automatically converts any model to a new stochastic form, compatible with sampling and uncertainty estimation. Stochastic noise can be inserted into a neural network in a variety of ways – from the weights to the activations, as well as being trainable or fixed, or even applied over the input itself.

Usage

Wrapping your model with capsa_torch.sample

from torch import nn
from capsa_torch import sample

# Define your model
model = nn.Sequential(...)

# Specify which distribution to use when modifying the model
dist = sample.Bernoulli(0.1)

# Build a wrapper with this distribution and wrap!
wrapper = sample.Wrapper(n_samples=5, distribution=dist)
wrapped_model = wrapper(model)

# or in one line
wrapped_model = sample.Wrapper(n_samples=5, distribution=dist)(model)

Calling your wrapped model

# By default, your wrapped model returns a prediction
prediction = wrapped_model(input_batch)

# By using `return_risk=True` you can also automatically get uncertainty too!
prediction, uncertainty = wrapped_model(input_batch, return_risk=True)

Examples

API Reference

capsa_torch.sample

Wrapping your model with capsa_tf.sample

#Import the module
from capsa_tf import sample

#Define the wrapper and arguments
dist = sample.Bernoulli(p=0.05)
wrapper = sample.Wrapper(n_samples=5, distribution=dist)

#Wrap the model
model.call_default = wrapper(model.call_default)

Calling your wrapped model

# By default, your wrapped model returns a prediction
prediction = model(input_batch)

# By using `return_risk` you can also automatically get uncertainty too!
prediction, uncertainty = model(input_batch, return_risk=True)

Examples

API Reference

capsa_tf.sample