API¶

This document describes the public API.

`ggga.minimize.Minimizer`	Configure the GGGA optimizer.
`ggga.minimize.OptimizationResult`	Results of one optimization run (multiple experiments).
`ggga.individual.Individual`	Parameters and result of a pending or completed experiment.
`ggga.space.Space`	Represents the parameter space inside which optimization is performed.
`ggga.surrogate_model.SurrogateModel`	interface - A regression model to predict the value of points.
`ggga.acquisition.AcquisitionStrategy`	A strategy to acquire new samples.
`ggga.outputs.OutputEventHandler`	Report progress and save results during optimization process.
`ggga.outputs.Output`	Control the output during optimization.

class mtrand.RandomState¶: see numpy.random.RandomState

Minimizer¶

class ggga.minimize.Minimizer(popsize=10, initial=10, max_nevals=100, relscale_initial=0.3, relscale_attenuation=0.9, surrogate_model_class=<class 'ggga.gpr.SurrogateModelGPR'>, surrogate_model_args={}, acquisition_strategy=None, time_source=<built-in function time>, select_via_posterior=False, fmin_via_posterior=True, n_replacements=1)¶

Configure the GGGA optimizer.

Parameters

popsize (int, optional) – How many samples are taken per generation. Defaults to 10.
initial (int, optional) – How many initial samples should be acquired before model-guided acquisition takes over. Default: 10.
max_nevals (int, optional) – How many samples may be taken in total per optimization run. Defaults to 100.
relscale_initial (float, optional) – Standard deviation for creating new samples, as percentage of each paramter’s range. Defaults to 0.3.
relscale_attenuation (float, optional) – Factor by which the relscale is reduced per generation. Defaults to 0.9.
surrogate_model_class (type[SurrogateModel], optional) – The regression model to fit the response surface. Defaults to SurrogateModelGPR.
surrogate_model_args (dict, optional) – Extra arguments for the surrogate model.
acquisition_strategy (AcquisitionStrategy or None, optional) – How new samples are acquired. Defaults to MutationAcquisition with breadth=10.
select_via_posterior (bool, optional) – Whether the model prediction should be used as a fitness function when selecting which samples proceed to the next generation. If false, the objective’s observed value incl. noise is used. Defaults to False.
fmin_via_posterior (bool, optional) – Whether the model prediction is used to find the current best point during optimization. If false, the objective’s observed value incl. noise is used. Defaults to True.
n_replacements (int, optional) – How many random samples are suggested per generation. Usually, new samples are created by random mutations of existing samples.

await minimize(objective, *, space, rng, outputs=None, historic_individuals=())¶

Minimize the objective.

Parameters

objective (async objective(sample, rng) -> (value, cost))) – A function to calculate the objective value. The sample is a list with the same order as the params in the space. The value and cost are floats. The cost is merely informative.
space (Space) – The parameter space inside which the objective is optimized.
rng (RandomState) –
outputs (Optional[OutputEventHandler]) – Controls what information is printed during optimization. Can e.g. be used to save evaluations into a CSV file. Defaults to Output.
historic_individuals (Iterable[Individual]) – Previous evaluations of the same objective/space that should be incorporated into the model. Can be useful in order to benefit from previous minimization runs. Potential drawbacks include a biased model, and that the tuner slows down with additional samples. Constraints: all individual must be fully initialized, and declare the -1th generation.

Return type

OptimizationResult

with_setting(*, popsize=None, initial=None, max_nevals=None, relscale_initial=None, relscale_attenuation=None, surrogate_model_class=None, surrogate_model_args=None, acquisition_strategy=None, time_source=None, select_via_posterior=None, fmin_via_posterior=None, n_replacements=None)¶

Clone a Minimizer but override some attributes.

Return type: Minimizer

OptimizationResult¶

class ggga.minimize.OptimizationResult(*, all_individuals, all_models, duration)¶

Results of one optimization run (multiple experiments).

best_n(how_many)¶

Select the best evaluation results.

Return type: List[Individual]

all_individuals = None¶

all results

Type: list[Individual]

all_models = None¶

all models

Type: list[SurrogateModel]

best_individual = None¶

best result

Type: Individual

duration = None¶

total duration

Type: float

fmin¶

Best observed value.

Return type: float

model¶

Final model.

Return type: SurrogateModel

xs¶

Input variables of all evaluations.

Return type: ndarray

ys¶

Output variables of all evaluations.

Return type: ndarray

Individual¶

class ggga.individual.Individual(sample, *, observation=None, gen=None, expected_improvement=None, prediction=None, cost=None)¶

Parameters and result of a pending or completed experiment.

Many fields are write-once.

Parameters: sample (list) – The input variables at which the experiment shall be evaluated.

is_fully_initialized()¶

Check whether all write-once fields have been provided. If true, the object is immutable.

Return type: bool

cost¶

The observed cost. Write-once.

Return type: float

expected_improvement¶

The expected improvement before the Individual was evaluated. Write-once.

Return type: float

gen¶

The generation in which the Individual was evaluated. Write-once.

Return type: int

observation¶

The observed value. Write-once.

Return type: float

prediction¶

The predicted value. Write-once.

Return type: float

Space¶

class ggga.space.Space(*params, constraints=None, constrained_bounds_suggestions=None)¶

Represents the parameter space inside which optimization is performed.

Parameters

*params (Real or Integer) –

The parameters that make up the space.

class Real(name, lo, hi)¶

class Integer(name, lo, hi, *, scale?)¶

SurrogateModel¶

class ggga.surrogate_model.SurrogateModel¶

interface - A regression model to predict the value of points. This is used to guide the acquisition of new samples.

Subclasses must override estimate() and predict_transformed_a().

Known implementations:

`ggga.gpr.SurrogateModelGPR`(kernel, …)
`ggga.knn.SurrogateModelKNN`(estimator, *, …)

abstractmethod classmethod estimate(mat_x, vec_y, *, space, rng, prior, **kwargs)¶

Fit a new model to the given data.

Parameters

mat_x (ndarray) –
vec_y (ndarray) –
space (Space) –
rng (RandomState) –
prior (Optional[SurrogateModel]) –
**kwargs – Extra arguments for the concrete SurrogateModel class.

Return type

SurrogateModel

length_scales()¶

Length scales for the paramters, estimated by the fitted model.

Longer length scales indicate less relevant parameters. By default, the scale is 1.

Return type: ndarray

predict(vec_x, *, return_std=True)¶

Return type

Tuple[float, Optional[float]]

Returns

mean (float)
std (float or None)

predict_a(mat_x, *, return_std=True)¶

Return type

Tuple[ndarray, Optional[ndarray]]

Returns

vec_mean (np.ndarray)
vec_std (np.ndarray or None)

abstractmethod predict_transformed_a(mat_x_transformed, *, return_std=True)¶

Predict multiple values at the same time.

Return type

Tuple[ndarray, Optional[ndarray]]

Returns

vec_mean (np.ndarray)
vec_std (np.ndarray or None)

class ggga.gpr.SurrogateModelGPR¶

class ggga.knn.SurrogateModelKNN¶

AcquisitionStrategy¶

class ggga.acquisition.AcquisitionStrategy¶

A strategy to acquire new samples.

Implementations:

`ChainedAcquisition`(*strategies)	Perform multi-stage acquisition.
`HedgedAcquisition`(*strategies)	Randomly assign parent individuals to a sub-strategy.
`RandomReplacementAcquisition`(n_replacements)	Replace bad individuals with random samples.
`MutationAcquisition`(breadth)	Randomly mutate each parent to create new samples in their neighborhood.
`RandomWalkAcquisition`(breadth, steps[, …])
`GradientAcquisition`(breadth)	Use gradient optimization to find optimal samples.

abstractmethod acquire(population, *, model, relscale, rng, fmin, space)¶

Acquire new individuals.

Parameters

population (Iterable[Individual]) – Previous population/parents.
model (SurrogateModel) – Current model of the utility landscape.
relscale (ndarray) – suggested normalized standard deviation for mutating individuals.
fmin (float) – Current best observed value (useful for EI)
space (Space) –
rng (RandomState) –

Returns

A finite sequence of individuals that may be selected for evaluation.

Return type

typing.Iterable[Individual]

class ggga.acquisition.ChainedAcquisition(*strategies)¶

Perform multi-stage acquisition.

Each stage operates on the results of the previous stage.

class ggga.acquisition.HedgedAcquisition(*strategies)¶: Randomly assign parent individuals to a sub-strategy.

class ggga.acquisition.RandomReplacementAcquisition(n_replacements, hedge_via_prediction=True, relscale_initial=0.1, subacquisition=NOTHING)¶

Replace bad individuals with random samples.

This is not suitable as a primary acquisition strategy.

Can be used to guard against premature convergence by replacing worst estimated offspring with random individuals. This is not completely random, but is controlled by Metropolis sampling.

class ggga.acquisition.MutationAcquisition(breadth)¶: Randomly mutate each parent to create new samples in their neighborhood.

class ggga.acquisition.RandomWalkAcquisition(breadth, steps, relscale_attenuation=0.5)¶

class ggga.acquisition.GradientAcquisition(breadth)¶: Use gradient optimization to find optimal samples.

OutputEventHandler¶

class ggga.outputs.OutputEventHandler¶

Report progress and save results during optimization process. (interface)

event_acquisition_completed(*, duration)¶

Called when new samples have been acquired.

Return type: None

event_evaluations_completed(individuals, *, duration)¶

Called when evaluations of a generation have completed.

Return type: None

event_model_trained(generation, model, *, duration)¶

Called when a new model has been trained.

Return type: None

event_new_generation(gen, *, relscale)¶

Called when a new generation is started.

Return type: None

Output¶

class ggga.outputs.Output(*, space, evaluation_csv_file=None, model_file=None, log_file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)¶

Control the output during optimization. Default implementation of OutputEventHandler.

Parameters

space (Space) – The parameter space.
evaluation_csv_file (typing.TextIO, optional) – If present, all evaluations are recorded in this file.
model_file (typing.TextIO, optional) – If present, metadata of the models is recorded in this file, using a JSON-per-line format.
log_file (typing.TextIO, optional) – Where to write human-readable output. Defaults to sys.stdout. If set to None, output is suppressed.

add(logger)¶

Return type: None

event_acquisition_completed(*, duration)¶

Called when new samples have been acquired.

Return type: None

event_evaluations_completed(individuals, *, duration)¶

Called when evaluations of a generation have completed.

Return type: None

event_model_trained(generation, model, *, duration)¶

Called when a new model has been trained.

Return type: None

acquisition_durations = None¶

time spent per acquisition

Type: list[float]

evaluation_durations = None¶

time spent per evaluation

Type: list[float]

training_durations = None¶

time spent per training/fitting

Type: list[float]

PartialDependence¶

class ggga.visualization.PartialDependence(*, model, space, rng, resolution=40, quality=250)¶

Visualize and analyze individual contributions of each parameter.

Parameters

model (SurrogateModel) –
space (Space) –
rng (RandomState) –
resolution (int) – How many samples are used along one parameter. Default: 20.
quality (int) – How many samples are used along all other parameters to get a precise estimate of the average value. Default: 250.

along_one_dimension(dim)¶

Calculate contributions along one dimension.

Return type

Tuple[ndarray, ndarray, ndarray, ndarray, ndarray]

Returns

xs – Sample locations along this dimension.
ys_mean – Mean response at the samples.
ys_min – Minimal/optimal response at the samples.
ys_mean_std – Mean model uncertainty.
ys_min_std – Model uncertainty at the minimum.

along_two_dimensions(dim_1, dim_2)¶

Calculate contributions along two dimensions.

Return type

Tuple[ndarray, ndarray, ndarray]

Returns

xs_1 – Sample locations along dim_1
xs_2 – Sample locations along dim_2.
ys – Mean response at the samples.

plot_grid(x_observed, y_observed, *, x_min=None, dual_style=None, single_mean_style=None, single_min_style=None, subplot_size=2, progress_cb=<function _default_progress_cb>)¶

Plot a visualization of parameter influences.

Parameters

x_observed (ndarray) –
y_observed (ndarray) –
x_min (list or None) – Location of the best sample. Defaults to the sample that minimizes y_observed.
dual_style (Optional[DualDependenceStyle]) –
single_mean_style (Optional[SingleDependenceStyle]) –
single_min_style (Optional[SingleDependenceStyle]) –
subplot_size (float) – How large each plot in the grid of all parameters should be. The whole figure will have size (n_params × subplot_size)².
progress_cb ((dim_1_name, dim_2_name?) -> None) – Called prior to rendering each sub-plot with the names of the parameters in the sub-plot. The dim_2_name is only provided for interaction plots.

Returns

(fig, axes) – The plotted figure.

Return type

tuple

DualDependenceStyle¶

class ggga.visualization.DualDependenceStyle(cmap='viridis_r', contour_args=None, contour_filled=True, contour_filled_args=None, contour_levels=10, contour_lines=False, contour_lines_args=None, contour_scatter_args=None, contour_xmin_scatter_args=None, scatter_args=None, xmin_scatter_args=None)¶

Control the appearance of the parameter interaction visualization.

The interaction (contour) plot has four layers that can be configured separately:

filled contour plot: get_contour_filled_args()
contour lines: get_contour_line_args()
scatter plot of all samples: get_scatter_args()
“scatter” plot of the best sample: get_xmin_scatter_args()

Parameters

cmap (str) – The colour map used for the filled contour plot. Defaults to ‘viridis_r’.
contour_args (Optional[dict]) – Extra arguments for the contour plot (both filled and lines).
contour_filled (bool) – Whether filled contours are drawn. Either this or countour_lines should be True. Defaults to True.
contour_filled_args (Optional[dict]) – Extra arguments for the filled contour plot, overrides contour_args.
contour_levels (int) – Defaults to 10.
contour_lines (bool) – Whether contour lines are drawn. Either this or contour_filled should be True.
contour_lines_args (Optional[dict]) – Extra arguments the line contour plot, overrides contour_args.
contour_scatter_args (Optional[dict]) – Extra arguments for the scatter plot of all samples.
xmin_scatter_args (Optional[dict]) – Extra arguments to override the scatter plot appearance of the best point.

get_contour_filled_args()¶

Filled contour plot arguments.

locator: None, cmap: cmap, alpha: 0.8
contour_args
contour_filled_args

Return type: dict

get_contour_line_args()¶

Contour line plot arguments.

locator: None, colors: ‘k’, linewidths: 1
contour_args
contour_lines_args

Return type: dict

get_scatter_args()¶

Scatter plot arguments.

c: ‘k’, s: 10, lw: 0
scatter_args

Return type: dict

get_xmin_scatter_args()¶

“Scatter” plot of the best sample arguments.

get_scatter_args()
c: ‘r’
xmin_scatter_args

Return type: dict

SingleDependenceStyle¶

class ggga.visualization.SingleDependenceStyle(color, show=True, show_err=True, err_alpha=0.15, args={})¶

Control the appearance of the single dependence plot.

This style be provided separately for the mean and optimal responses.

Parameters

color (str) – Color for the response line/region.
show (bool) – Whether the response shall be shown.
show_err (bool) – Whether the uncertainty around the response shall be shown.
err_alpha (float) – Alpha value for uncertainty region shading.
args (dict) – Extra arguments for the response line.

benchmark_functions¶

A collection of optimization benchmark functions that can be used via the example runner. Some of them do not have a fixed number of parameters and can be implicitly used as any n-dimensional version. Read their docstrings for more information on behaviour, bounds, and optima.

ggga.benchmark_functions.goldstein_price(x_1, x_2)¶

ggga.benchmark_functions.easom(x_1, x_2, *, amplitude?)¶

ggga.benchmark_functions.himmelblau(x_1, x_2)¶

ggga.benchmark_functions.rastrigin(*xs, amplitude?)¶

ggga.benchmark_functions.rosenbrock(*xs)¶

ggga.benchmark_functions.sphere(*xs)¶

ggga.benchmark_functions.onemax(*xs)¶

ggga.benchmark_functions.trap(*xs, p_well?)¶