learners

Module of learners used to determine what parameters to try next given previous cost evaluations.

Each learner is created and controlled by a controller.

class mloop.learners.DifferentialEvolutionLearner(first_params=None, trust_region=None, evolution_strategy='best1', population_size=15, mutation_scale=(0.5, 1), cross_over_probability=0.7, restart_tolerance=0.01, **kwargs)

Bases: mloop.learners.Learner, threading.Thread

Adaption of the differential evolution algorithm in scipy.

Parameters:

• params_out_queue (queue) – Queue for parameters sent to controller.
• costs_in_queue (queue) – Queue for costs for gaussian process. This must be tuple
• end_event (event) – Event to trigger end of learner.

Keyword Arguments:  

• first_params (Optional [array]) – The first parameters to test. If None will just randomly sample the initial condition. Default None.
• trust_region (Optional [float or array]) – The trust region defines the maximum distance the learner will travel from the current best set of parameters. If None, the learner will search everywhere. If a float, this number must be between 0 and 1 and defines maximum distance the learner will venture as a percentage of the boundaries. If it is an array, it must have the same size as the number of parameters and the numbers define the maximum absolute distance that can be moved along each direction.
• evolution_strategy (Optional [string]) – the differential evolution strategy to use, options are ‘best1’, ‘best2’, ‘rand1’ and ‘rand2’. The default is ‘best1’.
• population_size (Optional [int]) – multiplier proportional to the number of parameters in a generation. The generation population is set to population_size * parameter_num. Default 15.
• mutation_scale (Optional [tuple]) – The mutation scale when picking new points. Otherwise known as differential weight. When provided as a tuple (min,max) a mutation constant is picked randomly in the interval. Default (0.5,1.0).
• cross_over_probability (Optional [float]) – The recombination constand or crossover probability, the probability a new points will be added to the population.
• restart_tolerance (Optional [float]) – when the current population have a spread less than the initial tolerance, namely stdev(curr_pop) < restart_tolerance stdev(init_pop), it is likely the population is now in a minima, and so the search is started again.

has_trust_region

Whether the learner has a trust region.

Type:bool

num_population_members

The number of parameters in a generation.

Type:int

params_generations

History of the parameters generations. A list of all the parameters in the population, for each generation created.

Type:list

costs_generations

History of the costs generations. A list of all the costs in the population, for each generation created.

Type:list

init_std

The initial standard deviation in costs of the population. Calculated after sampling (or resampling) the initial population.

Type:float

curr_std

The current standard deviation in costs of the population. Calculated after sampling each generation.

Type:float

OUT_TYPE = 'differential_evolution'

_best1(index)

Use best parameters and two others to generate mutation.

Parameters:index (int) – Index of member to mutate.

_best2(index)

Use best parameters and four others to generate mutation.

Parameters:index (int) – Index of member to mutate.

_rand1(index)

Use three random parameters to generate mutation.

Parameters:index (int) – Index of member to mutate.

_rand2(index)

Use five random parameters to generate mutation.

Parameters:index (int) – Index of member to mutate.

generate_population()

Sample a new random set of variables

mutate(index)

Mutate the parameters at index.

Parameters:index (int) – Index of the point to be mutated.

next_generation()

Evolve the population by a single generation

random_index_sample(index, num_picks)

Randomly select a num_picks of indexes, without index.

Parameters:

• index (int) – The index that is not included
• num_picks (int) – The number of picks.

run()

Runs the Differential Evolution Learner.

save_generation()

Save history of generations.

update_archive()

Update the archive.

class mloop.learners.GaussianProcessLearner(length_scale=None, length_scale_bounds=None, update_hyperparameters=True, cost_has_noise=True, noise_level=None, noise_level_bounds=None, **kwargs)

Bases: mloop.learners.MachineLearner, multiprocessing.context.Process

Gaussian process learner.

Generates new parameters based on a gaussian process fitted to all previous data.

Parameters:

• params_out_queue (queue) – Queue for parameters sent to controller.
• costs_in_queue (queue) – Queue for costs for gaussian process. This must be tuple.
• end_event (event) – Event to trigger end of learner.

Keyword Arguments:  

• length_scale (Optional [array]) – The initial guess for length scale(s) of the gaussian process. The array can either of size one or the number of parameters or None. If it is size one, it is assumed that all of the correlation lengths are the same. If it is an array with length equal to the number of the parameters then all the parameters have their own independent length scale. If it is set to None and a learner archive from a Gaussian process optimization is provided for training_filename, then it will be set to the value recorded for length_scale in that learner archive. If set to None but training_filename does not specify a learner archive from a Guassian process optimization, then it is assumed that all of the length scales should be independent and they are all given an initial value of equal to one tenth of their allowed range. Default None.
• length_scale_bounds (Optional [array]) – The limits on the fitted length scale values, specified as a single pair of numbers e.g. [min, max], or a list of pairs of numbers, e.g. [[min_0, max_0], …, [min_N, max_N]]. This only has an effect if update_hyperparameters is set to True. If one pair is provided, the same limits will be used for all length scales. Alternatively one pair of [min, max] can be provided for each length scale. For example, possible valid values include [1e-5, 1e5] and [[1e-2, 1e2], [5, 5], [1.6e-4, 1e3]] for optimizations with three parameters. If set to None, then the length scale will be bounded to be between 0.001 and 10 times the allowed range for each parameter.
• update_hyperparameters (Optional [bool]) – Whether the length scales and noise estimate should be updated when new data is provided. Default True.
• cost_has_noise (Optional [bool]) – If True the learner assumes there is common additive white noise that corrupts the costs provided. This noise is assumed to be on top of the uncertainty in the costs (if it is provided). If False, it is assumed that there is no noise in the cost (or if uncertainties are provided no extra noise beyond the uncertainty). Default True.
• noise_level (Optional [float]) – The initial guess for the noise level (variance, not standard deviation) in the costs. This is only used if cost_has_noise is True. If it is set to None and a learner archive from a Gaussian process optimization is provided for training_filename, then it will be set to the value recorded for noise_level in that learner archive. If set to None but training_filename does not specify a learner archive from a Guassian process optimization, then it will automatically be set to the variance of the training data costs.
• noise_level_bounds (Optional [array]) – The limits on the fitted noise_level values, specified as a single pair of numbers [min, max]. This only has an effect if update_hyperparameters and cost_has_noise are both set to True. If set to None, the value [1e-5 * var, 1e5 * var] will be used where var is the variance of the training data costs. Default None.
• training_filename (Optional [str]) – The name of a learner archive from a previous optimization from which to extract past results for use in the current optimization. If None, no past results will be used. Default None.
• trust_region (Optional [float or array]) – The trust region defines the maximum distance the learner will travel from the current best set of parameters. If None, the learner will search everywhere. If a float, this number must be between 0 and 1 and defines maximum distance the learner will venture as a percentage of the boundaries. If it is an array, it must have the same size as the number of parameters and the numbers define the maximum absolute distance that can be moved along each direction.
• default_bad_cost (Optional [float]) – If a run is reported as bad and default_bad_cost is provided, the cost for the bad run is set to this default value. If default_bad_cost is None, then the worst cost received is set to all the bad runs. Default None.
• default_bad_uncertainty (Optional [float]) – If a run is reported as bad and default_bad_uncertainty is provided, the uncertainty for the bad run is set to this default value. If default_bad_uncertainty is None, then the uncertainty is set to a tenth of the best to worst cost range. Default None.
• minimum_uncertainty (Optional [float]) – The minimum uncertainty associated with provided costs. Must be above zero to avoid fitting errors. Default 1e-8.
• predict_global_minima_at_end (Optional [bool]) – If True attempts to find the global minima when the learner is ended. Does not if False. Default True.

all_params

Array containing all parameters sent to learner.

Type:array

all_costs

Array containing all costs sent to learner.

Type:array

all_uncers

Array containing all uncertainties sent to learner.

Type:array

scaled_costs

Array contaning all the costs scaled to have zero mean and a standard deviation of 1. Needed for training the gaussian process.

Type:array

bad_run_indexs

list of indexes to all runs that were marked as bad.

Type:list

best_cost

Minimum received cost, updated during execution.

Type:float

best_params

Parameters of best run. (reference to element in params array).

Type:array

best_index

index of the best cost and params.

Type:int

worst_cost

Maximum received cost, updated during execution.

Type:float

worst_index

index to run with worst cost.

Type:int

cost_range

Difference between worst_cost and best_cost.

Type:float

generation_num

Number of sets of parameters to generate each generation. Set to 4.

Type:int

length_scale_history

List of length scales found after each fit.

Type:list

noise_level_history

List of noise levels found after each fit.

Type:list

fit_count

Counter for the number of times the gaussian process has been fit.

Type:int

cost_count

Counter for the number of costs, parameters and uncertainties added to learner.

Type:int

params_count

Counter for the number of parameters asked to be evaluated by the learner.

Type:int

gaussian_process

Gaussian process that is fitted to data and used to make predictions

Type:GaussianProcessRegressor

cost_scaler

Scaler used to normalize the provided costs.

Type:StandardScaler

params_scaler

Scaler used to normalize the provided parameters.

Type:StandardScaler

has_trust_region

Whether the learner has a trust region.

Type:bool

OUT_TYPE = 'gaussian_process'

_check_length_scale_bounds()

Ensure self.length_scale_bounds has a valid value, otherwise raise a ValueError.

_check_noise_level_bounds()

Ensure self.noise_level has a valid value, otherwise raise a ValueError.

_transform_length_scale_bounds(length_scale_bounds, inverse=False)

Transform length scale bounds to or from scaled units.

This method functions similarly to self.transform_length_scales(), except that it transforms the bounds for the length scales. The same scalings used for the length scales themselves are applied here to the lower and upper bounds. To transform from real/unscaled units to scaled units, call this method with inverse set to False. To perform the inverse transformation, namely to transform length scale bounds from scaled units to real/unscaled units, call this method with inverse set to True.

The output array will have a separate scaled min/max value pair for each parameter length scale. In other words, the output will be an array with two columns (one for min values and one for max values) and one row for each parameter length scale. This will be the case even if length_scale_bounds consists of a single min/max value pair because the scalings are generally different for different parameters.

Note that although length_scale_bounds can be a 1D array with only two entries (a single min/max pair shared by all parameters), this method always returns a 2D array with a separate min/max pair for each parameter because the scaling factors aren’t generally the same for all of the parameters. This implies that transforming a 1D array then performing the inverse transformation will yield a 2D array of identical min/max pairs rather than the original 1D array.

Parameters:

• length_scale_bounds (array) – The bounds for the Gaussian process’s length scales which should be transformed to or from scaled units. This can either be (a) a 1D array with two entries of the form [min, max] or (b) a 2D array with two columns (min and max values respectively) and one row for each parameter length scale.
• inverse (bool) – This argument controls whether the forward or inverse transformation is applied. If False, then the forward transformation is applied, which takes length_scale_bounds in real/unscaled units and transforms them to scaled units. If True then this method assumes that length_scale_bounds are in scaled units and transforms them into real/unscaled units. Default False.

Raises:

ValueError – A ValueError is raised if length_scale_bounds does not have an acceptable shape. The allowed shapes are (2,) (a single min/max pair shared by all parameters) or (self.num_params, 2) (a separate min/max pair for each parameter).

Returns:

The transformed length

scale bounds. These will be in scaled units if inverse is False or in real/unscaled units if inverse is True. Note that transformed_length_scale_bounds will always be a 2D array of shape (self.num_params, 2) even if length_scale_bounds was a single pair of min/max values.

Return type:

transformed_length_scale_bounds (array)

_transform_length_scales(length_scales, inverse=False)

Transform length scales to or from scaled units.

This method uses self.params_scaler to transform length scales to/from scaled units. To transform from real/unscaled units to scaled units, call this method with inverse set to False. To perform the inverse transformation, namely to transform length scales from scaled units to real/unscaled units, call this method with inverse set to True.

Notably length scales should be scaled, but not offset, when they are transformed. For this reason, they should not simply be passed through self.params_scaler.transform() and instead should be passed through this method.

Although length_scales can be a fingle float, this method always returns a 1D array because the scaling factors aren’t generally the same for all of the parameters. This implies that transforming a float then performing the inverse transformation will yield a 1D array of identical entries rather than a single float.

Parameters:

• length_scales (float or array) – Length scale(s) for the Gaussian process which should be transformed to or from scaled units. Can be either a single float or a 1D array of length self.num_params.
• inverse (bool) – This argument controls whether the forward or inverse transformation is applied. If False, then the forward transformation is applied, which takes length_scales in real/unscaled units and transforms them to scaled units. If True then this method assumes that length_scales are in scaled units and transforms them into real/unscaled units. Default False.

Returns:

The transformed length scales.

These will be in scaled units if inverse is False or in real/unscaled units if inverse is True. Note that transformed_length_scales will be a 1D array even if length_scales was a single float.

Return type:

transformed_length_scales (array)

create_gaussian_process()

Create a Gaussian process.

find_global_minima()

Search for the global minima predicted by the Gaussian process.

This method will attempt to find the global minima predicted by the Gaussian process, but it is possible for it to become stuck in local minima of the predicted cost landscape.

This method does not return any values, but creates the attributes listed below.

predicted_best_parameters

The parameter values which are predicted to yield the best results, as a 1D array.

Type:array

predicted_best_cost

The predicted cost at the predicted_best_parameters, in a 1D 1-element array.

Type:array

predicted_best_uncertainty

The uncertainty of the predicted cost at predicted_best_parameters, in a 1D 1-element array.

Type:array

find_next_parameters()

Get the next parameters to test.

This method searches for the parameters expected to give the minimum biased cost, as predicted by the Gaussian process. The biased cost is not just the predicted cost, but a weighted sum of the predicted cost and the uncertainty in the predicted cost. See self.predict_biased_cost() for more information.

This method additionally increments self.params_count appropriately.

Returns:

The next parameter values to try, stored in a

1D array.

Return type:next_params (array)

fit_gaussian_process()

Fit the Gaussian process to the current data

predict_biased_cost(params, perform_scaling=True)

Predict the biased cost at the given parameters.

The biased cost is a weighted sum of the predicted cost and the uncertainty of the prediced cost. In particular, the bias function is:

biased_cost = cost_bias * pred_cost - uncer_bias * pred_uncer

Parameters:

• params (array) – A 1D array containing the values for each parameter. These should be in real/unscaled units if perform_scaling is True or they should be in scaled units if perform_scaling is False.
• perform_scaling (bool, optional) – Whether or not the parameters and biased costs should be scaled. If True then this method takes in parameter values in real/unscaled units then returns a biased predicted cost in real/unscaled units. If False, then this method takes parameter values in scaled units and returns a biased predicted cost in scaled units. Note that this method cannot determine on its own if the values in params are in real/unscaled units or scaled units; it is up to the caller to pass the correct values. Defaults to True.

Returns:

Biased cost predicted for the given

parameters. This will be in real/unscaled units if perform_scaling is True or it will be in scaled units if perform_scaling is False.

Return type:

pred_bias_cost (float)

predict_cost(params, perform_scaling=True, return_uncertainty=False)

Predict the cost for params using self.gaussian_process.

This method also optionally returns the uncertainty of the predicted cost.

By default (with perform_scaling=True) this method will use self.params_scaler to scale the input values and then use self.cost_scaler to scale the cost back to real/unscaled units. If perform_scaling is False, then this scaling will NOT be done. In that case, params should consist of already-scaled parameter values and the returned cost (and optional uncertainty) will be in scaled units.

Parameters:

• params (array) – A 1D array containing the values for each parameter. These should be in real/unscaled units if perform_scaling is True or they should be in scaled units if perform_scaling is False.
• perform_scaling (bool, optional) – Whether or not the parameters and costs should be scaled. If True then this method takes in parameter values in real/unscaled units then returns a predicted cost (and optionally the predicted cost uncertainty) in real/unscaled units. If False, then this method takes parameter values in scaled units and returns a cost (and optionally the predicted cost uncertainty) in scaled units. Note that this method cannot determine on its own if the values in params are in real/unscaled units or scaled units; it is up to the caller to pass the correct values. Defaults to True.
• return_uncertainty (bool, optional) – This optional argument controls whether or not the predicted cost uncertainty is returned with the predicted cost. The predicted cost uncertainty will be in real/unscaled units if perform_scaling is True and will be in scaled units if perform_scaling is False. Defaults to False.

Returns:

Predicted cost at params. The cost will be in

real/unscaled units if perform_scaling is True and will be in scaled units if perform_scaling is False.

uncertainty (float, optional): The uncertainty of the predicted

cost. This will be in the same units (either real/unscaled or scaled) as the returned cost. The cost_uncertainty will only be returned if return_uncertainty is True.

Return type:

cost (float)

run()

Starts running the Gaussian process learner. When the new parameters event is triggered, reads the cost information provided and updates the Gaussian process with the information. Then searches the Gaussian process for new optimal parameters to test based on the biased cost. Parameters to test next are put on the output parameters queue.

update_archive()

Update the archive.

update_bias_function()

Set the constants for the cost bias function.

class mloop.learners.Learner(num_params=None, min_boundary=None, max_boundary=None, learner_archive_filename='learner_archive', learner_archive_file_type='txt', start_datetime=None, param_names=None, **kwargs)

Bases: object

Base class for all learners. Contains default boundaries and some useful functions that all learners use.

The class that inherits from this class should also inherit from threading.Thread or multiprocessing.Process, depending if you need the learner to be a genuine parallel process or not.

Keyword Arguments:  

• num_params (Optional [int]) – The number of parameters to be optimized. If None defaults to 1. Default None.
• min_boundary (Optional [array]) – Array with minimum values allowed for each parameter. Note if certain values have no minimum value you can set them to -inf for example [-1, 2, float(‘-inf’)] is a valid min_boundary. If None sets all the boundaries to ‘-1’. Default None.
• max_boundary (Optional [array]) – Array with maximum values allowed for each parameter. Note if certain values have no maximum value you can set them to +inf for example [0, float(‘inf’),3,-12] is a valid max_boundary. If None sets all the boundaries to ‘1’. Default None.
• learner_archive_filename (Optional [string]) – Name for python archive of the learners current state. If None, no archive is saved. Default None. But this is typically overloaded by the child class.
• learner_archive_file_type (Optional [string]) – File type for archive. Can be either ‘txt’ a human readable text file, ‘pkl’ a python dill file, ‘mat’ a matlab file or None if there is no archive. Default ‘mat’.
• log_level (Optional [int]) – Level for the learners logger. If None, set to warning. Default None.
• start_datetime (Optional [datetime]) – Start date time, if None, is automatically generated.
• param_names (Optional [list of str]) – A list of names of the parameters for use e.g. in plot legends. Number of elements must equal num_params. If None, each name will be set to an empty sting. Default None.

params_out_queue

Queue for parameters created by learner.

Type:queue

costs_in_queue

Queue for costs to be used by learner.

Type:queue

end_event

Event to trigger end of learner.

Type:event

all_params

Array containing all parameters sent to learner.

Type:array

all_costs

Array containing all costs sent to learner.

Type:array

all_uncers

Array containing all uncertainties sent to learner.

Type:array

bad_run_indexs

list of indexes to all runs that were marked as bad.

Type:list

OUT_TYPE = ''

_parse_cost_message(message)

Parse a message sent from the controller via self.costs_in_queue.

Parameters:message (tuple) – A tuple put in self.costs_in_queue by the controller. It should be of the form (params, cost, uncer, bad) where params is an array specifying the parameter values used, cost is the measured cost for those parameter values, uncer is the uncertainty measured for those parameter values, and bad is a boolean indicating whether the run was bad. Raises:ValueError – A ValueError is raised if the number of parameters in the provided params doesn’t match self.num_params. Returns:

A tuple of the form (params, cost, uncer, bad). For more

information on the meaning of those parameters, see the entry for the message argument above.

Return type:tuple

_prepare_logger()

Prepare the logger.

If self.log already exists, then this method silently returns without changing anything.

_set_trust_region(trust_region)

Sets trust region properties for learner that have this. Common function for learners with trust regions.

Parameters:trust_region (float or array) – Property defines the trust region.

_shut_down()

Shut down and perform one final save of learner.

_update_run_data_attributes(params, cost, uncer, bad)

Update attributes that store the results returned by the controller.

Parameters:

• params (array) – Array of control parameter values.
• cost (float) – The cost measured for params.
• uncer (float) – The uncertainty measured for params.
• bad (bool) – Whether or not the run was bad.

check_in_boundary(param)

Check given parameters are within stored boundaries.

Parameters:param (array) – array of parameters Returns:True if the parameters are within boundaries, False otherwise. Return type:bool

check_in_diff_boundary(param)

Check given distances are less than the boundaries.

Parameters:param (array) – array of distances Returns:True if the distances are smaller or equal to boundaries, False otherwise. Return type:bool

check_num_params(param)

Check the number of parameters is right.

put_params_and_get_cost(params, **kwargs)

Send parameters to queue and whatever additional keywords.

Also saves sent and received variables in appropriate storage arrays.

Parameters:params (array) – array of values to be sent to file Returns:cost from the cost queue

save_archive()

Save the archive associated with the learner class. Only occurs if the filename for the archive is not None. Saves with the format previously set.

update_archive()

Update the dictionary of parameters and values to save to the archive.

Child classes should call this method and also updated self.archive_dict with any other parameters and values that need to be saved to the learner archive.

exception mloop.learners.LearnerInterrupt

Bases: Exception

Exception that is raised when the learner is ended with the end flag or event.

class mloop.learners.MachineLearner(trust_region=None, default_bad_cost=None, default_bad_uncertainty=None, minimum_uncertainty=1e-08, predict_global_minima_at_end=True, training_filename=None, **kwargs)

Bases: mloop.learners.Learner

A parent class for more specific machine learer classes.

This class is not intended to be used directly.

Keyword Arguments:  

• trust_region (Optional [float or array]) – The trust region defines the maximum distance the learner will travel from the current best set of parameters. If None, the learner will search everywhere. If a float, this number must be between 0 and 1 and defines maximum distance the learner will venture as a percentage of the boundaries. If it is an array, it must have the same size as the number of parameters and the numbers define the maximum absolute distance that can be moved along each direction.
• default_bad_cost (Optional [float]) – If a run is reported as bad and default_bad_cost is provided, the cost for the bad run is set to this default value. If default_bad_cost is None, then the worst cost received is set to all the bad runs. Default None.
• default_bad_uncertainty (Optional [float]) – If a run is reported as bad and default_bad_uncertainty is provided, the uncertainty for the bad run is set to this default value. If default_bad_uncertainty is None, then the uncertainty is set to a tenth of the best to worst cost range. Default `None.
• minimum_uncertainty (Optional [float]) – The minimum uncertainty associated with provided costs. Must be above zero to avoid fitting errors. Default 1e-8.
• predict_global_minima_at_end (Optional [bool]) – If True finds the global minima when the learner is ended. Does not if False. Default True.
• training_filename (Optional [str]) – The name of a learner archive from a previous optimization from which to extract past results for use in the current optimization. If None, no past results will be used. Default None.

all_params

Array containing all parameters sent to learner.

Type:array

all_costs

Array containing all costs sent to learner.

Type:array

all_uncers

Array containing all uncertainties sent to learner.

Type:array

scaled_costs

Array contaning all the costs scaled to have zero mean and a standard deviation of 1. Needed for training the learner.

Type:array

bad_run_indexs

list of indexes to all runs that were marked as bad.

Type:list

best_cost

Minimum received cost, updated during execution.

Type:float

best_params

Parameters of best run. (reference to element in params array).

Type:array

best_index

index of the best cost and params.

Type:int

worst_cost

Maximum received cost, updated during execution.

Type:float

worst_index

index to run with worst cost.

Type:int

cost_range

Difference between worst_cost and best_cost

Type:float

params_count

Counter for the number of parameters asked to be evaluated by the learner.

Type:int

has_trust_region

Whether the learner has a trust region.

Type:bool

_find_predicted_minimum(scaled_figure_of_merit_function, scaled_search_region, params_scaler, scaled_jacobian_function=None)

Find the predicted minimum of scaled_figure_of_merit_function().

The search for the minimum is constrained to be within scaled_search_region.

The scaled_figure_of_merit_function() should take inputs in scaled units and generate outputs in scaled units. This is necessary because scipy.optimize.minimize() (which is used internally here) can struggle if the numbers are too small or too large. Using scaled parameters and figures of merit brings the numbers closer to ~1, which can improve the behavior of scipy.optimize.minimize().

Parameters:

• scaled_figure_of_merit_function (function) – This should be a function which accepts an array of scaled parameter values and returns a predicted figure of merit. Importantly, both the input parameter values and the returned value should be in scaled units.
• scaled_search_region (array) – The scaled parameter-space bounds for the search. The returned minimum position will be constrained to be within this region. The scaled_search_region should be a 2D array of shape (self.num_params, 2) where the first column specifies lower bounds and the second column specifies upper bounds for each parameter (in scaled units).
• params_scaler (mloop.utilities.ParameterScaler) – A ParameterScaler instance for converting parameters to scaled units.
• scaled_jacobian_function (function, optional) – An optional function giving the Jacobian of scaled_figure_of_merit_function() which will be used by scipy.optimize.minimize() if provided. As with scaled_figure_of_merit_function(), the scaled_jacobian_function() should accept and return values in scaled units. If None then no Jacobian will be provided to scipy.optimize.minimize(). Defaults to None.

Returns:

The scaled parameter values which

minimize scaled_figure_of_merit_function() within scaled_search_region. They are provided as a 1D array of values in scaled units.

Return type:

best_scaled_params (array)

_reconcile_kwarg_and_training_val(kwargs_, name, training_value)

Utility function for comparing values from kwargs to training values.

When a training archive is specified there can be two values specified for some parameters; one from user’s config/kwargs and one from the training archive. This function compares the values. If the values are the same then the value is returned, and if they are different a ValueError is raised. Care is taken not to raise that error though if one of the values is None since that can mean that a value wasn’t specified. In that case the other value is returned, or None is returned if they are both None.

Parameters:

• kwargs ([dict]) – The dictionary of keyword arguments passed to __init__().
• name ([str]) – The name of the parameter.
• training_value ([any]) – The value for the parameter in the training archive.

Raises:

ValueError – A ValueError is raised if the value of the parameter in the keyword arguments doesn’t match the value from the training archive.

Returns:

The value for the parameter, taken from either kwargs_ or

training_value, or both if they are the same.

Return type:

[any]

get_params_and_costs()

Get the parameters and costs from the queue and place in their appropriate all_[type] arrays.

Also updates bad costs, best parameters, and search boundaries given trust region.

update_archive()

Update the archive.

update_bads()

Best and/or worst costs have changed, update the values associated with bad runs accordingly.

update_search_params()

Update the list of parameters to use for the next search.

update_search_region()

If trust boundaries is not none, updates the search boundaries based on the defined trust region.

wait_for_new_params_event()

Waits for a new parameters event and starts a new parameter generation cycle.

Also checks end event and will break if it is triggered.

class mloop.learners.NelderMeadLearner(initial_simplex_corner=None, initial_simplex_displacements=None, initial_simplex_scale=None, **kwargs)

Bases: mloop.learners.Learner, threading.Thread

Nelder–Mead learner. Executes the Nelder–Mead learner algorithm and stores the needed simplex to estimate the next points.

Parameters:

• params_out_queue (queue) – Queue for parameters from controller.
• costs_in_queue (queue) – Queue for costs for nelder learner. The queue should be populated with cost (float) corresponding to the last parameter sent from the Nelder–Mead Learner. Can be a float(‘inf’) if it was a bad run.
• end_event (event) – Event to trigger end of learner.

Keyword Arguments:  

• initial_simplex_corner (Optional [array]) – Array for the initial set of parameters, which is the lowest corner of the initial simplex. If None the initial parameters are randomly sampled if the boundary conditions are provided, or all are set to 0 if boundary conditions are not provided.
• initial_simplex_displacements (Optional [array]) – Array used to construct the initial simplex. Each array is the positive displacement of the parameters above the init_params. If None and there are no boundary conditions, all are set to 1. If None and there are boundary conditions assumes the initial conditions are scaled. Default None.
• initial_simplex_scale (Optional [float]) – Creates a simplex using a the boundary conditions and the scaling factor provided. If None uses the init_simplex if provided. If None and init_simplex is not provided, but boundary conditions are is set to 0.5. Default None.

init_simplex_corner

Parameters for the corner of the initial simple used.

Type:array

init_simplex_disp

Parameters for the displacements about the simplex corner used to create the initial simple.

Type:array

simplex_params

Parameters of the current simplex

Type:array

simplex_costs

Costs associated with the parameters of the current simplex

Type:array

OUT_TYPE = 'nelder_mead'

run()

Runs Nelder–Mead algorithm to produce new parameters given costs, until end signal is given.

update_archive()

Update the archive.

class mloop.learners.NeuralNetLearner(update_hyperparameters=False, **kwargs)

Bases: mloop.learners.MachineLearner, multiprocessing.context.Process

Learner that uses a neural network for function approximation.

Parameters:

• params_out_queue (queue) – Queue for parameters sent to controller.
• costs_in_queue (queue) – Queue for costs.
• end_event (event) – Event to trigger end of learner.

Keyword Arguments:  

• update_hyperparameters (Optional [bool]) – Whether the hyperparameters used to prevent overfitting should be tuned by trying out different values. Setting to True can reduce overfitting of the model, but can slow down the fitting due to the computational cost of trying different values. Default False.
• training_filename (Optional [str]) – The name of a learner archive from a previous optimization from which to extract past results for use in the current optimization. If None, no past results will be used. Default None.
• trust_region (Optional [float or array]) – The trust region defines the maximum distance the learner will travel from the current best set of parameters. If None, the learner will search everywhere. If a float, this number must be between 0 and 1 and defines maximum distance the learner will venture as a fraction of the boundaries. If it is an array, it must have the same size as the number of parameters and the numbers define the maximum absolute distance that can be moved along each direction.
• default_bad_cost (Optional [float]) – If a run is reported as bad and default_bad_cost is provided, the cost for the bad run is set to this default value. If default_bad_cost is None, then the worst cost received is set to all the bad runs. Default None.
• default_bad_uncertainty (Optional [float]) – If a run is reported as bad and default_bad_uncertainty is provided, the uncertainty for the bad run is set to this default value. If default_bad_uncertainty is None, then the uncertainty is set to one tenth of the best to worst cost range. Default None.
• minimum_uncertainty (Optional [float]) – The minimum uncertainty associated with provided costs. Must be above zero to avoid fitting errors. Default 1e-8.
• predict_global_minima_at_end (Optional [bool]) – If True, find the global minima when the learner is ended. Does not if False. Default True.

all_params

Array containing all parameters sent to learner.

Type:array

all_costs

Array containing all costs sent to learner.

Type:array

all_uncers

Array containing all uncertainties sent to learner.

Type:array

scaled_costs

Array contaning all the costs scaled to have zero mean and a standard deviation of 1.

Type:array

bad_run_indexs

list of indexes to all runs that were marked as bad.

Type:list

best_cost

Minimum received cost, updated during execution.

Type:float

best_params

Parameters of best run. (reference to element in params array).

Type:array

best_index

Index of the best cost and params.

Type:int

worst_cost

Maximum received cost, updated during execution.

Type:float

worst_index

Index to run with worst cost.

Type:int

cost_range

Difference between worst_cost and best_cost

Type:float

generation_num

Number of sets of parameters to generate each generation. Set to 3.

Type:int

cost_count

Counter for the number of costs, parameters and uncertainties added to learner.

Type:int

params_count

Counter for the number of parameters asked to be evaluated by the learner.

Type:int

neural_net

Neural net that is fitted to data and used to make predictions.

Type:NeuralNet

cost_scaler

Scaler used to normalize the provided costs.

Type:StandardScaler

cost_scaler_init_index

The number of params to use to initialise cost_scaler.

Type:int

has_trust_region

Whether the learner has a trust region.

Type:bool

OUT_TYPE = 'neural_net'

_fit_neural_net(index)

Fits a neural net to the data.

cost_scaler must have been fitted before calling this method.

_init_cost_scaler()

Initialises the cost scaler. cost_scaler_init_index must be set.

create_neural_net()

Creates the neural net. Must be called from the same process as fit_neural_net, predict_cost and predict_costs_from_param_array.

find_global_minima(net_index=None)

Search for the global minima predicted by the neural net.

This method will attempt to find the global minima predicted by the neural net, but it is possible for it to become stuck in local minima of the predicted cost landscape.

This method does not return any values, but creates the attributes listed below.

Parameters:net_index (int, optional) – The index of the neural net to use to predict the cost. If None then a net will be randomly chosen. Defaults to None.

predicted_best_parameters

The parameter values which are predicted to yield the best results, as a 1D array.

Type:array

predicted_best_cost

The predicted cost at the predicted_best_parameters, in a 1D 1-element array.

Type:array

find_next_parameters(net_index=None)

Get the next parameters to test.

This method searches for the parameters expected to give the minimum cost, as predicted by a neural net.

This method additionally increments self.params_count appropriately.

Parameters:net_index (int, optional) – The index of the neural net to use to predict the cost. If None then a net will be randomly chosen. Defaults to None. Returns:

The next parameter values to try, stored in a

1D array.

Return type:next_params (array)

get_losses()

get_regularization_histories()

Get the regularization coefficient values used by the nets.

Returns:

The values used by the neural nets for the

regularization coefficient. There is one list per net, which includes all of the regularization coefficient values used by that net during the optimization. If the optimization was run with update_hyperparameters set to False, then each net’s list will only have one entry, namely the initial default value for the regularization coefficient. If the optimization was run with updated_hyperparameters set to True then the list will also include the optimal values for the regularization coefficient determined during each hyperparameter fitting.

Return type:list of list of float

import_neural_net()

Imports neural net parameters from the training dictionary provided at construction. Must be called from the same process as fit_neural_net, predict_cost and predict_costs_from_param_array. You must call exactly one of this and create_neural_net before calling other methods.

predict_cost(params, net_index=None, perform_scaling=True)

Predict the cost from the neural net for params.

This method is a wrapper around mloop.neuralnet.NeuralNet.predict_cost().

Parameters:

• params (array) – A 1D array containing the values for each parameter. These should be in real/unscaled units if perform_scaling is True or they should be in scaled units if perform_scaling is False.
• net_index (int, optional) – The index of the neural net to use to predict the cost. If None then a net will be randomly chosen. Defaults to None.
• perform_scaling (bool, optional) – Whether or not the parameters and costs should be scaled. If True then this method takes in parameter values in real/unscaled units then returns a predicted cost in real/unscaled units. If False, then this method takes parameter values in scaled units and returns a cost in scaled units. Note that this method cannot determine on its own if the values in params are in real/unscaled units or scaled units; it is up to the caller to pass the correct values. Defaults to True.

Returns:

Predicted cost for params. This will be in

real/unscaled units if perform_scaling is True or it will be in scaled units if perform_scaling is False.

Return type:

cost (float)

predict_cost_gradient(params, net_index=None, perform_scaling=True)

Predict the gradient of the cost function at params.

This method is a wrapper around mloop.neuralnet.NeuralNet.predict_cost_gradient().

Parameters:

• params (array) – A 1D array containing the values for each parameter. These should be in real/unscaled units if perform_scaling is True or they should be in scaled units if perform_scaling is False.
• net_index (int, optional) – The index of the neural net to use to predict the cost gradient. If None then a net will be randomly chosen. Defaults to None.
• perform_scaling (bool, optional) – Whether or not the parameters and costs should be scaled. If True then this method takes in parameter values in real/unscaled units then returns a predicted cost gradient in real/unscaled units. If False, then this method takes parameter values in scaled units and returns a cost gradient in scaled units. Note that this method cannot determine on its own if the values in params are in real/unscaled units or scaled units; it is up to the caller to pass the correct values. Defaults to True.

Returns:

The predicted gradient at params. This

will be in real/unscaled units if perform_scaling is True or it will be in scaled units if perform_scaling is False.

Return type:

cost_gradient (np.float64)

predict_costs_from_param_array(params, net_index=None, perform_scaling=True)

Produces an array of predictsd costs from an array of params.

This method is similar to self.predict_cost() except that it accepts many different sets of parameter values simultaneously and returns a predicted cost for each set of parameter values.

Parameters:

• params (array) – A 2D array containing the values for each parameter. This should essentially be a list of sets of parameters values, where each set specifies one value for each parameter. In other words, each row of the array should be one set of parameter values which could be passed to self.predict_cost(). These should be in real/unscaled units if perform_scaling is True or they should be in scaled units if perform_scaling is False.
• net_index (int, optional) – The index of the neural net to use to predict the cost. If None then a net will be randomly chosen. Defaults to None.
• perform_scaling (bool, optional) – Whether or not the parameters and costs should be scaled. If True then this method takes in parameter values in real/unscaled units then returns a predicted cost in real/unscaled units. If False, then this method takes parameter values in scaled units and returns a cost in scaled units. Note that this method cannot determine on its own if the values in params are in real/unscaled units or scaled units; it is up to the caller to pass the correct values. Defaults to True.

Returns:

A list of floats giving the predicted cost for each set of

parameter values.

Return type:

list

run()

Starts running the neural network learner. When the new parameters event is triggered, reads the cost information provided and updates the neural network with the information. Then searches the neural network for new optimal parameters to test based on the biased cost. Parameters to test next are put on the output parameters queue.

update_archive()

Update the archive.

class mloop.learners.RandomLearner(trust_region=None, first_params=None, **kwargs)

Bases: mloop.learners.Learner, threading.Thread

Random learner. Simply generates new parameters randomly with a uniform distribution over the boundaries. Learner is perhaps a misnomer for this class.

Parameters:

**kwargs (Optional dict) – Other values to be passed to Learner.

Keyword Arguments:  

• min_boundary (Optional [array]) – If set to None, overrides default learner values and sets it to a set of value 0. Default None.
• max_boundary (Optional [array]) – If set to None overides default learner values and sets it to an array of value 1. Default None.
• first_params (Optional [array]) – The first parameters to test. If None will just randomly sample the initial condition.
• trust_region (Optional [float or array]) – The trust region defines the maximum distance the learner will travel from the current best set of parameters. If None, the learner will search everywhere. If a float, this number must be between 0 and 1 and defines maximum distance the learner will venture as a percentage of the boundaries. If it is an array, it must have the same size as the number of parameters and the numbers define the maximum absolute distance that can be moved along each direction.

OUT_TYPE = 'random'

run()

Puts the next parameters on the queue which are randomly picked from a uniform distribution between the minimum and maximum boundaries when a cost is added to the cost queue.