visualizations¶

Module of classes used to create visualizations of data produced by the experiment and learners.

class mloop.visualizations.ControllerVisualizer(filename, file_type=None, **kwargs)¶

Bases: object

ControllerVisualizer creates figures from a Controller Archive.

Note that the data from the training archive, if one was provided to the learner at the beginning of the optimization, is NOT included in the controller archive generated during the optimization. Therefore any data from the training archive is not included in the plots generated by this class. This is in contrast to some of the learner visualizer classes.

Keyword Arguments:
Parameters:	filename (String) – Filename of the controller archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None.

create_visualizations(plot_cost_vs_run=True, plot_parameters_vs_run=True, plot_parameters_vs_cost=True, max_parameters_per_plot=None)¶

Runs the plots for a controller file.

Keyword Arguments:

plot_cost_vs_run (Optional [bool]) – If True plot cost versus run number, else do not. Default True.
plot_parameters_vs_run (Optional [bool]) – If True plot parameters versus run number, else do not. Default True.
plot_parameters_vs_cost (Optional [bool]) – If True plot parameters versus cost number, else do not. Default True.
max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. Default None.

plot_cost_vs_run()¶

Create a plot of the costs versus run number.

Note that the data from the training archive, if one was provided to the learner at the beginning of the optimization, will NOT be plotted here.

plot_parameters_vs_cost(parameter_subset=None)¶

Create a plot of the parameters versus run number.

Note that the data from the training archive, if one was provided to the learner at the beginning of the optimization, will NOT be plotted here.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

plot_parameters_vs_run(parameter_subset=None)¶

Create a plot of the parameters versus run number.

Note that the data from the training archive, if one was provided to the learner at the beginning of the optimization, will NOT be plotted here.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

class mloop.visualizations.DifferentialEvolutionVisualizer(filename, file_type=None, **kwargs)¶

Bases: object

DifferentialEvolutionVisualizer creates figures from a differential evolution archive.

Keyword Arguments:
Parameters:	filename (String) – Filename of the DifferentialEvolutionVisualizer archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None.

create_visualizations(plot_params_vs_generations=True, plot_costs_vs_generations=True, max_parameters_per_plot=None)¶

Runs the plots from a differential evolution learner file.

Keyword Arguments:

plot_params_generations (Optional [bool]) – If True plot parameters vs generations, else do not. Default True.
plot_costs_generations (Optional [bool]) – If True plot costs vs generations, else do not. Default True.
max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. Default None.

plot_costs_vs_generations()¶: Create a plot of the costs versus run number.

plot_params_vs_generations(parameter_subset=None)¶

Create a plot of the parameters versus run number.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

class mloop.visualizations.GaussianProcessVisualizer(filename, file_type=None, **kwargs)¶

Bases: mloop.learners.GaussianProcessLearner

GaussianProcessVisualizer extends of GaussianProcessLearner, designed not to be used as a learner, but to instead post process a GaussianProcessLearner archive file and produce useful data for visualization of the state of the learner. Fixes the Gaussian process hyperparameters to what was last found during the run.

If a training archive was provided at the start of the optimization as gp_training_filename and that training archive was generated by a Gaussian process optimization, then some of its data is saved in the new learner archive generated during the optimization. That implies that some of the data, such as fitted hyperparameter values, from the training archive will be included in the plots generated by this class.

Keyword Arguments:
Parameters:	filename (String) – Filename of the GaussianProcessLearner archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None.

create_visualizations(plot_cross_sections=True, plot_hyperparameters_vs_fit=True, plot_noise_level_vs_fit=True, max_parameters_per_plot=None, **kwargs)¶

Runs the plots from a gaussian process learner file.

Keyword Arguments:

plot_cross_sections (Optional [bool]) – If True plot predicted landscape cross sections, else do not. Default True.
plot_hyperparameters_vs_fit (Optional [bool]) – If True plot fitted hyperparameters as a function of fit number, else do not. Default True.
plot_noise_level_vs_fit (Optional [bool]) – If True plot the fitted noise level as a function of fit number, else do not. If there is no fitted noise level (i.e. cost_has_noise was set to False), then this plot will not be made regardless of the value passed for plot_noise_level_vs_fit. Default True.
max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. Default None.

plot_cross_sections(parameter_subset=None)¶

Produce a figure of the cross section about best cost and parameters.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

plot_hyperparameters_vs_fit(parameter_subset=None)¶

Produce a figure of the hyperparameters as a function of fit number.

Only one fit is performed per generation, and multiple parameter sets are run each generation. Therefore the number of fits is generally less than the number of runs.

The plot generated will include the data from the training archive if one was provided as gp_training_filename and the training archive was generated by a Gaussian process optimization.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

plot_hyperparameters_vs_run(*args, **kwargs)¶: Deprecated. Use plot_hyperparameters_vs_fit() instead.

plot_noise_level_vs_fit()¶

This method plots the fitted noise level as a function of fit number.

The noise_level approximates the variance of values that would be measured if the cost were repeatedly measured for the same set of parameters. Note that this is the variance in those costs; not the standard deviation.

This plot is only relevant to optimizations for which cost_has_noise is True. If cost_has_noise is False then this method does nothing and silently returns.

Only one fit is performed per generation, and multiple parameter sets are run each generation. Therefore the number of fits is generally less than the number of runs.

The plot generated will include the data from the training archive if one was provided as gp_training_filename and the training archive was generated by a Gaussian process optimization.

plot_noise_level_vs_run(*args, **kwargs)¶: Deprecated. Use plot_noise_level_vs_fit() instead.

return_cross_sections(points=100, cross_section_center=None)¶

Finds the predicted global minima, then returns a list of vectors of parameters values, costs and uncertainties, corresponding to the 1D cross sections along each parameter axis through the predicted global minima.

Keyword Arguments:
	points (int) – the number of points to sample along each cross section. Default value is 100. cross_section_center (array) – parameter array where the centre of the cross section should be taken. If None, the parameters for the best returned cost are used.
Returns:	a tuple (cross_arrays, cost_arrays, uncer_arrays) cross_parameter_arrays (list): a list of arrays for each cross section, with the values of the varied parameter going from the minimum to maximum value. cost_arrays (list): a list of arrays for the costs evaluated along each cross section about the minimum. uncertainty_arrays (list): a list of uncertainties

run()¶: Overides the GaussianProcessLearner multiprocessor run routine. Does nothing but makes a warning.

class mloop.visualizations.NeuralNetVisualizer(filename, file_type=None, **kwargs)¶

Bases: mloop.learners.NeuralNetLearner

NeuralNetVisualizer extends of NeuralNetLearner, designed not to be used as a learner, but to instead post process a NeuralNetLearner archive file and produce useful data for visualization of the state of the learner.

Keyword Arguments:
Parameters:	filename (String) – Filename of the NeuralNetLearner archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None.

create_visualizations(plot_cross_sections=True, max_parameters_per_plot=None)¶

Creates plots from a neural net’s learner file.

Keyword Arguments:

plot_cross_sections (Optional [bool]) – If True plot predicted landscape cross sections, else do not. Default True.
max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. Default None.

do_cross_sections(parameter_subset=None)¶

Produce a figure of the cross section about best cost and parameters.

Parameters:	parameter_subset (list-like) – The indices of parameters to plot. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be between 0 and the number of parameters minus one, inclusively. If set to None, then all parameters will be plotted. Default None.

plot_density_surface()¶: Produce a density plot of the cost surface (only works when there are 2 parameters)

plot_losses()¶

Produce a figure of the loss as a function of epoch for each net.

The loss is the mean-squared fitting error of the neural net plus the regularization loss, which is the regularization coefficient times the mean L2 norm of the neural net weight arrays (without the square root). Note that the fitting error is calculated after normalizing the data, so it is in arbitrary units.

As the neural nets are fit, the loss is recorded every 10 epochs. The number of epochs per fit varies, and may be different for different nets. The loss will generally increase at the begining of each fit as new data points will have been added.

Also note that a lower loss isn’t always better; a loss that is too low can be a sign of overfitting.

plot_regularization_history()¶

Produces a plot of the regularization coefficient values used.

The neural nets use L2 regularization to smooth their predicted landscapes in an attempt to avoid overfitting the data. The strength of the regularization is set by the regularization coefficient, which is a hyperparameter that is tuned during the optimization if update_hyperparameters is set to True. Generally larger regularization coefficient values force the landscape to be smoother while smaller values allow it to vary more quickly. A value too large can lead to underfitting while a value too small can lead to overfitting. The ideal regularization coefficient value will depend on many factors, such as the shape of the actual cost landscape, the SNR of the measured costs, and even the number of measured costs.

This method plots the initial regularization coefficient value and the optimal values found for the regularization coefficient when performing the hyperparameter tuning. One curve showing the history of values used for the regularization coefficient is plotted for each neural net. If update_hyperparameters was set to False during the optimization, then only the initial default value will be plotted.

plot_surface()¶: Produce a figure of the cost surface (only works when there are 2 parameters)

return_cross_sections(points=100, cross_section_center=None)¶

Finds the predicted global minima, then returns a list of vectors of parameters values, costs and uncertainties, corresponding to the 1D cross sections along each parameter axis through the predicted global minima.

Keyword Arguments:
	points (int) – the number of points to sample along each cross section. Default value is 100. cross_section_center (array) – parameter array where the centre of the cross section should be taken. If None, the parameters for the best returned cost are used.
Returns:	a tuple (cross_arrays, cost_arrays, uncer_arrays) cross_parameter_arrays (list): a list of arrays for each cross section, with the values of the varied parameter going from the minimum to maximum value. cost_arrays (list): a list of arrays for the costs evaluated along each cross section about the minimum. uncertainty_arrays (list): a list of uncertainties

run()¶: Overides the GaussianProcessLearner multiprocessor run routine. Does nothing but makes a warning.

mloop.visualizations._color_from_controller_name(controller_name)¶: Gives a color (as a number between zero an one) corresponding to each controller name string.

mloop.visualizations._color_list_from_num_of_params(num_of_params)¶: Gives a list of colors based on the number of parameters.

mloop.visualizations._ensure_parameter_subset_valid(visualizer, parameter_subset)¶

Make sure indices in parameter_subset are acceptable.

Parameters:

visualizer (ControllerVisualizer-like) – An instance of one of the visualization classes defined in this module, which should have the attributes param_numbers and log.
parameter_subset (list-like) – The indices corresponding to a subset of the optimization parameters. The indices should be 0-based, i.e. the first parameter is identified with index 0. Generally the values of the indices in parameter_subset should be integers between 0 and the number of parameters minus one, inclusively.

mloop.visualizations.configure_plots()¶: Configure the setting for the plots.

mloop.visualizations.create_controller_visualizations(filename, file_type=None, **kwargs)¶

Runs the plots for a controller file.

Keyword Arguments:
Parameters:	filename (String) – Filename of the controller archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None. **kwargs – Additional keyword arguments are passed to the visualizer’s create_visualizations() method.

mloop.visualizations.create_differential_evolution_learner_visualizations(filename, file_type=None, **kwargs)¶

Runs the plots from a differential evolution learner file.

Keyword Arguments:
Parameters:	filename (String) – Filename for the differential evolution learner archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None. **kwargs – Additional keyword arguments are passed to the visualizer’s create_visualizations() method.

mloop.visualizations.create_gaussian_process_learner_visualizations(filename, file_type=None, **kwargs)¶

Runs the plots from a gaussian process learner file.

Keyword Arguments:
Parameters:	filename (String) – Filename for the gaussian process learner archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None. **kwargs – Additional keyword arguments are passed to the visualizer’s create_visualizations() method.

mloop.visualizations.create_learner_visualizations(filename, max_parameters_per_plot=None, learner_visualization_kwargs=None, learner_visualizer_init_kwargs=None)¶

Runs the plots for a learner archive file.

Parameters:

filename (str) – Filename for the learner archive.

Keyword Arguments:

max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. If a value for max_parameters_per_plot is included in learner_visualization_kwargs, then the value in that dictionary will override this setting. Default None.
learner_visualization_kwargs (dict) – Keyword arguments to pass to the learner visualizer’s create_visualizations() method. If set to None, no additional keyword arguments will be passed. Default None.
learner_visualizer_init_kwargs (dict) – Keyword arguments to pass to the learner visualizer’s __init__() method. If set to None, no additional keyword arguments will be passed. Default None.

mloop.visualizations.create_learner_visualizer_from_archive(filename, controller_type=None, **kwargs)¶

Create an instance of the appropriate visualizer class for a learner archive.

Keyword Arguments:
Parameters:	filename (String) – Filename of the learner archive.
	controller_type (String) – The type of controller used during the optimization that created the provided learner archive. Options include ‘gaussian_process’, ‘neural_net’, and ‘differential_evolution’. If set to None, then controller_type will be determined automatically from the archive. Default None. **kwargs – Additional keyword arguments are passed to the visualizer’s __init__() method.
Returns:	An instance of the appropriate visualizer class for plotting data from filename.
Return type:	visualizer

mloop.visualizations.create_neural_net_learner_visualizations(filename, file_type=None, **kwargs)¶

Creates plots from a neural net’s learner file.

Keyword Arguments:
Parameters:	filename (String) – Filename for the neural net learner archive.
	file_type (String) – Can be ‘mat’ for matlab, ‘pkl’ for pickle or ‘txt’ for text. If set to None, then the type will be determined from the extension in filename. Default None. **kwargs – Additional keyword arguments are passed to the visualizer’s create_visualizations() method.

mloop.visualizations.set_legend_location(loc=None)¶

Set the location of the legend in future figures.

Note that this function doesn’t change the location of legends in existing figures. It only changes where legends will appear in figures generated after the call to this function. If called without arguments, the legend location for future figures will revert to its default value.

Keyword Arguments:
	loc (Optional str, int, or pair of floats) – The value to use for loc in the calls to matplotlib’s legend(). Can be e.g. 2, ‘upper right’, (1, 0). See matplotlib’s documentation for more options and additional information. If set to None then the legend location will be set back to its default value. Default None.

mloop.visualizations.show_all_default_visualizations(controller, show_plots=True, max_parameters_per_plot=None)¶

Plots all visualizations available for a controller, and it’s internal learners.

Parameters:

controller (Controller) – The controller to extract plots from

Keyword Arguments:

show_plots (Optional, bool) – Determine whether to run plt.show() at the end or not. For debugging. Default True.
max_parameters_per_plot (Optional, int) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. Default None.

mloop.visualizations.show_all_default_visualizations_from_archive(controller_filename, learner_filename, controller_type=None, show_plots=True, max_parameters_per_plot=None, controller_visualization_kwargs=None, learner_visualization_kwargs=None, learner_visualizer_init_kwargs=None)¶

Plots all visualizations available for a controller and its learner from their archives.

Parameters:

controller_filename (str) – The filename, including path, of the controller archive.
learner_filename (str) – The filename, including path, of the learner archive.

Keyword Arguments:

controller_type (str) – The value of controller_type type used in the optimization corresponding to the learner learner archive, e.g. ‘gaussian_process’, ‘neural_net’, or ‘differential_evolution’. If set to None then controller_type will be determined automatically. Default None.
show_plots (bool) – Determine whether to run plt.show() at the end or not. For debugging. Default True.
max_parameters_per_plot (Optional [int]) – The maximum number of parameters to include in plots that display the values of parameters. If the number of parameters is larger than parameters_per_plot, then the parameters will be divided into groups and each group will be plotted in its own figure. If set to None, then all parameters will be included in the same plot regardless of how many there are. If a value for max_parameters_per_plot is included in controller_visualization_kwargs, then the value in that dictionary will override this setting. The same applies to learner_visualization_kwargs. Default None.
controller_visualization_kwargs (dict) – Keyword arguments to pass to the controller visualizer’s create_visualizations() method. If set to None, no additional keyword arguments will be passed. Default None.
learner_visualization_kwargs (dict) – Keyword arguments to pass to the learner visualizer’s create_visualizations() method. If set to None, no additional keyword arguments will be passed. Default None.
learner_visualizer_init_kwargs (dict) – Keyword arguments to pass to the learner visualizer’s __init__() method. If set to None, no additional keyword arguments will be passed. Default None.

visualizations¶

M-LOOP

Navigation

Related Topics