ESN¶

Implements the Echo State Network (ESN) used in Reservoir Computing

ESNWrapper Class¶

class rescomp.esn.ESNWrapper¶

Convenience Wrapper for the ESN class

For all normal usage of the rescomp package, this is the class to use.

Methods:

`create_input_matrix`(x_dim[, w_in_scale, ...])	create the input matrix _w_in. Can be used to create the _w_in before "train", otherwise it is called in "train" :param x_dim: dimension of the input data :type x_dim: int :param w_in_scale: maximum absolute value of the (random) w_in elements :type w_in_scale: float :param w_in_sparse: If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018) :type w_in_sparse: bool :param w_in_orderd: If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible) :type w_in_orderd: bool.
`create_network`([n_dim, n_rad, n_avg_deg, ...])	Creates the internal network used as reservoir in RC
`get_instance_version`()	Returns the rescomp package version used to create the class instance
`get_network`()	Returns the network used as reservoir
`get_network_parameters`()	Returns all network parameters
`get_w_in`()	Returns the input matrix w_in
`get_w_out`()	Returns the output matrix w_out
`predict`([x_pred, sync_steps, pred_steps, ...])	Synchronize the reservoir, then predict the system evolution
`set_console_logger`(log_level)	Set loglevel for the console output
`set_file_logger`(log_level, log_file_path)	Set's the logging file path
`synchronize`(x[, save_r])	Synchronize the reservoir state with the input time series
`to_pickle`(path[, compression, protocol])	Pickle (serialize) object to file.
`train`(x_train, sync_steps[, reg_param, ...])	Synchronize, then train the reservoir
`train_and_predict`(x_data, train_sync_steps, ...)	Train, then predict the evolution directly following the train data

create_input_matrix(x_dim, w_in_scale=1.0, w_in_sparse=True, w_in_ordered=False)¶

create the input matrix _w_in. Can be used to create the _w_in before “train”, otherwise it is called in “train” :param x_dim: dimension of the input data :type x_dim: int :param w_in_scale: maximum absolute value of the (random) w_in

elements

Parameters

w_in_sparse (bool) – If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018)
w_in_orderd (bool) – If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible)

create_network(n_dim=500, n_rad=0.1, n_avg_deg=6.0, n_type_flag='erdos_renyi', network_creation_attempts=10)¶

Creates the internal network used as reservoir in RC

Parameters

n_dim (int) – Nr. of nodes in the network
n_rad (float) – Spectral radius of the network. Must be >0. Should be < 1 to satisfy the echo state property
n_avg_deg (float) – Average node degree (number of connections). Used in the random network generations
n_type_flag (int_or_str) –
Type of Network to be generated. Possible flags and their synonyms are:
- ”random”, “erdos_renyi”
- ”scale_free”, “barabasi_albert”
- ”small_world”, “watts_strogatz”
network_creation_attempts (int) – How often the network generation should be attempted. It can fail due to a not converging eigenvalue calculation

get_instance_version()¶

Returns the rescomp package version used to create the class instance

Returns: Rescomp package version
Return type: str

get_network()¶

Returns the network used as reservoir

Returns: network saved as scipy.sparse matrix
Return type: csr_matrix

get_network_parameters()¶

Returns all network parameters

Returns: Dictionary of network parameters
Return type: dict

get_w_in()¶

Returns the input matrix w_in

Returns: w_in
Return type: np.ndarray

get_w_out()¶

Returns the output matrix w_out

Returns: w_out
Return type: np.ndarray

predict(x_pred=None, sync_steps=0, pred_steps=None, save_r=False, save_input=False)¶

Synchronize the reservoir, then predict the system evolution

Changes self._last_r and self._last_r_gen to stay synchronized to the new system state

Parameters

x_pred (np.ndarray) – Input data used to synchronize the reservoir, and then used as comparison for the prediction by being returned as y_pred in the output
sync_steps (int) – How many steps to use for synchronization before the prediction starts
pred_steps (int) – How many steps to predict
save_r (bool) – If true, saves r(t) internally
save_input (bool) – If true, saves the input data internally

Returns

2-element tuple containing:

y_pred (np.ndarray): Predicted future states
y_test (np.ndarray_or_None): Data taken from the input to compare the prediction with. If the prediction were “perfect” y_pred and y_test would be equal. Be careful though, y_test might be shorter than y_pred, or even None, if pred_steps is not None

Return type

tuple

set_console_logger(log_level)¶

Set loglevel for the console output

Parameters: () (log_level) – console loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels

set_file_logger(log_level, log_file_path)¶

Set’s the logging file path

Parameters

() (log_level) – file loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels
log_file_path (str) – path, including file type, to store the logfile in. E.g: “folder_structure/log_file.txt”

synchronize(x, save_r=False)¶

Synchronize the reservoir state with the input time series

This is usually done automatically in the training and prediction functions.

Parameters

x (np.ndarray) – Input data to be used for the synchronization, shape (T, d)
save_r (bool) – If true, saves and returns r

Returns

All r states if save_r is True, None if False

Return type

np.ndarray_or_None

to_pickle(path, compression='infer', protocol=5)¶

Pickle (serialize) object to file.

Disables logging as logging handlers can not be pickled. Uses pandas functions internally.

Parameters

path (str) – File path where the pickled object will be stored.
compression ({'infer', 'gzip', 'bz2', 'zip', 'xz', None}) – A string representing the compression to use in the output file. By default, infers from the file extension in specified path.
protocol (int) – Int which indicates which protocol should be used by the pickler. The possible values are 0, 1, 2, 3, 4. A negative value for the protocol parameter is equivalent to setting its value to HIGHEST_PROTOCOL.

train(x_train, sync_steps, reg_param=1e-05, w_in_scale=1.0, w_in_sparse=True, w_in_ordered=False, w_in_no_update=False, act_fct_flag='tanh_simple', bias_scale=0, mix_ratio=0.5, save_r=False, save_input=False, w_out_fit_flag='simple', loc_nbhd=None)¶

Synchronize, then train the reservoir

Parameters

x_train (np.ndarray) – Input data used to synchronize and then train the reservoir
sync_steps (int) – How many steps to use for synchronization before the prediction starts
reg_param (float) – weight for the Tikhonov-regularization term
w_in_scale (float) – maximum absolute value of the (random) w_in elements
w_in_sparse (bool) – If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018)
w_in_orderd (bool) – If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible)
w_in_no_update (bool) – If true and the input matrix W_in does already exist from a previous training run, W_in does not get updated, regardless of all other parameters concerning it.
act_fct_flag (int_or_str) –
Specifies the activation function to be used during training (and prediction). Possible flags and their synonyms are:
- ”tanh_simple”, “simple”
- ”tanh_bias”
bias_scale (float) – Bias to be used in some activation functions
mix_ratio (float) – Ratio of normal tanh vs squared tanh activation functions if act_fct_flag “mixed” is chosen
save_r (bool) – If true, saves r(t) internally
save_input (bool) – If true, saves the input data internally
w_out_fit_flag (str) – Type of nonlinear transformation applied to the reservoir states r to be used during the fit (and future prediction)
loc_nbhd (np.ndarray) – The local neighborhood used for the generalized local states approach. For more information, please see the docs.

train_and_predict(x_data, train_sync_steps, train_steps, pred_sync_steps=0, pred_steps=None, disc_steps=0, **kwargs)¶

Train, then predict the evolution directly following the train data

Parameters

x_data (np.ndarray) – Data used for synchronization, training and prediction (start and comparison)
train_sync_steps (int) – Steps to synchronize the reservoir with before the ‘real’ training begins
train_steps (int) – Steps to use for training and fitting w_in
pred_sync_steps (int) – steps to sync the reservoir with before prediction
pred_steps (int) – How many steps to predict the evolution for
**kwargs – further arguments passed to train() and predict()

Returns

2-element tuple containing:

y_pred (np.ndarray): Predicted future states
y_test (np.ndarray_or_None): Data taken from the input to compare the prediction with. If the prediction were “perfect” y_pred and y_test would be equal. Be careful though, y_test might be shorter than y_pred, or even None, if pred_steps is not None

Return type

tuple

ESNGenLoc Class¶

class rescomp.esn.ESNGenLoc¶

Generalized Local State Implementation of RC

For details on the how and why behind this implementation, please see the general_local_states_example in bin and/or the package documentation.

Idea behind the Local Neighborhoods matrix implementation: Human readable and plottable locality neighborhoods where each entry specifies the nature of a corresponding dimension. A simple example might look like this:

[1, 2, 1, 0]

[0, 1, 2, 1]

[2, 1, 1, 2]

where 0 means “not part of the neighborhood”, 1 means “part of the neighborhood, but not a core” and 2 means “core dimension” Each row is a “neighborhood” specifying which dimensions are grouped together as input for each ESN instance. As such there can only be one 2 per input dimension (column) as otherwise combining the different neighborhood’s predictions after each prediction step is not possible. In theory there can be no 2 in a dimension/column though. In that case the corresponding dimension just doesn’t appear in the prediction output.

The code is written under the assumption that all internal ESNs have the same hyperparameters. This does not necessarily need to be the case though, and a more generalized implementation would allow it.

Methods:

`create_network`(**kwargs)	Same idea as for the ESN Wrapper Class
`get_last_r_states`()	Returns:
`predict`(x_pred, sync_steps[, pred_steps, ...])	Same idea as for the ESN Wrapper Class
`set_console_logger`(log_level)	Set loglevel for the console output
`set_file_logger`(log_level, log_file_path)	Set's the logging file path
`synchronize`(x_sync, **kwargs)	Same idea as for the ESN Wrapper Class
`to_pickle`(path[, compression, protocol])	Pickle (serialize) object to file.
`train`(x_train, sync_steps[, loc_nbhds, ...])	Same idea as for the ESN Wrapper Class
`train_and_predict`(x_data, train_sync_steps, ...)	Same idea as for the ESN Wrapper Class

create_network(**kwargs)¶: Same idea as for the ESN Wrapper Class

get_last_r_states()¶: Returns:

predict(x_pred, sync_steps, pred_steps=None, ESN_sync_kwargs=None, **kwargs)¶: Same idea as for the ESN Wrapper Class

set_console_logger(log_level)¶

Set loglevel for the console output

Parameters: () (log_level) – console loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels

set_file_logger(log_level, log_file_path)¶

Set’s the logging file path

Parameters

() (log_level) – file loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels
log_file_path (str) – path, including file type, to store the logfile in. E.g: “folder_structure/log_file.txt”

synchronize(x_sync, **kwargs)¶: Same idea as for the ESN Wrapper Class

to_pickle(path, compression='infer', protocol=5)¶

Pickle (serialize) object to file.

Disables logging as logging handlers can not be pickled. Uses pandas functions internally.

Parameters

path (str) – File path where the pickled object will be stored.
compression ({'infer', 'gzip', 'bz2', 'zip', 'xz', None}) – A string representing the compression to use in the output file. By default, infers from the file extension in specified path.
protocol (int) – Int which indicates which protocol should be used by the pickler. The possible values are 0, 1, 2, 3, 4. A negative value for the protocol parameter is equivalent to setting its value to HIGHEST_PROTOCOL.

train(x_train, sync_steps, loc_nbhds=None, train_core_only=True, ESN_train_kwargs=None)¶: Same idea as for the ESN Wrapper Class

train_and_predict(x_data, train_sync_steps, train_steps, loc_nbhds=None, pred_sync_steps=0, pred_steps=None, disc_steps=0, **kwargs)¶: Same idea as for the ESN Wrapper Class

ESN Class¶

class rescomp.esn.ESN¶

Implements basic RC functionality

This is to be written such that one can easily implement both “normal” RC as well as local RC, or other generalizations, by using this class as a building block to be called with the right training and prediction data

Methods:

`create_input_matrix`(x_dim[, w_in_scale, ...])	create the input matrix _w_in. Can be used to create the _w_in before "train", otherwise it is called in "train" :param x_dim: dimension of the input data :type x_dim: int :param w_in_scale: maximum absolute value of the (random) w_in elements :type w_in_scale: float :param w_in_sparse: If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018) :type w_in_sparse: bool :param w_in_orderd: If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible) :type w_in_orderd: bool.
`create_network`([n_dim, n_rad, n_avg_deg, ...])	Creates the internal network used as reservoir in RC
`get_instance_version`()	Returns the rescomp package version used to create the class instance
`get_network`()	Returns the network used as reservoir
`get_network_parameters`()	Returns all network parameters
`get_w_in`()	Returns the input matrix w_in
`get_w_out`()	Returns the output matrix w_out
`predict`([x_pred, sync_steps, pred_steps, ...])	Synchronize the reservoir, then predict the system evolution
`set_console_logger`(log_level)	Set loglevel for the console output
`set_file_logger`(log_level, log_file_path)	Set's the logging file path
`synchronize`(x[, save_r])	Synchronize the reservoir state with the input time series
`to_pickle`(path[, compression, protocol])	Pickle (serialize) object to file.
`train`(x_train, sync_steps[, reg_param, ...])	Synchronize, then train the reservoir

create_input_matrix(x_dim, w_in_scale=1.0, w_in_sparse=True, w_in_ordered=False)¶

create the input matrix _w_in. Can be used to create the _w_in before “train”, otherwise it is called in “train” :param x_dim: dimension of the input data :type x_dim: int :param w_in_scale: maximum absolute value of the (random) w_in

elements

Parameters

w_in_sparse (bool) – If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018)
w_in_orderd (bool) – If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible)

create_network(n_dim=500, n_rad=0.1, n_avg_deg=6.0, n_type_flag='erdos_renyi', network_creation_attempts=10)¶

Creates the internal network used as reservoir in RC

Parameters

n_dim (int) – Nr. of nodes in the network
n_rad (float) – Spectral radius of the network. Must be >0. Should be < 1 to satisfy the echo state property
n_avg_deg (float) – Average node degree (number of connections). Used in the random network generations
n_type_flag (int_or_str) –
Type of Network to be generated. Possible flags and their synonyms are:
- ”random”, “erdos_renyi”
- ”scale_free”, “barabasi_albert”
- ”small_world”, “watts_strogatz”
network_creation_attempts (int) – How often the network generation should be attempted. It can fail due to a not converging eigenvalue calculation

get_instance_version()¶

Returns the rescomp package version used to create the class instance

Returns: Rescomp package version
Return type: str

get_network()¶

Returns the network used as reservoir

Returns: network saved as scipy.sparse matrix
Return type: csr_matrix

get_network_parameters()¶

Returns all network parameters

Returns: Dictionary of network parameters
Return type: dict

get_w_in()¶

Returns the input matrix w_in

Returns: w_in
Return type: np.ndarray

get_w_out()¶

Returns the output matrix w_out

Returns: w_out
Return type: np.ndarray

predict(x_pred=None, sync_steps=0, pred_steps=None, save_r=False, save_input=False)¶

Synchronize the reservoir, then predict the system evolution

Changes self._last_r and self._last_r_gen to stay synchronized to the new system state

Parameters

x_pred (np.ndarray) – Input data used to synchronize the reservoir, and then used as comparison for the prediction by being returned as y_pred in the output
sync_steps (int) – How many steps to use for synchronization before the prediction starts
pred_steps (int) – How many steps to predict
save_r (bool) – If true, saves r(t) internally
save_input (bool) – If true, saves the input data internally

Returns

2-element tuple containing:

y_pred (np.ndarray): Predicted future states
y_test (np.ndarray_or_None): Data taken from the input to compare the prediction with. If the prediction were “perfect” y_pred and y_test would be equal. Be careful though, y_test might be shorter than y_pred, or even None, if pred_steps is not None

Return type

tuple

set_console_logger(log_level)¶

Set loglevel for the console output

Parameters: () (log_level) – console loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels

set_file_logger(log_level, log_file_path)¶

Set’s the logging file path

Parameters

() (log_level) – file loglevel as specified in: https://docs.python.org/3/library/logging.html#logging-levels
log_file_path (str) – path, including file type, to store the logfile in. E.g: “folder_structure/log_file.txt”

synchronize(x, save_r=False)¶

Synchronize the reservoir state with the input time series

This is usually done automatically in the training and prediction functions.

Parameters

x (np.ndarray) – Input data to be used for the synchronization, shape (T, d)
save_r (bool) – If true, saves and returns r

Returns

All r states if save_r is True, None if False

Return type

np.ndarray_or_None

to_pickle(path, compression='infer', protocol=5)¶

Pickle (serialize) object to file.

Disables logging as logging handlers can not be pickled. Uses pandas functions internally.

Parameters

path (str) – File path where the pickled object will be stored.
compression ({'infer', 'gzip', 'bz2', 'zip', 'xz', None}) – A string representing the compression to use in the output file. By default, infers from the file extension in specified path.
protocol (int) – Int which indicates which protocol should be used by the pickler. The possible values are 0, 1, 2, 3, 4. A negative value for the protocol parameter is equivalent to setting its value to HIGHEST_PROTOCOL.

train(x_train, sync_steps, reg_param=1e-05, w_in_scale=1.0, w_in_sparse=True, w_in_ordered=False, w_in_no_update=False, act_fct_flag='tanh_simple', bias_scale=0, mix_ratio=0.5, save_r=False, save_input=False, w_out_fit_flag='simple', loc_nbhd=None)¶

Synchronize, then train the reservoir

Parameters

x_train (np.ndarray) – Input data used to synchronize and then train the reservoir
sync_steps (int) – How many steps to use for synchronization before the prediction starts
reg_param (float) – weight for the Tikhonov-regularization term
w_in_scale (float) – maximum absolute value of the (random) w_in elements
w_in_sparse (bool) – If true, creates w_in such that one element in each row is non-zero (Lu,Hunt, Ott 2018)
w_in_orderd (bool) – If true and w_in_sparse is true, creates w_in such that elements are ordered by dimension and number of elements for each dimension is equal (as far as possible)
w_in_no_update (bool) – If true and the input matrix W_in does already exist from a previous training run, W_in does not get updated, regardless of all other parameters concerning it.
act_fct_flag (int_or_str) –
Specifies the activation function to be used during training (and prediction). Possible flags and their synonyms are:
- ”tanh_simple”, “simple”
- ”tanh_bias”
bias_scale (float) – Bias to be used in some activation functions
mix_ratio (float) – Ratio of normal tanh vs squared tanh activation functions if act_fct_flag “mixed” is chosen
save_r (bool) – If true, saves r(t) internally
save_input (bool) – If true, saves the input data internally
w_out_fit_flag (str) – Type of nonlinear transformation applied to the reservoir states r to be used during the fit (and future prediction)
loc_nbhd (np.ndarray) – The local neighborhood used for the generalized local states approach. For more information, please see the docs.