Modely

Bases: Composer, Trainer, Loader, Validator, Exporter

Create the main object, the nnodely object, that will be used to create the network, train and export it.

Parameters:

visualizer (str, Visualizer, optional) – The visualizer to be used. Default is the ‘Standard’ visualizer.
exporter (str, Exporter, optional) – The exporter to be used. Default is the ‘Standard’ exporter.
seed (int, optional) – Set the seed for all the random modules inside the nnodely framework. Default is None.
workspace (str) – The path of the workspace where all the exported files will be saved.
log_internal (bool) – Whether or not save the logs. Default is False.
save_history (bool) – Whether or not save the history. Default is False.

Example

>>> model = Modely()

property constants

property internals

property json

property neuralized

property parameters

resetSeed(seed: int) → None[source]

Resets the random seed for reproducibility.

This method sets the seed for various random number generators used in the project to ensure reproducibility of results.

Parameters:: seed (int) – The seed value to be used for the random number generators.

Example

>>> model = nnodely()
>>> model.resetSeed(42)

property states

property traced

trainAndAnalyze(*, test_dataset: str | list | dict | None = None, test_batch_size: int = 128, **kwargs)[source]

Trains the model using the provided datasets and parameters. After training, it analyzes the results on the training, validation, and test datasets.

Parameters:

test_dataset (str or None, optional) – The name of the datasets used for test. Default is None.
test_batch_size (int, optional) – The batch size for testing. Default is 1.
models (list or None, optional) – A list of models to train. Default is None.
train_dataset (str or None, optional) – The name of datasets to use for training. Default is None.
validation_dataset (str or None, optional) – The name of datasets to use for validation. Default is None.
dataset (str or None, optional) – The name of the datasets to use for training, validation and test.
splits (list or None, optional) – A list of 3 elements specifying the percentage of splits for training, validation, and testing. The three elements must sum up to 100! The parameter splits is only used when dataset is not None
closed_loop (dict or None, optional) – A dictionary specifying closed loop connections. The keys are input names and the values are output names. Default is None.
connect (dict or None, optional) – A dictionary specifying connections. The keys are input names and the values are output names. Default is None.
step (int or None, optional) – The step size for training. A big value will result in less data used for each epochs and a faster train. Default is None.
prediction_samples (int or None, optional) – The size of the prediction horizon. Number of samples at each recurrent window Default is None.
shuffle_data (bool or None, optional) – Whether to shuffle the data during training. Default is None.
early_stopping (Callable or None, optional) – A callable for early stopping. Default is None.
early_stopping_params (dict or None, optional) – A dictionary of parameters for early stopping. Default is None.
select_model (Callable or None, optional) – A callable for selecting the best model. Default is None.
select_model_params (dict or None, optional) – A dictionary of parameters for selecting the best model. Default is None.
minimize_gain (dict or None, optional) – A dictionary specifying the gain for each minimization loss function. Default is None.
num_of_epochs (int or None, optional) – The number of epochs to train the model. Default is None.
train_batch_size (int or None, optional) – The batch size for training. Default is None.
val_batch_size (int or None, optional) – The batch size for validation. Default is None.
optimizer (Optimizer or None, optional) – The optimizer to use for training. Default is None.
lr (float or None, optional) – The learning rate. Default is None.
lr_param (dict or None, optional) – A dictionary of learning rate parameters. Default is None.
optimizer_params (list or dict or None, optional) – A dictionary of optimizer parameters. Default is None.
optimizer_defaults (dict or None, optional) – A dictionary of default optimizer settings. Default is None.
training_params (dict or None, optional) – A dictionary of training parameters. Default is None.
add_optimizer_params (list or None, optional) – Additional optimizer parameters. Default is None.
add_optimizer_defaults (dict or None, optional) – Additional default optimizer settings. Default is None.

Raises:

RuntimeError – If no data is loaded or if there are no modules with learnable parameters.
KeyError – If the sample horizon is not positive.
ValueError – If an input or output variable is not in the model definition.

Examples

Example - basic feed-forward training:

>>> x = Input('x')
>>> F = Input('F')

>>> xk1 = Output('x[k+1]', Fir()(x.tw(0.2))+Fir()(F.last()))

>>> mass_spring_damper = Modely(seed=0)
>>> mass_spring_damper.addModel('xk1',xk1)
>>> mass_spring_damper.neuralizeModel(sample_time = 0.05)

>>> data_struct = ['time','x','dx','F']
>>> data_folder = os.path.join(os.path.dirname(os.path.realpath(__file__)),'dataset','data')
>>> mass_spring_damper.loadData(name='mass_spring_dataset', source=data_folder, format=data_struct, delimiter=';')

>>> params = {'num_of_epochs': 100,'train_batch_size': 128,'lr':0.001}
>>> mass_spring_damper.trainModel(splits=[70,20,10], training_params = params)

Example - recurrent training:

>>> x = Input('x')
>>> F = Input('F')

>>> xk1 = Output('x[k+1]', Fir()(x.tw(0.2))+Fir()(F.last()))

>>> mass_spring_damper = Modely(seed=0)
>>> mass_spring_damper.addModel('xk1',xk1)
>>> mass_spring_damper.addClosedLoop(xk1, x)
>>> mass_spring_damper.neuralizeModel(sample_time = 0.05)

>>> data_struct = ['time','x','dx','F']
>>> data_folder = os.path.join(os.path.dirname(os.path.realpath(__file__)),'dataset','data')
>>> mass_spring_damper.loadData(name='mass_spring_dataset', source=data_folder, format=data_struct, delimiter=';')

>>> params = {'num_of_epochs': 100,'train_batch_size': 128,'lr':0.001}
>>> mass_spring_damper.trainModel(splits=[70,20,10], prediction_samples=10, training_params = params)

nnodely.nnodely.clearNames(names: str | list | None = None)[source]

nnodely.nnodely.nnodely: alias of Modely

Model structured NN Inputs Outputs and Parameters

Input module

class nnodely.layers.input.Input(name: str, *, dimensions: int = 1)[source]

Represents an Input in the neural network model.

Parameters:

json_name (str) – The name of the JSON field to store the Input configuration.
name (str) – The name of the Input.
dimensions (int, optional) – The number of dimensions for the input. Default is 1.

json_name

The name of the JSON field to store the input configuration.

Type:: str

name

The name of the Input.

Type:: str

dim

A dictionary containing the dimensions of the Input.

Type:: dict

json

A dictionary containing the configuration of the Input.

Type:: dict

classmethod clearNames(names: str | list | None = None)

closedLoop(obj: Stream) → Input[source]

Update and return the current Input in a closed loop with a given Stream object.

Parameters:

obj (Stream) – The Stream object for update the Input.

Returns:

A Input with the connection to the obj Stream

Return type:

Input

Raises:

TypeError – If the provided object is not of type Input.
KeyError – If the Input variable is already connected.

connect(obj: Stream) → Input[source]

Update and return the current Input with a given Stream object.

Parameters:

obj (Stream) – The Stream object for update the Input.

Returns:

A Input with the connection to the obj Stream

Return type:

Input

Raises:

TypeError – If the provided object is not of type Input.
KeyError – If the Input variable is already connected.

last() → Stream[source]

Selects the last passed instant for the input.

Returns:: A Stream representing the SamplePart object with the last passed instant.
Return type:: Stream

names = ['SampleTime']

next() → Stream[source]

Selects the next instant for the input.

Returns:: A Stream representing the SamplePart object with the next instant.
Return type:: Stream

s(order: int, *, der_name: str | None = None, int_name: str | None = None, method: str = 'euler') → Stream[source]

Considering the Laplace transform notation. The function is used to operate an integral or derivate operation on the input. The order of the integral or the derivative operation is indicated by the order parameter.

Parameters:

order (int) – Order of the Laplace transform
method (str, optional) – Integration or derivation method

Returns:

A Stream of the signal represents the integral or derivation operation.

Return type:

Stream

sw(sw: int | list, offset: int | None = None) → Stream[source]

Selects a sample window for the Input.

Parameters:

sw (list, int) – The sample window. If a list, it should contain the start and end indices. If an int, it represents the number of steps in the past.
offset (int, optional) – The offset for the sample window. Default is None.

Returns:

A Stream representing the SamplePart object with the selected samples.

Return type:

Stream

Raises:

TypeError – If the sample window is not an integer or a list of integers.

Examples

Select a sample window considering a signal T = [-3,-2,-1,0,1,2] where the time vector 0 represent the last passed instant. If sw is an integer #1 represent the number of step in the past

>>> T.sw(2) #= [-1, 0] represents two sample step in the past

If sw is a list [#1,#2] the numbers represent the sample indexes in the vector with the second element excluded

>>> T.sw([-2,0])  #= [-1, 0] represents two time step in the past zero in the future
>>> T.sw([0,1])   #= [1]     the first time in the future
>>> T.sw([-4,-2]) #= [-3,-2]

The total number of samples can be computed #2-#1. The offset represent the index of the vector that need to be used to offset the window

>>> T.sw(2,offset=-2)       #= [0, 1]      the value of the window is [-1,0]
>>> T.sw([-2,2],offset=-1)  #= [-1,0,1,2]  the value of the window is [-1,0,1,2]

tw(tw: int | float | list, offset: int | float | None = None) → Stream[source]

Selects a time window for the Input.

Parameters:

tw (list or float) – The time window. If a list, it should contain the start and end values. If a float, it represents the time window size.
offset (float, optional) – The offset for the time window. Default is None.

Returns:

A Stream representing the TimePart object with the selected time window.

Return type:

Stream

Raises:

ValueError – If the time window is not positive.
IndexError – If the offset is not within the time window.

z(delay: int) → Stream[source]

Considering the Zeta transform notation. The function is used to selects a unitary delay from the Input.

Parameters:: delay (int) – The delay value.
Returns:: A Stream representing the SamplePart object with the selected delay.
Return type:: Stream

Examples

Select the unitary delay considering a signal T = [-3,-2,-1,0,1,2], where the time vector 0 represent the last passed instant: T.z(-1) = 1 T.z(0) = 0 # the last passed instant T.z(2) = -2

Output module

class nnodely.layers.output.Output(name: str, relation: Stream)[source]

Represents an output in the neural network model. This relation is what the network will give as output during inference.

Parameters:

name (str) – The name of the output.
relation (Stream) – The relation to be used for the output.

name

The name of the output.

Type:: str

json

A dictionary containing the configuration of the output.

Type:: dict

dim

A dictionary containing the dimensions of the output.

Type:: dict

classmethod clearNames(names: str | list | None = None)

names = ['SampleTime']

Parameter module

Represents a constant value in the neural network model.

Parameters:

name (str) – The name of the constant.
values (list, float, int, or np.ndarray) – The values of the constant.
tw (float or int, optional) – The time window for the constant. Default is None.
sw (int, optional) – The sample window for the constant. Default is None.

name

The name of the constant.

Type:: str

dim

A dictionary containing the dimensions of the constant.

Type:: dict

json

A dictionary containing the configuration of the constant.

Type:: dict

Examples

Example - passing a custom scalar value -> g.dim = {‘dim’: 1}:

>>> g = Constant('gravity',values=9.81)

Example - passing a custom vector value -> n.dim = {‘dim’: 4}:

>>> n = Constant('numbers', values=[1,2,3,4])

Example - passing a custom vector value with single sample window -> n.dim = {‘dim’: 4, ‘sw’: 1}:

>>> n = Constant('numbers', values=[[1,2,3,4]])

Example - passing a custom vector value with double sample window -> n.dim = {‘dim’: 4, ‘sw’: 2}:

>>> n = Constant('numbers', values=[[2,3,4],[1,2,3]])

Example - passing a custom vector value with double sample window -> n.dim = {‘dim’: 4, ‘sw’: 2}. If the value of the sw is differnt from the dimension of shape[0] an error will be raised.

>>> n = Constant('numbers', sw = 2, values=[[2,3,4],[1,2,3]])

Example - passing a custom vector value with time window -> n.dim = {‘dim’: 4, ‘tw’: 4}. In this case the samplingtime must be 0.5 otherwise an error will be raised. If the Constant have a time dimension, the input must have a len(shape) == 2.

>>> n = Constant('numbers', tw = 4, values=[[2,3,4],[1,2,3]])

class nnodely.layers.parameter.SampleTime[source]

Represents a constant value that is equal to the sample time.

name

The name of the constant.

Type:: str

dim

A dictionary containing the dimensions of the constant.

Type:: dict

json

A dictionary containing the configuration of the constant.

Type:: dict

Example

>>> dt = SampleTime()

Represents a trainable parameter in the neural network model.

Notes

Note

You can find some initialization functions for the ‘init’ and ‘init_params’ parameters inside the initializer module.

Parameters:

name (str) – The name of the parameter.
dimensions (int, list, tuple, or None, optional) – The dimensions of the parameter. Default is None.
tw (float or int, optional) – The time window for the parameter. Default is None.
sw (int, optional) – The sample window for the parameter. Default is None.
values (list, float, int, np.ndarray, or None, optional) – The values by which initialize the parameter. Default is None.
init (Callable, optional) – A callable for initializing the parameter values. Default is None.
init_params (dict, optional) – A dictionary of parameters for the initializer. Default is None.

name

The name of the parameter.

Type:: str

dim

A dictionary containing the dimensions of the parameter.

Type:: dict

json

A dictionary containing the configuration of the parameter.

Type:: dict

Examples

Example - basic usage:

>>> k = Parameter('k', dimensions=3, tw=4)

Example - initialize a parameter with values:

>>> x = Input('x')
>>> gravity = Parameter('g', dimensions=(4,1),values=[[[1],[2],[3],[4]]])
>>> out = Output('out', Linear(W=gravity)(x.sw(3)))

Example - initialize a parameter with a function:

>>> x = Input('x').last()
>>> p = Parameter('param', dimensions=1, sw=1, init=init_constant, init_params={'value':1})
>>> relation = Fir(parameter=param)(x)

Initializer module

nnodely.support.initializer.init_constant(indexes, params_size, dict_param={'value': 1})[source]

Initializes parameters to a constant value.

Parameters:

dict_param (dict, optional) –

Dictionary containing the initialization parameters. Default is {‘value’: 1}.

valueint or float
The constant value to initialize the parameters with.

nnodely.support.initializer.init_negexp(indexes, params_size, dict_param={'first_value': 1, 'lambda': 3, 'size_index': 0})[source]

Initializes parameters using a negative decay exponential function.

Parameters:

indexes (list) – List of indexes for the parameters.

params_size (list) – List of sizes for each dimension of the parameters.

dict_param (dict, optional) –

Dictionary containing the initialization parameters. Default is {‘size_index’: 0, ‘first_value’: 1, ‘lambda’: 3}.

size_indexint
The index of the dimension to apply the exponential function.

first_valueint or float
The value at the start of the range.

lambdaint or float
The decay rate parameter of the exponential function.

nnodely.support.initializer.init_exp(indexes, params_size, dict_param={'lambda': 3, 'max_value': 1, 'monotonicity': 'decreasing', 'size_index': 0})[source]

Initializes parameters using an increasing or decreasing exponential function.

Parameters:

indexes (list) – List of indexes for the parameters.

params_size (list) – List of sizes for each dimension of the parameters.

dict_param (dict, optional) –

Dictionary containing the initialization parameters. Default is {‘size_index’: 0, ‘max_value’: 1, ‘lambda’: 3, ‘monotonicity’: ‘decreasing’}.

size_indexint
The index of the dimension to apply the exponential function.

max_valueint or float
The maximum value of the exponential function.

lambdaint or float
The rate parameter of the exponential function.

monotonicitystr
The monotonicity of the exponential function. Can be ‘increasing’ or ‘decreasing’.

Raises:

ValueError – If the monotonicity is not ‘increasing’ or ‘decreasing’.

nnodely.support.initializer.init_lin(indexes, params_size, dict_param={'first_value': 1, 'last_value': 0, 'size_index': 0})[source]

Initializes parameters using a linear function.

Parameters:

indexes (list) – List of indexes for the parameters.

params_size (list) – List of sizes for each dimension of the parameters.

dict_param (dict, optional) –

Dictionary containing the initialization parameters. Default is {‘size_index’: 0, ‘first_value’: 1, ‘last_value’: 0}.

size_indexint
The index of the dimension to apply the linear function.

first_valueint or float
The value at the start of the range.

last_valueint or float
The value at the end of the range.

Relation module

class nnodely.basic.relation.Stream(name, json, dim, count=1)[source]

Represents a stream of data inside the neural network. A Stream is automatically create when you operate over a Input, Parameter, or Constant object.

closedLoop(obj) → Stream[source]

Update the Stream adding a closed loop connection with a given input object.

Parameters:

obj (Input) – The Input object to create a closed loop with.

Returns:

A Stream of the signal that updates the Inputs with the connection.

Return type:

Stream

Raises:

TypeError – If the provided object is not of type Input.
KeyError – If the input variable is already connected.

connect(obj) → Stream[source]

Update the Stream adding a connects with a given input object.

Parameters:

obj (Input) – The Input object to connect to.

Returns:

A Stream of the signal that updates the Inputs with the connection.

Return type:

Stream

Raises:

TypeError – If the provided object is not of type Input.
KeyError – If the input variable is already connected.

delay(delay: int | float, *, name: str | None = None) → Stream[source]

The function is used to delay a Stream. The value of the delay can be only positive.

Parameters:: delay (int, float) – The delay value.
Returns:: A Stream representing the delayed Stream
Return type:: Stream

s(order: int, *, int_name: str | None = None, der_name: str | None = None, method: str = 'euler') → Stream[source]

Considering the Laplace transform notation. The function is used to operate an integral or derivate operation on a Stream. The order of the integral or the derivative operation is indicated by the order parameter.

Parameters:

order (int) – Order of the Laplace transform
method (str, optional) – Integration or derivation method

Returns:

A Stream of the signal represents the integral or derivation operation.

Return type:

Stream

sw(sw: int | list, offset: int | None = None, *, name: str | None = None) → Stream[source]

Selects a sample window on Stream. It is possible to create a smaller or bigger window on the stream. The Sample Window must be in the past not in the future.

Parameters:

sw (int, list) – The sample window represents the number of steps in the past. If a list, it should contain the start and end indices, both indexes must be in the past.
offset (int, optional) – The offset for the sample window. Default is None.
name (str, None) – The name of the internal variable

Returns:

A Stream representing the SamplePart object with the selected samples.

Return type:

Stream

Selects a time window on Stream. It is possible to create a smaller or bigger time window on the stream. The Time Window must be in the past not in the future.

Parameters:

tw (float, int, list) – The time window represents the time in the past. If a list, it should contain the start and end times, both indexes must be in the past.
offset (float, int, optional) – The offset for the sample window. Default is None.
name (str, None) – The name of the internal variable

Returns:

A Stream representing the TimePart object with the selected time window.

Return type:

Stream

z(delay: int | float, *, name: str | None = None) → Stream[source]

Considering the Zeta transform notation. The function is used to delay a Stream. The value of the delay can be only positive.

Parameters:: delay (int) – The delay value.
Returns:: A Stream representing the delayed Stream
Return type:: Stream

Model structured NN building blocks

Activation module

class nnodely.layers.activation.Relu(*args, **kwargs)[source]

Implement the Rectified-Linear Unit (ReLU) relation function.

See also

Official PyTorch ReLU documentation: torch.nn.ReLU

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = Relu(x)

class nnodely.layers.activation.ELU(*args, **kwargs)[source]

Implement the Exponential-Linear Unit (ELU) relation function.

See also

Official PyTorch ReLU documentation: torch.nn.ELU

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = ELU(x)

class nnodely.layers.activation.Sigmoid(*args, **kwargs)[source]

Implement the Sigmoid relation function. The Sigmoid function is defined as:

See also

Official PyTorch Softmax documentation: Sigmoid function

\[\sigma(x) = \frac{1}{1 + e^{-x}}\]

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = Sigmoid(x)

class nnodely.layers.activation.Softmax(*args, **kwargs)[source]

Implement the Softmax relation function.

See also

Official PyTorch Softmax documentation: torch.nn.Softmax

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = Softmax(x)

class nnodely.layers.activation.Identity(*args, **kwargs)[source]

Implement the Identity relation function that simply returns the input vector x.

See also

Official PyTorch Identity documentation: torch.nn.Identity

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = Identity(x)

Arithmetic module

class nnodely.layers.arithmetic.Add(*args, **kwargs)[source]

Implement the addition function between two tensors. (it is also possible to use the classical math operator ‘+’)

See also

Official PyTorch Add documentation: torch.add

Parameters:

input1 – the first element of the addition
input2 – the second element of the addition

Example

>>> add = Add(relation1, relation2)
or
>>> add = relation1 + relation2

class nnodely.layers.arithmetic.Sub(*args, **kwargs)[source]

Implement the subtraction function between two tensors. (it is also possible to use the classical math operator ‘-‘)

Parameters:

input1 – the first element of the subtraction
input2 – the second element of the subtraction

Example

>>> sub = Sub(relation1, relation2)
or
>>> sub = relation1 - relation2

class nnodely.layers.arithmetic.Mul(*args, **kwargs)[source]

Implement the multiplication function between two tensors. (it is also possible to use the classical math operator ‘*’)

Parameters:

input1 – the first element of the multiplication
input2 – the second element of the multiplication

Example

>>> mul = Mul(relation1, relation2)
or
>>> mul = relation1 * relation2

class nnodely.layers.arithmetic.Div(*args, **kwargs)[source]

Implement the division function between two tensors. (it is also possible to use the classical math operator ‘/’)

Parameters:

input1 – the numerator of the division
input2 – the denominator of the division

Example

>>> div = Div(relation1, relation2)
or
>>> div = relation1 / relation2

class nnodely.layers.arithmetic.Pow(*args, **kwargs)[source]

Implement the power function given an input and an exponent. (it is also possible to use the classical math operator ‘**’)

See also

Official PyTorch pow documentation: torch.pow

Parameters:

input – the base of the power function
exp – the exponent of the power function

Example

>>> pow = Pow(relation, exp)
or
>>> pow = relation1 ** relation2

class nnodely.layers.arithmetic.Neg(*args, **kwargs)[source]

Implement the negate function given an input.

Parameters:: input – the input to negate

Example

>>> x = Neg(x)

Fir module

class nnodely.layers.fir.Fir(*args, **kwargs)[source]

Represents a Finite Impulse Response (FIR) relation in the neural network model.

Notes

Note

The FIR relation works along the time dimension (second dimension) of the input tensor. You can find some initialization functions inside the initializer module.

Parameters:

output_dimension (int, optional) – The output dimension of the FIR relation.
W_init (Callable, str, optional) – A callable for initializing the parameters.
W_init_params (dict, optional) – A dictionary of parameters for the parameter initializer.
b_init (Callable, str, optional) – A callable for initializing the bias.
b_init_params (dict, optional) – A dictionary of parameters for the bias initializer.
W (Parameter or str, optional) – The parameter object or tag. The parameter can be defined using the relative class ‘Parameter’. If not given a new parameter will be auto-generated.
b (bool, str, or Parameter, optional) – The bias parameter object, tag, or a boolean indicating whether to use bias. If set to ‘True’ a new parameter will be auto-generated.
dropout (int or float, optional) – The dropout rate. Default is 0.

relation_name

The name of the relation.

Type:: str

W_init

The parameter initializer.

Type:: Callable

W_init_params

The parameters for the parameter initializer.

Type:: dict

W

The parameter object or name.

Type:: Parameter or str

b_init

The bias initializer.

Type:: Callable

b_init_params

The parameters for the bias initializer.

Type:: dict

b

The bias object, name, or a boolean indicating whether to use bias.

Type:: bool, str, or Parameter

pname

The name of the parameter.

Type:: str

bname

The name of the bias.

Type:: str

dropout

The dropout rate.

Type:: int or float

output_dimension

The output dimension of the FIR relation.

Type:: int

Examples

Example - basic usage:

>>> input = Input('in')
>>> relation = Fir(input.tw(0.05))

Example - passing a parameter:

>>> input = Input('in')
>>> par = Parameter('par', dimensions=3, sw=2, init='init_constant')
>>> relation = Fir(W=par)(input.sw(2))

Example - parameters initialization:

>>> x = Input('x')
>>> F = Input('F')
>>> fir_x = Fir(W_init='init_negexp')(x.tw(0.2))
>>> fir_F = Fir(W_init='init_constant', W_init_params={'value':1})(F.last())

Fuzzify module

Represents a Fuzzify relation in the neural network model.

Parameters:

output_dimension (int, optional) – The output dimension of the Fuzzify relation. If provided, range must also be provided and centers must be None.
range (list, optional) – A list containing the start and end values for the range. Required if output_dimension is provided.
centers (list, optional) – A list of center values for the fuzzy functions. Required if output_dimension is None. The output_dimension will be inferred from the number of centers provided.
functions (str, list, or Callable, optional) – The fuzzy functions to use. Can be a string specifying a predefined function type, a custom function, or a list of callable functions. Default is ‘Triangular’.

Notes

Note

The predefined function types are ‘Triangular’ and ‘Rectangular’. It is also possible to pass a list of custom functions. In this case, each center will be associated with the respective function in the list.

relation_name

The name of the relation.

Type:: str

output_dimension

The output dimension of the Fuzzify relation.

Type:: dict

json

A dictionary containing the configuration of the Fuzzify relation.

Type:: dict

Examples

Example - basic usage:

>>> x = Input('x')
>>> fuz = Fuzzify(output_dimension=5, range=[1,5])
>>> out = Output('out', fuz(x.last()))

Example - passing the centers:

>>> fuz = Fuzzify(centers=[-1,0,3,5], functions='Rectangular')
>>> out = Output('out', fuz(x.last()))

Example - using a custom function:

>>> def fun(x):
>>>     import torch
>>>     return torch.tanh(x)
>>> fuz = Fuzzify(output_dimension=11, range=[-5,5], functions=fun)
>>> out = Output('out', fuz(x.last()))

Equation Learner module

class nnodely.layers.equationlearner.EquationLearner(functions: list, *, linear_in: Linear | None = None, linear_out: Linear | None = None)[source]

Represents a nnodely implementation of the Task-Parametrized Equation Learner block.

See also

Task-Parametrized Equation Learner official paper: Equation Learner

Parameters:

functions (list) – A list of callable functions to be used as activation functions.
linear_in (Linear, optional) – A Linear layer to process the input before applying the activation functions. If not provided a random initialized linear layer will be used instead.
linear_out (Linear, optional) – A Linear layer to process the output after applying the activation functions. Can be omitted.

relation_name

The name of the relation.

Type:: str

linear_in

The Linear layer to process the input.

Type:: Linear or None

linear_out

The Linear layer to process the output.

Type:: Linear or None

functions

The list of activation functions.

Type:: list

func_parameters

A dictionary mapping function indices to the number of parameters they require.

Type:: dict

n_activations

The total number of activation functions.

Type:: int

Examples

Example - basic usage:

>>> x = Input('x')

>>> equation_learner = EquationLearner(functions=[Tan, Sin, Cos])
>>> out = Output('out',equation_learner(x.last()))

Example - passing a linear layer:

>>> x = Input('x')

>>> linear_layer = Linear(output_dimension=3, W_init=init_constant, W_init_params={'value':0})
>>> equation_learner = EquationLearner(functions=[Tan, Sin, Cos], linear_in=linear_layer)

>>> out = Output('out',equation_learner(x.last()))

Example - passing a custom parametric function and multiple inputs:

>>> x = Input('x')
>>> F = Input('F')

>>> def myFun(K1,p1):
        return K1*p1

>>> K = Parameter('k', dimensions =  1, sw = 1,values=[[2.0]])
>>> parfun = ParamFun(myFun, parameters = [K] )

>>> equation_learner = EquationLearner([parfun])
>>> out = Output('out',equation_learner((x.last(),F.last())))

Linear module

class nnodely.layers.linear.Linear(*args, **kwargs)[source]

Represents a Linear relation in the neural network model.

Notes

Note

The Linear relation works along the input dimension (third dimension) of the input tensor. You can find some initialization functions inside the initializer module.

Parameters:

output_dimension (int, optional) – The output dimension of the Linear relation.
W_init (Callable, optional) – A callable for initializing the weights.
W_init_params (dict, optional) – A dictionary of parameters for the weight initializer.
b_init (Callable, optional) – A callable for initializing the bias.
b_init_params (dict, optional) – A dictionary of parameters for the bias initializer.
W (Parameter or str, optional) – The weight parameter object or name. If not given a new parameter will be auto-generated.
b (bool, str, or Parameter, optional) – The bias parameter object, name, or a boolean indicating whether to use bias. If set to ‘True’ a new parameter will be auto-generated.
dropout (int or float, optional) – The dropout rate. Default is 0.

relation_name

The name of the relation.

Type:: str

W_init

The weight initializer.

Type:: Callable

W_init_params

The parameters for the weight initializer.

Type:: dict

b_init

The bias initializer.

Type:: Callable

b_init_params

The parameters for the bias initializer.

Type:: dict

W

The weight parameter object or name.

Type:: Parameter or str

b

The bias parameter object, name, or a boolean indicating whether to use bias.

Type:: bool, str, or Parameter

Wname

The name of the weight parameter.

Type:: str

bname

The name of the bias parameter.

Type:: str

dropout

The dropout rate.

Type:: int or float

output_dimension

The output dimension of the Linear relation.

Type:: int

Examples

Example - basic usage:

>>> input = Input('in').tw(0.05)
>>> relation = Linear(input)

Example - passing a weight and bias parameter:

>>> input = Input('in').last()
>>> weight = Parameter('W', values=[[[1]]])
>>> bias = Parameter('b', values=[[1]])
>>> relation = Linear(W=weight, b=bias)(input)

Example - parameters initialization:

>>> input = Input('in').last()
>>> relation = Linear(b=True, W_init=init_negexp, b_init=init_constant, b_init_params={'value':1})(input)

Localmodel module

class nnodely.layers.localmodel.LocalModel(input_function: Callable | None = None, output_function: Callable | None = None, *, pass_indexes: bool = False)[source]

Represents a Local Model relation in the neural network model.

Parameters:

input_function (Callable, optional) – A callable function to process the inputs.
output_function (Callable, optional) – A callable function to process the outputs.
pass_indexes (bool, optional) – A boolean indicating whether to pass indexes to the functions. Default is False.

relation_name

The name of the relation.

Type:: str

pass_indexes

A boolean indicating whether to pass indexes to the functions.

Type:: bool

input_function

The function to process the inputs.

Type:: Callable

output_function

The function to process the outputs.

Type:: Callable

Examples

Example - basic usage:

>>> x = Input('x')
>>> activation = Fuzzify(2,[0,1],functions='Triangular')(x.last())
>>> loc = LocalModel(input_function=Fir())
>>> out = Output('out', loc(x.tw(1), activation))

Example - passing a custom function:

>>> def myFun(in1,p1,p2):
>>>     return p1*in1+p2

>>> x = Input('x')
>>> activation = Fuzzify(2,[0,1],functions='Triangular')(x.last())
>>> loc = LocalModel(input_function = lambda:ParamFun(myFun), output_function = lambda:Fir)(x.last(), activation)
>>> out = Output('out', loc)

Example - custom function with multiple activations:

>>> x = Input('x')
>>> F = Input('F')
>>> activationA = Fuzzify(2,[0,1],functions='Triangular')(x.tw(1))
>>> activationB = Fuzzify(2,[0,1],functions='Triangular')(F.tw(1))

>>> def myFun(in1,p1,p2):
>>>     return p1*in1+p2

>>> loc = LocalModel(input_function = lambda:ParamFun(myFun), output_function = Fir(3))(x.tw(1),(activationA,activationB))
>>> out = Output('out', loc)

Part module

class nnodely.layers.part.Part(*args, **kwargs)[source]

Represents a selection of a sub-part from a relation in the neural network model.

Notes

Note

The Part relation works along the object dimension (third dimension) of the input.

Parameters:

obj (Stream) – The stream object to create a part from.
i (int) – The starting index of the part.
j (int) – The ending index of the part.

name

The name of the part.

Type:: str

dim

A dictionary containing the dimensions of the part.

Type:: dict

json

A dictionary containing the configuration of the part.

Type:: dict

Examples

Example

>>> x = Input('x', dimensions=3).last()
>>> relation = Part(x, 0, 1)

Raises:: IndexError – If the indices i and j are out of range.

class nnodely.layers.part.Select(*args, **kwargs)[source]

Represents a selection of a single element from a relation in the neural network model.

Notes

Note

The Select relation works along the object dimension (third dimension) of the input.

Parameters:

obj (Stream) – The stream object to select an element from.
i (int) – The index of the element to select.

name

The name of the selection.

Type:: str

dim

A dictionary containing the dimensions of the selection.

Type:: dict

json

A dictionary containing the configuration of the selection.

Type:: dict

Examples

Example

>>> x = Input('x', dimensions=3).last()
>>> relation = Select(x, 1)

Raises:: IndexError – If the index i is out of range.

class nnodely.layers.part.SamplePart(*args, **kwargs)[source]

Represents a selection of a sub-part from a relation in the neural network model.

Notes

Note

The SamplePart relation works along the time dimension (second dimension) of the input.

Parameters:

obj (Stream) – The stream object to create a part from.
i (int) – The starting index of the part.
j (int) – The ending index of the part.
offset (int, optional) – The offset for the part. Default is None.

name

The name of the part.

Type:: str

dim

A dictionary containing the dimensions of the part.

Type:: dict

json

A dictionary containing the configuration of the part.

Type:: dict

Examples

Example

>>> x = Input('x').sw(3)
>>> relation = SamplePart(x, 0, 1)

Raises:

KeyError – If the input does not have a sample window.
ValueError – If the indices i and j are out of range or if i is not smaller than j.
IndexError – If the offset is not within the sample window.

class nnodely.layers.part.SampleSelect(*args, **kwargs)[source]

Represents a selection of a single element from a relation in the neural network model.

Notes

Note

The SampleSelect relation works along the time dimension (second dimension) of the input.

Parameters:

obj (Stream) – The stream object to select an element from.
i (int) – The index of the element to select.

name

The name of the selection.

Type:: str

dim

A dictionary containing the dimensions of the selection.

Type:: dict

json

A dictionary containing the configuration of the selection.

Type:: dict

Examples

Example

>>> x = Input('x').sw(3)
>>> relation = SampleSelect(x, 1)

Raises:

IndexError – If the index i is out of range.
KeyError – If the input does not have a sample window.
IndexError – If the offset is not within the sample window.

class nnodely.layers.part.TimePart(*args, **kwargs)[source]

Represents a part of a stream in the neural network model along the time dimension (second dimension).

Parameters:

obj (Stream) – The stream object to create a part from.
i (int or float) – The starting index of the part.
j (int or float) – The ending index of the part.
offset (int or float, optional) – The offset for the part. Default is None.

name

The name of the part.

Type:: str

dim

A dictionary containing the dimensions of the part.

Type:: dict

json

A dictionary containing the configuration of the part.

Type:: dict

Examples

Example

>>> x = Input('x').sw(10)
>>> time_part = TimePart(x, i=0, j=5)

Raises:

KeyError – If the input does not have a time window.
ValueError – If the indices i and j are out of range or if i is not smaller than j.
IndexError – If the offset is not within the time window.

Trigonometric module

class nnodely.layers.trigonometric.Sin(*args, **kwargs)[source]

Implement the sine function given an input relation.

See also

Official PyTorch Sin documentation: torch.sin

Parameters:: obj (Stream) – the input relation stream

Example

>>> sin = Sin(relation)

class nnodely.layers.trigonometric.Cos(*args, **kwargs)[source]

Implement the cosine function given an input relation.

See also

Official PyTorch Cos documentation: torch.cos

Parameters:: obj (Stream) – the input relation stream

Example

>>> cos = Cos(relation)

class nnodely.layers.trigonometric.Tan(*args, **kwargs)[source]

Implement the tangent function given an input relation.

See also

Official PyTorch Tan documentation: torch.tan

Parameters:: obj (Stream) – the input relation stream

Example

>>> tan = Tan(relation)

class nnodely.layers.trigonometric.Tanh(*args, **kwargs)[source]

Implement the Hyperbolic Tangent (Tanh) relation function.

See also

Official PyTorch tanh documentation: torch.nn.Tanh

Parameters:: obj (Stream) – The relation stream.

Example

>>> x = Tanh(x)

class nnodely.layers.trigonometric.Cosh(*args, **kwargs)[source]

Returns a new tensor with the hyperbolic cosine of the elements of input.

See also

Official PyTorch Cosh documentation: torch.cosh

Parameters:: obj (Stream) – the input relation stream

Example

>>> cosh = Cosh(relation)

class nnodely.layers.trigonometric.Sech(*args, **kwargs)[source]

Returns a new tensor with the hyperbolic secant of the elements of input.

Parameters:: obj (Stream) – the input relation stream

Example

>>> sech = Sech(relation)

Parametric Function module

class nnodely.layers.parametricfunction.ParamFun(param_fun: Callable, parameters_and_constants: list | dict | None = None, *, map_over_batch: bool = False)[source]

Represents a parametric function in the neural network model.

Parameters:

param_fun (Callable) – The parametric function to be used.
constants (list or dict or None, optional) – A list or dictionary of constants to be used in the function. Default is None.
parameters_dimensions (list or dict or None, optional) – A list or dictionary specifying the dimensions of the parameters. Default is None.
parameters (list or dict or None, optional) – A list or dictionary of parameters to be used in the function. Default is None.
map_over_batch (bool, optional) – A boolean indicating whether to map the function over the batch dimension. Default is False.

relation_name

The name of the relation.

Type:: str

param_fun

The parametric function to be used.

Type:: Callable

constants

A list or dictionary of constants to be used in the function.

Type:: list or dict or None

parameters_dimensions

A list or dictionary specifying the dimensions of the parameters.

Type:: list or dict or None

parameters

A list or dictionary of parameters to be used in the function.

Type:: list or dict or None

map_over_batch

A boolean indicating whether to map the function over the batch dimension.

Type:: bool

output_dimension

A dictionary containing the output dimensions of the function.

Type:: dict

json

A dictionary containing the configuration of the function.

Type:: dict

Examples

Example

>>> input1 = Input('input1')
>>> input2 = Input('input2')

>>> def my_function(x, y, param1, const1):
>>>     return param1 * x + const1 * y

>>> param_fun = ParamFun(my_function, constants={'const1': 1.0}, parameters_dimensions={'param1': 1})
>>> result = param_fun(input1, input2)

Training

optimizer module

class nnodely.basic.optimizer.SGD(optimizer_defaults={}, optimizer_params=[])[source]

Stochastic Gradient Descent (SGD) optimizer.

See also

Official PyTorch SGD documentation: torch.optim.SGD

Parameters:

name (str) – The name of the optimizer.
optimizer_defaults (dict) – A dictionary of default optimizer settings.
optimizer_params (list) – A list of parameter groups for the optimizer.

name

The name of the optimizer.

Type:: str

lr

Learning rate. Default is 0.01.

Type:: float, optional

momentum

Momentum factor. Default is 0.0.

Type:: float, optional

dampening

Dampening for momentum. Default is 0.0.

Type:: float, optional

weight_decay

Weight decay (L2 penalty). Default is 0.0.

Type:: float, optional

nesterov

Enables Nesterov momentum. Default is False.

Type:: bool, optional

add_defaults(option_name, params, overwrite=True)

Adds default settings to the optimizer.

Parameters:

option_name (str) – The name of the option to add.
params (any) – The parameters for the option.
overwrite (bool, optional) – Whether to overwrite existing settings. Default is True.

set_defaults(optimizer_defaults)

Sets the default optimizer settings.

Parameters:: optimizer_defaults (dict) – A dictionary of default optimizer settings.

set_params(optimizer_params)

Sets the parameter groups for the optimizer.

Parameters:: optimizer_params (list) – A list of parameter groups for the optimizer.

set_params_to_train(all_params, params_to_train)

Sets the parameters to be trained by the optimizer.

Parameters:

all_params (dict) – A dictionary of all parameters in the model.
params_to_train (list) – A list of parameters to be trained.

unfold(params)

Unfolds the parameter groups into a flat list.

Parameters:: params (list) – A list of parameter groups.
Returns:: A flat list of parameter groups.
Return type:: list
Raises:: KeyError – If the params argument is not a list.

class nnodely.basic.optimizer.Adam(optimizer_defaults={}, optimizer_params=[])[source]

Stochastic Gradient Descent (SGD) optimizer.

See also

Official PyTorch Adam documentation: torch.optim.Adam

Parameters:

name (str) – The name of the optimizer.
optimizer_defaults (dict) – A dictionary of default optimizer settings.
optimizer_params (list) – A list of parameter groups for the optimizer.

name

The name of the optimizer.

Type:: str

lr

Learning rate. Default is 0.001.

Type:: float, optional

betas

Coefficients used for computing running averages of gradient and its square. Default is (0.9, 0.999).

Type:: tuple of (float, float), optional

eps

Term added to the denominator to improve numerical stability. Default is 1e-8.

Type:: float, optional

weight_decay

Weight decay (L2 penalty). Default is 0.0.

Type:: float, optional

amsgrad

Whether to use the AMSGrad variant of this algorithm. Default is False.

Type:: bool, optional

add_defaults(option_name, params, overwrite=True)

Adds default settings to the optimizer.

Parameters:

option_name (str) – The name of the option to add.
params (any) – The parameters for the option.
overwrite (bool, optional) – Whether to overwrite existing settings. Default is True.

set_defaults(optimizer_defaults)

Sets the default optimizer settings.

Parameters:: optimizer_defaults (dict) – A dictionary of default optimizer settings.

set_params(optimizer_params)

Sets the parameter groups for the optimizer.

Parameters:: optimizer_params (list) – A list of parameter groups for the optimizer.

set_params_to_train(all_params, params_to_train)

Sets the parameters to be trained by the optimizer.

Parameters:

all_params (dict) – A dictionary of all parameters in the model.
params_to_train (list) – A list of parameters to be trained.

unfold(params)

Unfolds the parameter groups into a flat list.

Parameters:: params (list) – A list of parameter groups.
Returns:: A flat list of parameter groups.
Return type:: list
Raises:: KeyError – If the params argument is not a list.

earlystopping module

nnodely.support.earlystopping.early_stop_patience(train_losses, val_losses, params)[source]

Determines whether to stop training early based on validation loss and patience.

Parameters:

train_losses (dict) – A dictionary where keys are epoch numbers and values are lists of training loss values.
val_losses (dict) – A dictionary where keys are epoch numbers and values are lists of validation loss values.
params (dict) – A dictionary of parameters. Should contain ‘patience’ which is the number of epochs to wait for improvement. Optionally, it can contain ‘error’ which specifies the type of loss to be used.

Returns:

True if training should be stopped early, False otherwise.

Return type:

bool

nnodely.support.earlystopping.select_best_model(train_losses, val_losses, params)[source]

Selects the best model based on the validation or training losses.

Parameters:

train_losses (dict) – A dictionary where keys are epoch numbers and values are lists of training loss values.
val_losses (dict) – A dictionary where keys are epoch numbers and values are lists of validation loss values.
params (dict) – A dictionary of parameters.

Returns:

True if the current model is the best model, False otherwise.

Return type:

bool

nnodely.support.earlystopping.mean_stopping(train_losses, val_losses, params)[source]

Determines whether to stop training early based on the mean difference between training and validation losses.

Parameters:

train_losses (dict) – A dictionary where keys are epoch numbers and values are lists of training loss values.
val_losses (dict) – A dictionary where keys are epoch numbers and values are lists of validation loss values.
params (dict) – A dictionary of parameters. Should contain ‘tol’ which is the tolerance value for early stopping.

Returns:

True if training should be stopped early, False otherwise.

Return type:

bool

nnodely.support.earlystopping.standard_early_stopping(train_losses, val_losses, params)[source]

Determines whether to stop training early based on training and validation losses.

Parameters:

train_losses (dict) – A dictionary where keys are epoch numbers and values are lists of training loss values.
val_losses (dict) – A dictionary where keys are epoch numbers and values are lists of validation loss values.
params (dict) – A dictionary of parameters. Should contain ‘tol’ which is the tolerance value for early stopping.

Returns:

True if training should be stopped early, False otherwise.

Return type:

bool

Additional information

Overview on signal shape

In this section is explained the shape of the input/output of the network.

Input and output shape from the structured neural model

The structured network can be called in two way:

The shape of the inputs not sampled are [total time window size, dim] Sampled inputs are reconstructed as soon as the maximum size of the time window is known. ‘dim’ represents the size of the input if is not 1 means that the input is a vector.
The shape of the sampled inputs are [number of samples = batch, size of time window for a sample, dim] In the example presented before in the first call the shape for x are [1,5,1] for F are [1,1,1] in the second call for x are [2,5,1] for F are [2,1,1]. In both cases the last dimensions is ignored as the input are scalar. The output of the structured neural model The outputs are defined in this way for the different cases:
if the shape is [batch, 1, 1] the final two dimensions are collapsed result [batch]
if the shape is [batch, window, 1] the last dimension is collapsed result [batch, window]
if the shape is [batch, window, dim] the output is equal to [batch, window, dim]
if the shape is [batch, 1, dim] the output is equal to [batch, 1, dim] In the example x_z_est has the shape of [1] in the first call and [2] because the the window and the dim were equal to 1.

Shape of elementwise Arithmetic, Activation, Trigonometric

The shape and time windows remain unchanged, for the binary operators shape must be equal.

input shape = [batch, window, dim] -> output shape = [batch, window, dim]

Shape of Fir input/output

The input must be scalar, the fir compress di time dimension (window) that goes to 1. A vector input is not allowed. The output dimension of the Fir is moved on the last dimension for create a vector output.

input shape = [batch, window, 1] -> output shape = [batch, 1, output dimension of Fir = output_dimension]

Shape of Linear input/output

The window remains unchanged and the output dimension is user defined.

input shape = [batch, window, dimension] -> output shape = [batch, window, output dimension of Linear = output_dimension]

Shape of Fuzzy input/output

The function fuzzify the input and creates a vector for output. The window remains unchanged, input must be scalar. Vector input are not allowed.

input shape = [batch, window, 1] -> output shape = [batch, window, number of centers of Fuzzy = len(centers)]

Shape of Part and Select input/output

Part selects a slice of the vector input, the input must be a vector. Select operation the dimension becomes 1, the input must be a vector. For both operation if there is a time component it remains unchanged.

Part input shape = [batch, window, dimension] -> output shape = [batch, window, selected dimension = [j-i]]
Select input shape = [batch, window, dimension] -> output shape = [batch, window, 1]

Shape of TimePart, SimplePart, SampleSelect input/output

The TimePart selects a time window from the signal (works like timewindow tw([i,j]) but in this the i,j are absolute). The SamplePart selects a list of samples from the signal (works like samplewindow sw([i,j]) but in this the i,j are absolute). The SampleSelect selects a specific index from the signal (works like zeta operation z(index) but in this the index are absolute). For all the operation the shape remains unchanged.

SamplePart input shape = [batch, window, dimension] -> output shape = [batch, selected sample window = [j-i], dimension]
SampleSelect input shape = [batch, window, dimension] -> output shape = [batch, 1, dimension]
TimePart input shape = [batch, window, dimension] -> output shape = [batch, selected time window = [j-i]/sample_time, dimension]

Shape of LocalModel input/output

The local model has two main inputs, activation functions and inputs. Activation functions have shape of the fuzzy

input shape = [batch, window, 1] -> output shape = [batch, window, number of centers of Fuzzy = len(centers)]

Inputs go through input function and output function. The input shape of the input function can be anything as long as the output shape of the input function have the following dimensions [batch, window, 1] so input functions for example cannot be a Fir with output_dimension different from 1. The input shape of the output function is [batch, window, 1] while the shape of the output of the output functions can be any

Shape of Parameters input/output

Parameter shape are defined as follows [window = sw or tw/sample_time, dim] the dimensions can be defined as a tuple and are appended to window When the time dimension is not defined it is configured to 1

Shape of Parametric Function input/output

The Parametric functions take inputs and parameters as inputs Parameter dimensions are the same as defined by the parameters if the dimensions are not defined they will be equal to [window = 1,dim = 1] Dimensions of the inputs inside the parametric function are the same as those managed within the Pytorch framework equal to [batch, window, dim] Output dimensions must follow the same convention `[batch, window, dim]