Equation Learner Layer
Represents a nnodely implementation of the Task-Parametrized Equation Learner block.
Official Paper: Task-Parametrized Equation Learner
[1]:
# uncomment the command below to install the nnodely package
#!pip install nnodely
from nnodely import *
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>-- nnodely_v1.5.0 --<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Basic Usage
Create an simple equation learner block using nnodely trigonometric functions (Tan, Sin, Cos).
The initial linear layer is created using random initialization.
[3]:
x = Input('x')
equation_learner = EquationLearner(functions=[Tan, Sin, Cos])
Output('out',equation_learner(x.last()))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[3]:
==================================== Output ====================================
{'Constants': {},
'Functions': {},
'Info': {},
'Inputs': {'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate10'},
'Parameters': {'PLinear4W': {'dim': [1, 3]}, 'PLinear4b': {'dim': 3}},
'Relations': {'Concatenate10': ['Concatenate', ['Concatenate7', 'Cos9']],
'Concatenate7': ['Concatenate', ['Tan4', 'Sin6']],
'Cos9': ['Cos', ['Select8']],
'Linear2': ['Linear',
['SamplePart1'],
'PLinear4W',
'PLinear4b',
0],
'SamplePart1': ['SamplePart', ['x'], -1, [-1, 0]],
'Select3': ['Select', ['Linear2'], 3, 0],
'Select5': ['Select', ['Linear2'], 3, 1],
'Select8': ['Select', ['Linear2'], 3, 2],
'Sin6': ['Sin', ['Select5']],
'Tan4': ['Tan', ['Select3']]}}
--------------------------- out {'dim': 3, 'sw': 1} ----------------------------
Input layer
Create an simple equation learner block using nnodely trigonometric functions and passing an input layer.
In this case the ‘output_dimension’ must match the sum of number of inputs of the activation functions.
[4]:
x = Input('x')
input_layer = Linear(output_dimension=3)
equation_learner = EquationLearner(functions=[Tan, Sin, Cos], linear_in=input_layer)
Output('out', equation_learner(x.last()))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[4]:
==================================== Output ====================================
{'Constants': {},
'Functions': {},
'Info': {},
'Inputs': {'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate21'},
'Parameters': {'PLinear9W': {'dim': [1, 3]}},
'Relations': {'Concatenate18': ['Concatenate', ['Tan15', 'Sin17']],
'Concatenate21': ['Concatenate', ['Concatenate18', 'Cos20']],
'Cos20': ['Cos', ['Select19']],
'Linear13': ['Linear', ['SamplePart12'], 'PLinear9W', None, 0],
'SamplePart12': ['SamplePart', ['x'], -1, [-1, 0]],
'Select14': ['Select', ['Linear13'], 3, 0],
'Select16': ['Select', ['Linear13'], 3, 1],
'Select19': ['Select', ['Linear13'], 3, 2],
'Sin17': ['Sin', ['Select16']],
'Tan15': ['Tan', ['Select14']]}}
--------------------------- out {'dim': 3, 'sw': 1} ----------------------------
Input layer and output layer
Create an simple equation learner block using nnodely trigonometric functions and passing an input layer and also a linear output layer
(By default, there is no linear output layer)
[5]:
x = Input('x')
input_layer = Linear(output_dimension=3)
output_layer = Linear(output_dimension=1)
equation_learner = EquationLearner(functions=[Tan, Sin, Cos], linear_in=input_layer, linear_out=output_layer)
Output('out', equation_learner(x.last()))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[5]:
==================================== Output ====================================
{'Constants': {},
'Functions': {},
'Info': {},
'Inputs': {'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Linear33'},
'Parameters': {'PLinear14W': {'dim': [1, 3]}, 'PLinear16W': {'dim': [3, 1]}},
'Relations': {'Concatenate29': ['Concatenate', ['Tan26', 'Sin28']],
'Concatenate32': ['Concatenate', ['Concatenate29', 'Cos31']],
'Cos31': ['Cos', ['Select30']],
'Linear24': ['Linear', ['SamplePart23'], 'PLinear14W', None, 0],
'Linear33': ['Linear', ['Concatenate32'], 'PLinear16W', None, 0],
'SamplePart23': ['SamplePart', ['x'], -1, [-1, 0]],
'Select25': ['Select', ['Linear24'], 3, 0],
'Select27': ['Select', ['Linear24'], 3, 1],
'Select30': ['Select', ['Linear24'], 3, 2],
'Sin28': ['Sin', ['Select27']],
'Tan26': ['Tan', ['Select25']]}}
--------------------------- out {'dim': 1, 'sw': 1} ----------------------------
Multiple inputs
Create an simple equation learner block using nnodely trigonometric functions and passing multiple inputs when calling the equation layer block.
All the given inputs will be concatenated before going through the linear input layer. The input and output dimensions of the input layer are 2, 3 respectively.
[6]:
x = Input('x')
F = Input('F')
equation_learner = EquationLearner(functions=[Tan, Sin, Cos])
Output('out',equation_learner(inputs=(x.last(),F.last())))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[6]:
==================================== Output ====================================
{'Constants': {},
'Functions': {},
'Info': {},
'Inputs': {'F': {'dim': 1, 'sw': [-1, 0]}, 'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate47'},
'Parameters': {'PLinear23W': {'dim': [2, 3]}, 'PLinear23b': {'dim': 3}},
'Relations': {'Concatenate38': ['Concatenate',
['SamplePart35', 'SamplePart37']],
'Concatenate44': ['Concatenate', ['Tan41', 'Sin43']],
'Concatenate47': ['Concatenate', ['Concatenate44', 'Cos46']],
'Cos46': ['Cos', ['Select45']],
'Linear39': ['Linear',
['Concatenate38'],
'PLinear23W',
'PLinear23b',
0],
'SamplePart35': ['SamplePart', ['x'], -1, [-1, 0]],
'SamplePart37': ['SamplePart', ['F'], -1, [-1, 0]],
'Select40': ['Select', ['Linear39'], 3, 0],
'Select42': ['Select', ['Linear39'], 3, 1],
'Select45': ['Select', ['Linear39'], 3, 2],
'Sin43': ['Sin', ['Select42']],
'Tan41': ['Tan', ['Select40']]}}
--------------------------- out {'dim': 3, 'sw': 1} ----------------------------
Multi-parameter functions
Create an equation learner block with functions that take 2 parameters (add, sub, mul …).
Be careful to the output dimension that the linear input layer should have to connect correctly all the activation functions.
In the example below, both the Add and Mul relations take 2 parameters so the total number of output dimension is 7 instead of 5.
[7]:
x = Input('x')
F = Input('F')
linear_layer_in_1 = Linear(output_dimension=7)
equation_learner_1 = EquationLearner(functions=[Tan, Add, Sin, Mul, Identity], linear_in=linear_layer_in_1)
Output('out',equation_learner_1(x.last()))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'F' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[7]:
==================================== Output ====================================
{'Constants': {},
'Functions': {},
'Info': {},
'Inputs': {'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate66'},
'Parameters': {'PLinear29W': {'dim': [1, 7]}},
'Relations': {'Add55': ['Add', ['Select53', 'Select54']],
'Concatenate56': ['Concatenate', ['Tan52', 'Add55']],
'Concatenate59': ['Concatenate', ['Concatenate56', 'Sin58']],
'Concatenate63': ['Concatenate', ['Concatenate59', 'Mul62']],
'Concatenate66': ['Concatenate',
['Concatenate63', 'Identity65']],
'Identity65': ['Identity', ['Select64']],
'Linear50': ['Linear', ['SamplePart49'], 'PLinear29W', None, 0],
'Mul62': ['Mul', ['Select60', 'Select61']],
'SamplePart49': ['SamplePart', ['x'], -1, [-1, 0]],
'Select51': ['Select', ['Linear50'], 7, 0],
'Select53': ['Select', ['Linear50'], 7, 1],
'Select54': ['Select', ['Linear50'], 7, 2],
'Select57': ['Select', ['Linear50'], 7, 3],
'Select60': ['Select', ['Linear50'], 7, 4],
'Select61': ['Select', ['Linear50'], 7, 5],
'Select64': ['Select', ['Linear50'], 7, 6],
'Sin58': ['Sin', ['Select57']],
'Tan52': ['Tan', ['Select51']]}}
--------------------------- out {'dim': 5, 'sw': 1} ----------------------------
Using custom parametric functions
Create an equation learner block with simple parametric functions
[8]:
import torch
def func1(K1):
return torch.sin(K1)
def func2(K2):
return torch.cos(K2)
x = Input('x')
parfun1 = ParamFun(func1)
parfun2 = ParamFun(func2)
equation_learner = EquationLearner([parfun1, parfun2])
Output('out',equation_learner(x.last()))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[8]:
==================================== Output ====================================
{'Constants': {},
'Functions': {'FParamFun34': {'code': 'def func1(K1):\n'
' return torch.sin(K1)\n',
'in_dim': [{'dim': 1, 'sw': 1}],
'map_over_dim': False,
'n_input': 1,
'name': 'func1',
'params_and_consts': []},
'FParamFun35': {'code': 'def func2(K2):\n'
' return torch.cos(K2)\n',
'in_dim': [{'dim': 1, 'sw': 1}],
'map_over_dim': False,
'n_input': 1,
'name': 'func2',
'params_and_consts': []}},
'Info': {},
'Inputs': {'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate74'},
'Parameters': {'PLinear37W': {'dim': [1, 2]}, 'PLinear37b': {'dim': 2}},
'Relations': {'Concatenate74': ['Concatenate', ['ParamFun71', 'ParamFun73']],
'Linear69': ['Linear',
['SamplePart68'],
'PLinear37W',
'PLinear37b',
0],
'ParamFun71': ['ParamFun', ['Select70'], 'FParamFun34'],
'ParamFun73': ['ParamFun', ['Select72'], 'FParamFun35'],
'SamplePart68': ['SamplePart', ['x'], -1, [-1, 0]],
'Select70': ['Select', ['Linear69'], 2, 0],
'Select72': ['Select', ['Linear69'], 2, 1]}}
--------------------------- out {'dim': 2, 'sw': 1} ----------------------------
Using parametric functions with parameters
Create an equation learner block with simple parametric functions
[9]:
def myFun(K1,K2,p1,p2):
return K1*p1+K2*p2
x = Input('x')
F = Input('F')
K1 = Parameter('k1', dimensions = 1, sw = 1, values=[[2.0]])
K2 = Parameter('k2', dimensions = 1, sw = 1, values=[[3.0]])
parfun = ParamFun(myFun, parameters_and_constants=[K1,K2])
equation_learner = EquationLearner([parfun, Sin, Add])
Output('out',equation_learner((x.last(),F.last())))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'F' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[9]:
==================================== Output ====================================
{'Constants': {},
'Functions': {'FParamFun45': {'code': 'def myFun(K1,K2,p1,p2):\n'
' return K1*p1+K2*p2\n',
'in_dim': [{'dim': 1, 'sw': 1},
{'dim': 1, 'sw': 1}],
'map_over_dim': False,
'n_input': 2,
'name': 'myFun',
'params_and_consts': ['k1', 'k2']}},
'Info': {},
'Inputs': {'F': {'dim': 1, 'sw': [-1, 0]}, 'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate90'},
'Parameters': {'PLinear47W': {'dim': [2, 5]},
'PLinear47b': {'dim': 5},
'k1': {'dim': 1,
'init_values': [[2.0]],
'sw': 1,
'values': [[2.0]]},
'k2': {'dim': 1,
'init_values': [[3.0]],
'sw': 1,
'values': [[3.0]]}},
'Relations': {'Add89': ['Add', ['Select87', 'Select88']],
'Concatenate79': ['Concatenate',
['SamplePart76', 'SamplePart78']],
'Concatenate86': ['Concatenate', ['ParamFun83', 'Sin85']],
'Concatenate90': ['Concatenate', ['Concatenate86', 'Add89']],
'Linear80': ['Linear',
['Concatenate79'],
'PLinear47W',
'PLinear47b',
0],
'ParamFun83': ['ParamFun',
['Select81', 'Select82'],
'FParamFun45'],
'SamplePart76': ['SamplePart', ['x'], -1, [-1, 0]],
'SamplePart78': ['SamplePart', ['F'], -1, [-1, 0]],
'Select81': ['Select', ['Linear80'], 5, 0],
'Select82': ['Select', ['Linear80'], 5, 1],
'Select84': ['Select', ['Linear80'], 5, 2],
'Select87': ['Select', ['Linear80'], 5, 3],
'Select88': ['Select', ['Linear80'], 5, 4],
'Sin85': ['Sin', ['Select84']]}}
--------------------------- out {'dim': 3, 'sw': 1} ----------------------------
Parametric functions and fuzzy layers
Create an equation learner block with parametric functions and fuzzy layers
[10]:
def myFun(K1,p1):
return K1*p1
x = Input('x')
F = Input('F')
K = Parameter('k', dimensions = 1, sw = 1,values=[[2.0]])
parfun = ParamFun(myFun, parameters_and_constants = [K])
fuzzi = Fuzzify(centers=[0,1,2,3])
equation_learner = EquationLearner([parfun, fuzzi])
Output('out',equation_learner((x.last(),F.last())))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'F' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[10]:
==================================== Output ====================================
{'Constants': {},
'Functions': {'FFuzzify55': {'centers': [0, 1, 2, 3],
'dim_out': {'dim': 4},
'functions': 'Triangular',
'names': 'Triangular'},
'FParamFun54': {'code': 'def myFun(K1,p1):\n return K1*p1\n',
'in_dim': [{'dim': 1, 'sw': 1}],
'map_over_dim': False,
'n_input': 1,
'name': 'myFun',
'params_and_consts': ['k']}},
'Info': {},
'Inputs': {'F': {'dim': 1, 'sw': [-1, 0]}, 'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Concatenate101'},
'Parameters': {'PLinear57W': {'dim': [2, 2]},
'PLinear57b': {'dim': 2},
'k': {'dim': 1,
'init_values': [[2.0]],
'sw': 1,
'values': [[2.0]]}},
'Relations': {'Concatenate101': ['Concatenate', ['ParamFun98', 'Fuzzify100']],
'Concatenate95': ['Concatenate',
['SamplePart92', 'SamplePart94']],
'Fuzzify100': ['Fuzzify', ['Select99'], 'FFuzzify55'],
'Linear96': ['Linear',
['Concatenate95'],
'PLinear57W',
'PLinear57b',
0],
'ParamFun98': ['ParamFun', ['Select97'], 'FParamFun54'],
'SamplePart92': ['SamplePart', ['x'], -1, [-1, 0]],
'SamplePart94': ['SamplePart', ['F'], -1, [-1, 0]],
'Select97': ['Select', ['Linear96'], 2, 0],
'Select99': ['Select', ['Linear96'], 2, 1]}}
--------------------------- out {'dim': 5, 'sw': 1} ----------------------------
Cascade Equation Learner Blocks
Create a cascade of equation learner blocks with various functions and temporal window inputs
[11]:
x = Input('x')
F = Input('F')
def myFun(K1,K2,p1,p2):
return K1*p1+K2*p2
K1 = Parameter('k1', dimensions = 1, sw = 1, values=[[2.0]])
K2 = Parameter('k2', dimensions = 1, sw = 1, values=[[3.0]])
parfun = ParamFun(myFun, parameters_and_constants = [K1,K2])
input_layer_1 = Linear(output_dimension=5, W_init='init_constant', W_init_params={'value':1}, b_init='init_constant', b_init_params={'value':0})
input_layer_2 = Linear(output_dimension=7, W_init='init_constant', W_init_params={'value':1}, b_init='init_constant', b_init_params={'value':0})
output_layer = Linear(output_dimension=1, W_init='init_constant', W_init_params={'value':1}, b=True)
equation_learner = EquationLearner([parfun, Sin, Add], linear_in=input_layer_1)
equation_learner_2 = EquationLearner(functions=[Tan, Add, Sin, Mul, Identity], linear_in=input_layer_2, linear_out=output_layer)
Output('out',equation_learner_2(equation_learner((x.sw(1),F.sw(1)))))
[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'F' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'k1' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'k2' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
[11]:
==================================== Output ====================================
{'Constants': {},
'Functions': {'FParamFun65': {'code': 'def myFun(K1,K2,p1,p2):\n'
' return K1*p1+K2*p2\n',
'in_dim': [{'dim': 1, 'sw': 1},
{'dim': 1, 'sw': 1}],
'map_over_dim': False,
'n_input': 2,
'name': 'myFun',
'params_and_consts': ['k1', 'k2']}},
'Info': {},
'Inputs': {'F': {'dim': 1, 'sw': [-1, 0]}, 'x': {'dim': 1, 'sw': [-1, 0]}},
'Outputs': {'out': 'Linear135'},
'Parameters': {'PLinear66W': {'dim': [2, 5],
'init_fun': {'name': 'init_constant',
'params': {'value': 1}}},
'PLinear68W': {'dim': [3, 7],
'init_fun': {'name': 'init_constant',
'params': {'value': 1}}},
'PLinear70W': {'dim': [5, 1],
'init_fun': {'name': 'init_constant',
'params': {'value': 1}}},
'PLinear70b': {'dim': 1},
'k1': {'dim': 1,
'init_values': [[2.0]],
'sw': 1,
'values': [[2.0]]},
'k2': {'dim': 1,
'init_values': [[3.0]],
'sw': 1,
'values': [[3.0]]}},
'Relations': {'Add116': ['Add', ['Select114', 'Select115']],
'Add123': ['Add', ['Select121', 'Select122']],
'Concatenate106': ['Concatenate',
['SamplePart103', 'SamplePart105']],
'Concatenate113': ['Concatenate', ['ParamFun110', 'Sin112']],
'Concatenate117': ['Concatenate', ['Concatenate113', 'Add116']],
'Concatenate124': ['Concatenate', ['Tan120', 'Add123']],
'Concatenate127': ['Concatenate', ['Concatenate124', 'Sin126']],
'Concatenate131': ['Concatenate', ['Concatenate127', 'Mul130']],
'Concatenate134': ['Concatenate',
['Concatenate131', 'Identity133']],
'Identity133': ['Identity', ['Select132']],
'Linear107': ['Linear',
['Concatenate106'],
'PLinear66W',
None,
0],
'Linear118': ['Linear',
['Concatenate117'],
'PLinear68W',
None,
0],
'Linear135': ['Linear',
['Concatenate134'],
'PLinear70W',
'PLinear70b',
0],
'Mul130': ['Mul', ['Select128', 'Select129']],
'ParamFun110': ['ParamFun',
['Select108', 'Select109'],
'FParamFun65'],
'SamplePart103': ['SamplePart', ['x'], -1, [-1, 0]],
'SamplePart105': ['SamplePart', ['F'], -1, [-1, 0]],
'Select108': ['Select', ['Linear107'], 5, 0],
'Select109': ['Select', ['Linear107'], 5, 1],
'Select111': ['Select', ['Linear107'], 5, 2],
'Select114': ['Select', ['Linear107'], 5, 3],
'Select115': ['Select', ['Linear107'], 5, 4],
'Select119': ['Select', ['Linear118'], 7, 0],
'Select121': ['Select', ['Linear118'], 7, 1],
'Select122': ['Select', ['Linear118'], 7, 2],
'Select125': ['Select', ['Linear118'], 7, 3],
'Select128': ['Select', ['Linear118'], 7, 4],
'Select129': ['Select', ['Linear118'], 7, 5],
'Select132': ['Select', ['Linear118'], 7, 6],
'Sin112': ['Sin', ['Select111']],
'Sin126': ['Sin', ['Select125']],
'Tan120': ['Tan', ['Select119']]}}
--------------------------- out {'dim': 1, 'sw': 1} ----------------------------