Fir Layer

Represents a Finite Impulse Response (FIR) relation in the neural network model.

The FIR relation works along the time dimension (second dimension) of the input tensor.

The Fir relation can be initialized with the following modalities:

1 - Setting and initialize the weight vector

2 - Setting and initialize the bias vector

3 - Adding Dropout regularization

# uncomment the command below to install the nnodely package
#!pip install nnodely

from nnodely import *

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>-- nnodely_v1.5.0 --<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Basic Usage Example

Create a Fir Relation block with random Weights initialization. The Fir relation can be generated inplace by calling the Fir class with the input tensor.

(By default, there is no bias. to add also the bias parameter set the ‘b’ attribute to True)

input = Input('x')
fir_layer = Fir(b=True)
output = Output('out', fir_layer(input.tw(0.05)))
# Inplace creation with default initialization
output = Output('out2', Fir(input.tw(0.05)))

Setting a Weight and a Bias Name

It is possible to set the name of the weight and bias parameters, without setting the initialization function.

input = Input('x')
fir_layer = Fir(b='b', W='w')
output = Output('out', fir_layer(input.tw(0.05)))

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.

Passing a Parameter to the Fir Relation

Create a Fir Relation block and setting the weight and bias with a pre-defined parameter.

input = Input('x')

weight = Parameter('w', dimensions=3, sw=2, init='init_constant')
bias = Parameter('b', dimensions=3, init='init_constant')

fir_layer = Fir(W=weight, b=bias)(input.sw(2))
output = Output('out', fir_layer)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'w' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'b' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.

Initialize the parameters

Create a Fir Relation block and initialize the weight and bias using an initialization function.

Set the argument of the initialization function using a dictionary as shown below.

(you can find all the initialization function inside the ‘initializer’ module. You can also define your own initialization function!)

x = Input('x')
F = Input('F')

fir_x = Fir(W_init=init_negexp, b_init=init_exp)(x.tw(0.2))
fir_F = Fir(W_init=init_constant, W_init_params={'value':1}, b_init=init_constant, b_init_params={'value':0})(F.last())

output = Output('out', fir_x + fir_F)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.

Adding the Dropout

Create a Fir Relation block with dropout regularization

input = Input('x')

weight = Parameter('w', dimensions=3, sw=2, init=init_constant)

fir_layer = Fir(W=weight, b=False, dropout=0.2)(input.sw(2))
output = Output('out', fir_layer)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'w' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.