Linear Layer

Represents a Linear relation in the neural network model.

The Linear relation works along the input dimension (third dimension) of the input tensor.

The Linear relation can be instantiated with the following modalities:

1- Setting and initialize the weight vector

2- Setting and initialize the bias vector

3- Adding Dropout regularization

# uncomment the command below to install the nnodely package
#!pip install nnodely

from nnodely import *

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>-- nnodely_v1.5.0 --<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Basic Usage

Create a Linear Relation block with random Weights initialization and 5 outputs.

(By default, there is no bias. to add the bias parameter set the ‘b’ attribute to True)

x = Input('x').tw(0.05)
linear = Linear(output_dimension=5)(x)

Passing a Parameter

Create a Fir Relation block and setting the weight and bias with a pre-defined parameter.

x = Input('x').last()

weight = Parameter('W', values=[[1]])
bias = Parameter('b', values=[1])

linear = Linear(W=weight, b=bias)(x)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.

Initialization functions

Create a Fir Relation block and initialize the weight and bias using an initialization function.

Set the argument of the initialization function using a dictionary as shown below.

(you can find all the initialization function inside the ‘initializer’ module. You can also define your own initialization function!)

x = Input('x').last()
linear = Linear(output_dimension=5, b=True, W_init=init_negexp, b_init=init_constant, b_init_params={'value':1})(x)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.

Dropout

Create a Linear Relation block with 10 outputs and dropout regularization

input = Input('x')
linear = Linear(output_dimension=10, dropout=0.2)(input.sw(2))
output = Output('out', linear)

[check_names] The name 'x' is already in defined as NeuObj but it is overwritten.