Training
Here are listed all the modalities that can be used to perform the training process of a neural network model.
[ ]:
# uncomment the command below to install the nnodely package
#!pip install nnodely
from nnodely import *
Collecting nnodely
Downloading nnodely-0.14.3-py3-none-any.whl.metadata (19 kB)
Collecting numpy==1.26.4 (from nnodely)
Downloading numpy-1.26.4.tar.gz (15.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.8/15.8 MB 1.0 MB/s eta 0:00:0000:0100:01m
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting onnx==1.16.2 (from nnodely)
Downloading onnx-1.16.2.tar.gz (12.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.3/12.3 MB 1.6 MB/s eta 0:00:00a 0:00:01
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting pandas==2.2.2 (from nnodely)
Downloading pandas-2.2.2.tar.gz (4.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 1.9 MB/s eta 0:00:00a 0:00:01
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
INFO: pip is looking at multiple versions of nnodely to determine which version is compatible with other requirements. This could take a while.
Collecting nnodely
Downloading nnodely-0.14.2-py3-none-any.whl.metadata (19 kB)
Downloading nnodely-0.14.1-py3-none-any.whl.metadata (19 kB)
Collecting numpy (from nnodely)
Using cached numpy-2.3.2-cp313-cp313-macosx_14_0_arm64.whl.metadata (62 kB)
Collecting onnx (from nnodely)
Downloading onnx-1.18.0-cp313-cp313-macosx_12_0_universal2.whl.metadata (6.9 kB)
Collecting pandas (from nnodely)
Downloading pandas-2.3.1-cp313-cp313-macosx_11_0_arm64.whl.metadata (91 kB)
Collecting torch (from nnodely)
Downloading torch-2.7.1-cp313-none-macosx_11_0_arm64.whl.metadata (29 kB)
Collecting reportlab (from nnodely)
Downloading reportlab-4.4.3-py3-none-any.whl.metadata (1.7 kB)
Collecting matplotlib (from nnodely)
Downloading matplotlib-3.10.5-cp313-cp313-macosx_11_0_arm64.whl.metadata (11 kB)
Collecting contourpy>=1.0.1 (from matplotlib->nnodely)
Using cached contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl.metadata (5.5 kB)
Collecting cycler>=0.10 (from matplotlib->nnodely)
Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib->nnodely)
Using cached fonttools-4.59.0-cp313-cp313-macosx_10_13_universal2.whl.metadata (107 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib->nnodely)
Using cached kiwisolver-1.4.8-cp313-cp313-macosx_11_0_arm64.whl.metadata (6.2 kB)
Requirement already satisfied: packaging>=20.0 in /Users/giovanni/Desktop/Projects/nnodely/venv/lib/python3.13/site-packages (from matplotlib->nnodely) (25.0)
Collecting pillow>=8 (from matplotlib->nnodely)
Using cached pillow-11.3.0-cp313-cp313-macosx_11_0_arm64.whl.metadata (9.0 kB)
Collecting pyparsing>=2.3.1 (from matplotlib->nnodely)
Using cached pyparsing-3.2.3-py3-none-any.whl.metadata (5.0 kB)
Requirement already satisfied: python-dateutil>=2.7 in /Users/giovanni/Desktop/Projects/nnodely/venv/lib/python3.13/site-packages (from matplotlib->nnodely) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in /Users/giovanni/Desktop/Projects/nnodely/venv/lib/python3.13/site-packages (from python-dateutil>=2.7->matplotlib->nnodely) (1.17.0)
Collecting protobuf>=4.25.1 (from onnx->nnodely)
Using cached protobuf-6.31.1-cp39-abi3-macosx_10_9_universal2.whl.metadata (593 bytes)
Collecting typing_extensions>=4.7.1 (from onnx->nnodely)
Using cached typing_extensions-4.14.1-py3-none-any.whl.metadata (3.0 kB)
Collecting pytz>=2020.1 (from pandas->nnodely)
Using cached pytz-2025.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas->nnodely)
Using cached tzdata-2025.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting charset-normalizer (from reportlab->nnodely)
Downloading charset_normalizer-3.4.2-cp313-cp313-macosx_10_13_universal2.whl.metadata (35 kB)
Collecting filelock (from torch->nnodely)
Using cached filelock-3.18.0-py3-none-any.whl.metadata (2.9 kB)
Collecting setuptools (from torch->nnodely)
Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting sympy>=1.13.3 (from torch->nnodely)
Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch->nnodely)
Using cached networkx-3.5-py3-none-any.whl.metadata (6.3 kB)
Collecting jinja2 (from torch->nnodely)
Using cached jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec (from torch->nnodely)
Using cached fsspec-2025.7.0-py3-none-any.whl.metadata (12 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch->nnodely)
Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch->nnodely)
Using cached MarkupSafe-3.0.2-cp313-cp313-macosx_11_0_arm64.whl.metadata (4.0 kB)
Downloading nnodely-0.14.1-py3-none-any.whl (71 kB)
Downloading matplotlib-3.10.5-cp313-cp313-macosx_11_0_arm64.whl (8.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.1/8.1 MB 1.3 MB/s eta 0:00:00a 0:00:01
Using cached contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl (274 kB)
Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)
Using cached fonttools-4.59.0-cp313-cp313-macosx_10_13_universal2.whl (2.8 MB)
Using cached kiwisolver-1.4.8-cp313-cp313-macosx_11_0_arm64.whl (65 kB)
Using cached numpy-2.3.2-cp313-cp313-macosx_14_0_arm64.whl (5.1 MB)
Using cached pillow-11.3.0-cp313-cp313-macosx_11_0_arm64.whl (4.7 MB)
Using cached pyparsing-3.2.3-py3-none-any.whl (111 kB)
Downloading onnx-1.18.0-cp313-cp313-macosx_12_0_universal2.whl (18.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 1.1 MB/s eta 0:00:0000:0100:01m
Using cached protobuf-6.31.1-cp39-abi3-macosx_10_9_universal2.whl (425 kB)
Using cached typing_extensions-4.14.1-py3-none-any.whl (43 kB)
Downloading pandas-2.3.1-cp313-cp313-macosx_11_0_arm64.whl (10.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 1.2 MB/s eta 0:00:00a 0:00:01
Using cached pytz-2025.2-py2.py3-none-any.whl (509 kB)
Using cached tzdata-2025.2-py2.py3-none-any.whl (347 kB)
Downloading reportlab-4.4.3-py3-none-any.whl (2.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 1.3 MB/s eta 0:00:00a 0:00:01
Downloading charset_normalizer-3.4.2-cp313-cp313-macosx_10_13_universal2.whl (199 kB)
Downloading torch-2.7.1-cp313-none-macosx_11_0_arm64.whl (68.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.6/68.6 MB 1.8 MB/s eta 0:00:0000:0100:020m
Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 1.7 MB/s eta 0:00:00a 0:00:01
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Using cached filelock-3.18.0-py3-none-any.whl (16 kB)
Using cached fsspec-2025.7.0-py3-none-any.whl (199 kB)
Using cached jinja2-3.1.6-py3-none-any.whl (134 kB)
Using cached MarkupSafe-3.0.2-cp313-cp313-macosx_11_0_arm64.whl (12 kB)
Using cached networkx-3.5-py3-none-any.whl (2.0 MB)
Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
Installing collected packages: pytz, mpmath, tzdata, typing_extensions, sympy, setuptools, pyparsing, protobuf, pillow, numpy, networkx, MarkupSafe, kiwisolver, fsspec, fonttools, filelock, cycler, charset-normalizer, reportlab, pandas, onnx, jinja2, contourpy, torch, matplotlib, nnodely
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26/26 [nnodely]5/26 [nnodely]ib]
Successfully installed MarkupSafe-3.0.2 charset-normalizer-3.4.2 contourpy-1.3.3 cycler-0.12.1 filelock-3.18.0 fonttools-4.59.0 fsspec-2025.7.0 jinja2-3.1.6 kiwisolver-1.4.8 matplotlib-3.10.5 mpmath-1.3.0 networkx-3.5 nnodely-0.14.1 numpy-2.3.2 onnx-1.18.0 pandas-2.3.1 pillow-11.3.0 protobuf-6.31.1 pyparsing-3.2.3 pytz-2025.2 reportlab-4.4.3 setuptools-80.9.0 sympy-1.14.0 torch-2.7.1 typing_extensions-4.14.1 tzdata-2025.2
[notice] A new release of pip is available: 25.1.1 -> 25.2
[notice] To update, run: pip install --upgrade pip
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>- nnodely_v0.14.1 --<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
[4]:
## Create a neural network model and Load a dataset
in1 = Input('in1')
target = Input('target')
relation = Fir(in1.tw(0.05))
output = Output('out', relation)
model = Modely(visualizer=TextVisualizer())
model.addMinimize('error', output, target.last())
model.addModel('model', output)
model.neuralizeModel(0.01)
train_folder = 'data'
data_struct = ['in1', '', 'target']
model.loadData(name='dataset', source=train_folder, format=data_struct, skiplines=1)
================================ nnodely Model =================================
{'Constants': {},
'Functions': {},
'Info': {'SampleTime': 0.01, 'ns': [5, 0], 'ntot': 5},
'Inputs': {'in1': {'dim': 1,
'ns': [5, 0],
'ntot': 5,
'sw': [0, 0],
'tw': [-0.05, 0]},
'target': {'dim': 1,
'ns': [1, 0],
'ntot': 1,
'sw': [-1, 0],
'tw': [0, 0]}},
'Minimizers': {'error': {'A': 'out', 'B': 'SamplePart7', 'loss': 'mse'}},
'Models': 'model',
'Outputs': {'out': 'Fir4'},
'Parameters': {'PFir2p': {'dim': 1, 'tw': 0.05}},
'Relations': {'Fir4': ['Fir', ['TimePart3'], 'PFir2p', None, 0],
'SamplePart7': ['SamplePart', ['target'], [-1, 0]],
'TimePart3': ['TimePart', ['in1'], [-0.05, 0]]},
'States': {}}
================================================================================
============================ nnodely Model Dataset =============================
Dataset Name: dataset
Number of files: 1
Total number of samples: 27
Shape of target: (27, 1, 1)
Shape of in1: (27, 5, 1)
================================================================================
Basic Parameters and Usage
With the trainModel function the user can perform a training process by setting different parameters of training such as:
models to train
training and validation dataset
number of epochs of training
data shuffling
train and validation batch size
learning rate
And many more advanced settings..
If no parameters are specified, the function will train all the built models using all the loaded dataset with the default parameters.
[5]:
training_parameters = model.trainModel()
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 100
update per epochs: 1.0
└>(n_samples-batch_size)/batch_size+1
shuffle data: True
train dataset: train_dataset_0.70
- num of samples: 19
- batch size: 19
- unused samples: 0.0
└>n_samples-update_per_epochs*batch_size
val dataset: validation_dataset_0.20
val {batch size, samples}: {5, 5}
test dataset: test_dataset_0.10
test {batch size, samples}: {3, 3}
minimizers: {'error': {'A': 'out',
'B': 'SamplePart7',
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.001}
optimizer params: [{'params': 'PFir2p'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | val | train | val |
|--------------------------------------------------|
| 10/100 |6.510e+02|2.514e+03|6.510e+02|2.514e+03|
| 20/100 |6.202e+02|2.400e+03|6.202e+02|2.400e+03|
| 30/100 |5.904e+02| 2.29e+03|5.904e+02| 2.29e+03|
| 40/100 |5.616e+02|2.183e+03|5.616e+02|2.183e+03|
| 50/100 |5.339e+02|2.081e+03|5.339e+02|2.081e+03|
| 60/100 |5.072e+02|1.981e+03|5.072e+02|1.981e+03|
| 70/100 |4.815e+02|1.886e+03|4.815e+02|1.886e+03|
| 80/100 |4.568e+02|1.794e+03|4.568e+02|1.794e+03|
| 90/100 |4.331e+02|1.706e+03|4.331e+02|1.706e+03|
| 100/100 |4.103e+02|1.621e+03|4.103e+02|1.621e+03|
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.0738527774810791
================================================================================
The selected model is the LAST model of the training.
=========== nnodely Model Results for train_dataset_0.70 ==========
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 4.081e+02 | 2.032e+00 | 2.052e+02 |
|-----------------------------------------------------------------|
|Total| 4.081e+02 | 2.032e+00 | 2.052e+02 |
|-----------------------------------------------------------------|
======== nnodely Model Results for validation_dataset_0.20 ========
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 1.621e+03 | 2.032e+00 | 1.144e+03 |
|-----------------------------------------------------------------|
|Total| 1.621e+03 | 2.032e+00 | 1.144e+03 |
|-----------------------------------------------------------------|
=========== nnodely Model Results for test_dataset_0.10 ===========
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 2.287e+03 | 2.032e+00 | 2.85e+03 |
|-----------------------------------------------------------------|
|Total| 2.287e+03 | 2.032e+00 | 2.85e+03 |
|-----------------------------------------------------------------|
[6]:
training_parameters = model.trainModel(models='model', train_dataset='dataset', num_of_epochs=10, lr=0.01, shuffle_data=True, train_batch_size=4)
======================== nnodely Model Train Parameters ========================
models: 'model'
num of epochs: 10
update per epochs: 6.75
└>(n_samples-batch_size)/batch_size+1
shuffle data: True
train dataset: dataset
- num of samples: 27
- batch size: 4
- unused samples: 0.0
└>n_samples-update_per_epochs*batch_size
minimizers: {'error': {'A': 'out',
'B': 'SamplePart7',
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.01}
optimizer params: [{'params': 'PFir2p'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | train |
|--------------------------------------------------|
| 1/10 | 7.389e+02 | 7.389e+02 |
| 2/10 | 4.521e+02 | 4.521e+02 |
| 3/10 | 3.187e+02 | 3.187e+02 |
| 4/10 | 1.909e+02 | 1.909e+02 |
| 5/10 | 9.889e+01 | 9.889e+01 |
| 6/10 | 4.236e+01 | 4.236e+01 |
| 7/10 | 1.896e+01 | 1.896e+01 |
| 8/10 | 6.181e+00 | 6.181e+00 |
| 9/10 | 3.443e+00 | 3.443e+00 |
| 10/10 | 3.040e+00 | 3.040e+00 |
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.028616905212402344
================================================================================
The selected model is the LAST model of the training.
================ nnodely Model Results for dataset ================
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 4.007e+00 | 1.828e-02 | 1.508e+02 |
|-----------------------------------------------------------------|
|Total| 4.007e+00 | 1.828e-02 | 1.508e+02 |
|-----------------------------------------------------------------|
To automatically splits the dataset between training set and validation set, just provide the datasets to use inside the dataset parameter.
In this way, the parameter splits will be used to splits the data accordingly.
[8]:
training_parameters = model.trainModel(models='model', dataset='dataset', splits=[70, 20, 10], num_of_epochs=10, shuffle_data=True, train_batch_size=4)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[8], line 1
----> 1 training_parameters = model.trainModel(models='model', dataset='dataset', splits=[70, 20, 10], num_of_epochs=10, shuffle_data=True, train_batch_size=4)
TypeError: Modely.trainModel() got an unexpected keyword argument 'dataset'
Recurrent Training
The recurrent training can take place only when there are recurrent variables, closed-loops or connections between inputs.
The recurrent train can also be performed on the run by setting the closed_loop dictionary for closed connections and/or by setting the connect dictionary for direct connections.
In case of a recurrent training, the number of prediction horizon window (prediction_samples) must be specified. This is used to select for how many steps doing the recurrent loop
The step is used to decide whether to skip samples at each epoch. this will ensure a faster training time expecially when the prediction horizon is too high.
[6]:
target = Input('target')
x = Input('x')
relation = Fir(x.last())
relation.closedLoop(x)
output = Output('out', relation)
test = Modely(visualizer=TextVisualizer(), seed=42)
test.addModel('model', output)
test.addMinimize('error', target.next(), relation)
test.neuralizeModel(0.01)
dataset = {'x': [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20], 'target': [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40]}
test.loadData(name='dataset', source=dataset)
[check_names] The name 'target' is already in defined as NeuObj but it is overwritten.
[check_names] The name 'out' is already in defined as NeuObj but it is overwritten.
================================ nnodely Model =================================
{'Constants': {},
'Functions': {},
'Info': {'SampleTime': 0.01,
'nnodely_version': '1.5.0',
'ns': [1, 1],
'ntot': 2,
'num_parameters': 1},
'Inputs': {'target': {'dim': 1, 'ns': [0, 1], 'ntot': 1, 'sw': [0, 1]},
'x': {'closedLoop': 'Fir7',
'dim': 1,
'local': 1,
'ns': [1, 0],
'ntot': 1,
'sw': [-1, 0]}},
'Minimizers': {'error': {'A': 'SamplePart9', 'B': 'Fir7', 'loss': 'mse'}},
'Models': 'model',
'Outputs': {'out': 'Fir7'},
'Parameters': {'PFir8W': {'dim': 1,
'sw': 1,
'values': [[0.8822692632675171]]}},
'Relations': {'Fir7': ['Fir', ['SamplePart6'], 'PFir8W', None, 0],
'SamplePart6': ['SamplePart', ['x'], -1, [-1, 0]],
'SamplePart9': ['SamplePart', ['target'], -1, [0, 1]]}}
================================================================================
============================ nnodely Model Dataset =============================
Dataset Name: dataset
Number of files: 1
Total number of samples: 19
Shape of x: (19, 1, 1)
Shape of target: (19, 1, 1)
================================================================================
[7]:
training_parameters = test.trainModel(train_dataset='dataset', lr=0.01, num_of_epochs=10, train_batch_size=4, prediction_samples=2, step=1, shuffle_data=True)
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 10
update per epochs: 3
└>(n_samples-batch_size-prediction_samples+1)/(batch_size+step-1)+1
shuffle _data: True
prediction samples: 2
step: 1
closed loop: {}
connect: {}
train dataset: dataset
- num of samples: 19
- batch size: 4
- unused samples: 5
└>n_samples-prediction_samples-update_per_epochs*(batch_size+step-1)
minimizers: {'error': {'A': 'SamplePart9',
'B': 'Fir7',
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.01}
optimizer params: [{'params': 'PFir8W'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | train |
|--------------------------------------------------|
| 1/10 | 4.213e+02 | 4.213e+02 |
| 2/10 | 4.120e+02 | 4.120e+02 |
| 3/10 | 3.957e+02 | 3.957e+02 |
| 4/10 | 3.765e+02 | 3.765e+02 |
| 5/10 | 3.58e+02 | 3.58e+02 |
| 6/10 | 3.395e+02 | 3.395e+02 |
| 7/10 | 3.259e+02 | 3.259e+02 |
| 8/10 | 2.979e+02 | 2.979e+02 |
| 9/10 | 2.89e+02 | 2.89e+02 |
| 10/10 | 2.871e+02 | 2.871e+02 |
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.039998769760131836
================================================================================
Set different weight for the minimization functions
use the minimize_gain attribute to modify the importance of certain minimization functions by passing a dictionary with the gain factor for each minimizers.
[8]:
training_parameters = test.trainModel(train_dataset='dataset', minimize_gain={'error':0})
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 100
update per epochs: 1
└>(n_samples-batch_size-prediction_samples+1)/(batch_size+step-1)+1
shuffle _data: True
prediction samples: 0
step: 0
closed loop: {}
connect: {}
train dataset: dataset
- num of samples: 19
- batch size: 19
- unused samples: 0
└>n_samples-prediction_samples-update_per_epochs*(batch_size+step-1)
minimizers: {'error': {'A': 'SamplePart9',
'B': 'Fir7',
'gain': 0,
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.001}
optimizer params: [{'params': 'PFir8W'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | train |
|--------------------------------------------------|
| 10/100 | 0.e+00 | 0.e+00 |
| 20/100 | 0.e+00 | 0.e+00 |
| 30/100 | 0.e+00 | 0.e+00 |
| 40/100 | 0.e+00 | 0.e+00 |
| 50/100 | 0.e+00 | 0.e+00 |
| 60/100 | 0.e+00 | 0.e+00 |
| 70/100 | 0.e+00 | 0.e+00 |
| 80/100 | 0.e+00 | 0.e+00 |
| 90/100 | 0.e+00 | 0.e+00 |
| 100/100 | 0.e+00 | 0.e+00 |
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.08199954032897949
================================================================================
Setting the Optimizer
There are various ways to configure the optimizer to use during the training process.
This can be achieved by selecting the name of the optimizer to use from the pytorch list and then passing a dictionary of options to configure the desired optimizer
[9]:
optimizer_defaults = {
'lr': 0.1,
'betas': (0.5, 0.99)
}
training_parameters = test.trainModel(train_dataset='dataset', optimizer='Adam', optimizer_defaults=optimizer_defaults, num_of_epochs=10)
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 10
update per epochs: 1
└>(n_samples-batch_size-prediction_samples+1)/(batch_size+step-1)+1
shuffle _data: True
prediction samples: 0
step: 0
closed loop: {}
connect: {}
train dataset: dataset
- num of samples: 19
- batch size: 19
- unused samples: 0
└>n_samples-prediction_samples-update_per_epochs*(batch_size+step-1)
minimizers: {'error': {'A': 'SamplePart9',
'B': 'Fir7',
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'betas': (0.5, 0.99), 'lr': 0.1}
optimizer params: [{'params': 'PFir8W'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | train |
|--------------------------------------------------|
| 1/10 | 3.712e+02 | 3.712e+02 |
| 2/10 | 3.350e+02 | 3.350e+02 |
| 3/10 | 3.019e+02 | 3.019e+02 |
| 4/10 | 2.719e+02 | 2.719e+02 |
| 5/10 | 2.451e+02 | 2.451e+02 |
| 6/10 | 2.215e+02 | 2.215e+02 |
| 7/10 | 2.008e+02 | 2.008e+02 |
| 8/10 | 1.830e+02 | 1.830e+02 |
| 9/10 | 1.678e+02 | 1.678e+02 |
| 10/10 | 1.549e+02 | 1.549e+02 |
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.009000539779663086
================================================================================
Early stopping and Best Model
Use one of the built-in early stopping and selection model functions or use a custom one.
[10]:
from nnodely.support.earlystopping import early_stop_patience, select_best_model
training_parameters = test.trainModel(train_dataset='dataset', minimize_gain={'error':0}, early_stopping=early_stop_patience, select_model=select_best_model)
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 100
update per epochs: 1
└>(n_samples-batch_size-prediction_samples+1)/(batch_size+step-1)+1
shuffle _data: True
early stopping: early_stop_patience
early stopping params: {}
prediction samples: 0
step: 0
closed loop: {}
connect: {}
train dataset: dataset
- num of samples: 19
- batch size: 19
- unused samples: 0
└>n_samples-prediction_samples-update_per_epochs*(batch_size+step-1)
minimizers: {'error': {'A': 'SamplePart9',
'B': 'Fir7',
'gain': 0,
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.001}
optimizer params: [{'params': 'PFir8W'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | train |
|--------------------------------------------------|
| 10/100 | 0.e+00 | 0.e+00 |
| 20/100 | 0.e+00 | 0.e+00 |
| 30/100 | 0.e+00 | 0.e+00 |
| 32/100 | 0.e+00 | 0.e+00 || 40/100 | 0.e+00 | 0.e+00 |
| 50/100 | 0.e+00 | 0.e+00 |
============================ nnodely Training Time =============================
Total time of Training: 0.07100605964660645
================================================================================
Train and Analyze
the function trainAndAnalyze is a convenient way to do the training and validation at the same time.
With the following function the user can use a test dataset to assess the quality of the trained model.
[11]:
test_folder = 'data'
data_struct = ['in1', '', 'target']
model.loadData(name='dataset_test', source=test_folder, format=data_struct, skiplines=1)
model.loadData(name='dataset_val', source=test_folder, format=data_struct, skiplines=1)
training_parameters = model.trainAndAnalyze(models='model', train_dataset='dataset', validation_dataset='dataset_val', train_batch_size=4, test_dataset='dataset_test', test_batch_size=1, num_of_epochs=10, lr=0.01, shuffle_data=True)
============================ nnodely Model Dataset =============================
Dataset Name: dataset_test
Number of files: 1
Total number of samples: 27
Shape of in1: (27, 5, 1)
Shape of target: (27, 1, 1)
================================================================================
============================ nnodely Model Dataset =============================
Dataset Name: dataset_val
Number of files: 1
Total number of samples: 27
Shape of in1: (27, 5, 1)
Shape of target: (27, 1, 1)
================================================================================
======================== nnodely Model Train Parameters ========================
models: ['model']
num of epochs: 10
update per epochs: 6
└>(n_samples-batch_size)/batch_size+1
shuffle _data: True
train dataset: dataset
- num of samples: 27
- batch size: 4
- unused samples: 3
└>n_samples-update_per_epochs*batch_size
val dataset: dataset_val
val {batch size, samples}: {27, 27}
minimizers: {'error': {'A': 'Fir2',
'B': 'SamplePart4',
'loss': 'mse'}}
optimizer: Adam
optimizer defaults: {'lr': 0.01}
optimizer params: [{'params': 'PFir3W'}]
================================================================================
================= nnodely Training =================
| Epoch | error | Total |
| | Loss | Loss |
| | train | val | train | val |
|--------------------------------------------------|
| 1/10 |1.929e+00|1.635e+00|1.929e+00|1.635e+00|
| 2/10 |1.961e+00|1.623e+00|1.961e+00|1.623e+00|
| 3/10 |1.617e+00|1.776e+00|1.617e+00|1.776e+00|
| 4/10 |1.671e+00|1.432e+00|1.671e+00|1.432e+00|
| 5/10 |1.603e+00| 1.41e+00|1.603e+00| 1.41e+00|
| 6/10 |1.623e+00|1.408e+00|1.623e+00|1.408e+00|
| 7/10 |1.455e+00|1.406e+00|1.455e+00|1.406e+00|
| 8/10 |1.403e+00|1.358e+00|1.403e+00|1.358e+00|
| 9/10 |1.495e+00|1.288e+00|1.495e+00|1.288e+00|
| 10/10 |1.386e+00|1.249e+00|1.386e+00|1.249e+00|
|--------------------------------------------------|
============================ nnodely Training Time =============================
Total time of Training: 0.036997079849243164
================================================================================
================ nnodely Model Results for dataset ================
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 1.27e+00 | 1.464e-02 | 8.712e+01 |
|-----------------------------------------------------------------|
|Total| 1.27e+00 | 1.464e-02 | 8.712e+01 |
|-----------------------------------------------------------------|
============== nnodely Model Results for dataset_val ==============
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 1.249e+00 | 1.464e-02 | 9.337e+01 |
|-----------------------------------------------------------------|
|Total| 1.249e+00 | 1.464e-02 | 9.337e+01 |
|-----------------------------------------------------------------|
============== nnodely Model Results for dataset_test =============
| Loss| mse | FVU | AIC |
| | small better | small better | lower better |
|-----------------------------------------------------------------|
|error| 1.249e+00 | 1.464e-02 | 9.337e+01 |
|-----------------------------------------------------------------|
|Total| 1.249e+00 | 1.464e-02 | 9.337e+01 |
|-----------------------------------------------------------------|
[12]:
import pprint
print("Training completed with parameters:", pprint.pprint(training_parameters))
{'add_optimizer_defaults': {},
'add_optimizer_params': [],
'all_models': ['model'],
'closed_loop': {},
'connect': {},
'dataset': None,
'early_stopping': None,
'early_stopping_params': {},
'minimize_gain': {},
'minimizers': {'error': {'A': 'Fir2', 'B': 'SamplePart4', 'loss': 'mse'}},
'models': ['model'],
'n_samples_test': 27,
'n_samples_train': 27,
'n_samples_val': 27,
'name': None,
'num_of_epochs': 10,
'optimizer': 'Adam',
'optimizer_defaults': {'lr': 0.01},
'optimizer_params': [{'params': 'PFir3W'}],
'prediction_samples': -1,
'select_model': 'last',
'select_model_params': {},
'shuffle_data': True,
'splits': [100, 0, 0],
'step': 0,
'test_batch_size': 1,
'test_step': 0,
'test_tag': 'dataset_test',
'train_batch_size': 4,
'train_dataset': 'dataset',
'train_step': 0,
'train_tag': 'dataset',
'unused_samples': 3,
'update_per_epochs': 6,
'val_batch_size': 27,
'val_step': 0,
'val_tag': 'dataset_val',
'validation_dataset': 'dataset_val'}
Training completed with parameters: None