Nodes

This module contains the classes to define and create neural network layers. The abstract class LayerNode represents a generic NN layer, and its child ConcreteLayerNode defines the internal representation of all currently supported layers.

class pynever.nodes.LayerNode(identifier)

Bases: ABC

An abstract class used for our internal representation of a generic Layer.

identifier

Identifier of the LayerNode.

Type:

str

class pynever.nodes.ConcreteLayerNode(identifier, in_dims, out_dim)

Bases: LayerNode

An abstract class used for our internal representation of a generic Layer. Its concrete children correspond to real network layers.

identifier

Identifier of the ConcreteLayerNode.

Type:

str

in_dims

Dimension of the input Tensor as a list of tuples (ndarray.shape like).

Type:

list[tuple]

out_dim

Dimension of the output Tensor as a tuple (ndarray.shape like).

Type:

tuple

abstractmethod get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

get_output_dim()

Procedure to get the output dimension of the layer.

Returns:

The output dimensions of the layer.

Return type:

tuple

class pynever.nodes.ReLUNode(identifier, in_dim)

Bases: ConcreteLayerNode

A class used for our internal representation of a ReLU Layer.

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.ELUNode(identifier, in_dim, alpha=1.0)

Bases: ConcreteLayerNode

A class used for our internal representation of an ELU Layer.

alpha

The alpha value for the ELU formulation (default: 1.0).

Type:

float, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.CELUNode(identifier, in_dim, alpha=1.0)

Bases: ConcreteLayerNode

A class used for our internal representation of a CELU Layer.

alpha

The alpha value for the CELU formulation (default: 1.0).

Type:

float, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.LeakyReLUNode(identifier, in_dim, negative_slope=0.01)

Bases: ConcreteLayerNode

A class used for our internal representation of a Leaky ReLU Layer.

negative_slope

Controls the angle of the negative slope (default: 1e-2).

Type:

float, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.SigmoidNode(identifier, in_dim)

Bases: ConcreteLayerNode

A class used for our internal representation of a Sigmoid Layer.

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.TanhNode(identifier, in_dim)

Bases: ConcreteLayerNode

A class used for our internal representation of a Tanh Layer.

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.FullyConnectedNode(identifier, in_dim, out_features, weight=None, bias=None, has_bias=True)

Bases: ConcreteLayerNode

A class used for our internal representation of a Fully Connected layer

in_features

Number of input features of the fully connected layer.

Type:

int

out_features

Number of output features of the fully connected layer.

Type:

int

weight

Tensor containing the weight parameters of the fully connected layer.

Type:

torch.Tensor, Optional

bias

Tensor containing the bias parameters of the fully connected layer.

Type:

torch.Tensor, Optional

has_bias

Flag True if the fully connected layer has bias, False otherwise (default: True)

Type:

bool, Optional

get_layer_bias_as_two_dimensional()

This method expands the bias since they are memorized like one-dimensional vectors in FC nodes.

Returns:

The new bias with explicit dimensions

Return type:

torch.Tensor

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.BatchNormNode(identifier, in_dim, weight=None, bias=None, running_mean=None, running_var=None, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

Bases: ConcreteLayerNode

A class used for our internal representation of a one dimensional Batch Normalization Layer. N.B. There are some problem for compatibility between pytorch and onnx: pytorch provide 3 different kind of batchnorm layers which supports [(N, C) or (N, C, L)], (N, C, H, W) and (N, C, D, H, W) dimensional inputs respectively (BatchNorm1D, BatchNorm2D and BatchNorm3D). The batchnorm operation is always applied to the C dimension (N is the batch dimension which we do not keep track of). ONNX accepts input in the form of (N, C, D1, … , Dn) where N is the batch dimension and C is the dimension to which the batchnorm is applied. It should also be noted that at present the pytorch constructors do not support the setting of weight and bias explicitly.

num_features

Number of input and output feature of the Batch Normalization Layer.

Type:

int

weight

Tensor containing the weight parameters of the Batch Normalization Layer. (default: None)

Type:

torch.Tensor, Optional

bias

Tensor containing the bias parameter of the Batch Normalization Layer. (default: None)

Type:

torch.Tensor, Optional

running_mean

Tensor containing the running mean parameter of the Batch Normalization Layer. (default: None)

Type:

torch.Tensor, Optional

running_var

Tensor containing the running var parameter of the Batch Normalization Layer. (default: None)

Type:

torch.Tensor, Optional

eps

Value added to the denominator for numerical stability (default: 1e-5).

Type:

float, Optional

momentum

Value used for the running_mean and running_var computation. Can be set to None for cumulative moving average (default: 0.1)

Type:

float, Optional

affine

When set to True, the module has learnable affine parameter (default: True).

Type:

bool, Optional

track_running_stats

When set to True, the module tracks the running mean and variance, when set to false the module does not track such statistics and always uses batch statistics in both training and eval modes (default: True).

Type:

bool, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.ConvNode(identifier, in_dim, out_channels, kernel_size, stride, padding, dilation, groups, has_bias=False, bias=None, weight=None)

Bases: ConcreteLayerNode

A class used for our internal representation of a Convolutional layer. Also in this case the pytorch and onnx representation present incompatibilities. As in Batchnorm pytorch provide 3 different class for convolution based on the dimensionality of the input considered. Moreover, the padding is forced to be symmetric. The dimensionality supported for the input are (N, C, L), (N, C, H, W) and (N, C, D, H, W). In ONNX the padding can be asymmetric and the dimensionality supported is (N, C, D1, … , Dn) where D1, … Dn are the dimension on which the convolution is applied

in_channels

Number of input channels in Conv Layer.

Type:

int

out_channels

Number of output channels in Conv Layer.

Type:

int

kernel_size

The size of the kernel. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

stride

Stride along each spatial axis. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

padding

Padding for the beginning and ending along each spatial axis. Padding format should be as follows [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. Should have size equal to two times the number of dimension n (we don’t count the channel dimension).

Type:

tuple

dilation

Dilation value along each spatial axis of the filter

Type:

tuple

groups

Number of groups input channels and output channels are divided into

Type:

int

has_bias

Flag True if the convolutional layer has bias, False otherwise (default: False)

Type:

bool, Optional

bias

Tensor containing the bias parameter of the Conv Layer (default: None)

Type:

torch.Tensor, Optional

weight

Tensor containing the weight parameters of the Conv layer (default: None)

Type:

torch.Tensor, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.AveragePoolNode(identifier, in_dim, kernel_size, stride, padding, ceil_mode=False, count_include_pad=False)

Bases: ConcreteLayerNode

A class used for our internal representation of a AveragePool layer. Also in this case the pytorch and onnx representation present incompatibilities. As in Batchnorm pytorch provide 3 different class for pooling based on the dimensionality of the input considered. Moreover, the padding is forced to be symmetric and the parameter divisor_override is present (it is not clear what is its effect). The dimensionality supported for the input are (N, C, L), (N, C, H, W) and (N, C, D, H, W). In ONNX the padding can be asymmetric and the dimensionality supported is (N, C, D1, … , Dn) where D1, … Dn are the dimension on which the pooling is applied

kernel_size

The size of the kernel. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

stride

Stride along each spatial axis. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

padding

Padding for the beginning and ending along each spatial axis. Padding format should be as follows [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. Should have size equal to two times the number of dimension n (we don’t count the channel dimension).

Type:

tuple

ceil_mode

In order to use ceil mode. (default: False)

Type:

bool, Optional

count_include_pad

Whether include pad pixels when calculating values for the edges (default: False)

Type:

bool, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.MaxPoolNode(identifier, in_dim, kernel_size, stride, padding, dilation, ceil_mode=False, return_indices=False)

Bases: ConcreteLayerNode

A class used for our internal representation of a MaxPool layer. Also in this case the pytorch and onnx representation present incompatibilities. As in Batchnorm pytorch provide 3 different class for pooling based on the dimensionality of the input considered. Moreover, the padding is forced to be symmetric. The dimensionality supported for the input are (N, C, L), (N, C, H, W) and (N, C, D, H, W). In ONNX the padding can be asymmetric and the dimensionality supported is (N, C, D1, … , Dn) where D1, … Dn are the dimension on which the pooling is applied

kernel_size

The size of the kernel. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

stride

Stride along each spatial axis. Should have size equal to the number of dimension n (we don’t count the channel dimension).

Type:

tuple

padding

Padding for the beginning and ending along each spatial axis. Padding format should be as follows [x1_begin, x2_begin…x1_end, x2_end,…], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. Should have size equal to two times the number of dimension n (we don’t count the channel dimension).

Type:

tuple

dilation

Dilation value along each spatial axis of the filter

Type:

tuple

ceil_mode

In order to use ceil mode. (default: False)

Type:

bool, Optional

return_indices

If True it will return the max indices along with the outputs (default: False)

Type:

bool

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.LRNNode(identifier, in_dim, size, alpha=0.0001, beta=0.75, k=1.0)

Bases: ConcreteLayerNode

A class used for our internal representation of a LocalResponseNormalization Layer.

size

Amount of neighbouring channels used for normalization

Type:

int

alpha

Multiplicative factor (default: 0.0001)

Type:

float, Optional

beta

Exponent. (default: 0.75)

Type:

float, Optional

k

Additive factor (default: 1.0)

Type:

float, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.SoftMaxNode(identifier, in_dim, axis=-1)

Bases: ConcreteLayerNode

A class used for our internal representation of a SoftMax Layer.

axis

A dimension along which Softmax will be computed (so every slice along dim will sum to 1)

Type:

int, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.UnsqueezeNode(identifier, in_dim, axes)

Bases: ConcreteLayerNode

A class used for our internal representation of an Unsqueeze Layer. We follow the ONNX operator convention for attributes and definitions.

axes

List of indices at which to insert the singleton dimension.

Type:

tuple

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.ReshapeNode(identifier, in_dim, shape, allow_zero=False)

Bases: ConcreteLayerNode

A class used for our internal representation of a Reshape layer. We follow the ONNX operator convention for attributes and definitions.

shape

tuple which specifies the output shape

Type:

tuple

allow_zero

By default, when any value in the ‘shape’ input is equal to zero the corresponding dimension value is copied from the input tensor dynamically. allowzero=1 indicates that if any value in the ‘shape’ input is set to zero, the zero value is honored, similar to NumPy. (default: False)

Type:

bool, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.FlattenNode(identifier, in_dim, axis=0)

Bases: ConcreteLayerNode

A class used for our internal representation of a Flatten layer. We follow the ONNX operator convention for attributes and definitions.

axis

Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output. The value for axis must be in the range [-r, r], where r is the rank of the input tensor. Negative value means counting dimensions from the back. When axis = 0, the shape of the output tensor is (1, (d_0 X d_1 … d_n)), where the shape of the input tensor is (d_0, d_1, … d_n). N.B: it works assuming the initial batch dimension. (default: 0)

Type:

int, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.DropoutNode(identifier, in_dim, p=0.5)

Bases: ConcreteLayerNode

A class used for our internal representation of a Dropout Layer. The inplace parameter of pytorch and the seed attribute and training_mode of onnx are not supported.

p

Probability of an element to be zeroed (default: 0.5)

Type:

float, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.TransposeNode(identifier, in_dim, perm=None)

Bases: ConcreteLayerNode

A class used for our internal representation of a Dropout Layer. The inplace parameter of pytorch and the seed attribute and training_mode of onnx are not supported.

perm

Permutation to apply to the input dimensions

Type:

list, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.ConcatNode(identifier, in_dims, axis=-1)

Bases: ConcreteLayerNode

A class used for our internal representation of a Concat Layer. Concatenate two tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.

axis

Which axis to concat on. A negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r is the number of dimension of the input (default: -1).

Type:

int, Optional

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple

class pynever.nodes.SumNode(identifier, in_dims)

Bases: ConcreteLayerNode

A class used for our internal representation of a Sum Layer. Element-wise sum of each of the input tensors. All inputs and outputs must have the same data type.

get_input_dim()

Should be implemented in children depending on whether they have one or more input dimensions.

Returns:

The list of input dimensions if the layer has more than one input, otherwise the first element.

Return type:

list[tuple] | tuple