dislib.eddl#

class dislib.eddl.EncapsulatedFunctionsDistributedEddl(num_workers=10)[source]#

Bases: object

Object that encapsulates the different distributed trainings that can be done using PyCOMPSs. Each function implements a different version, the number of epochs and batches is specified in each of the functions.

There are mainly three different types of training.

  • Synchronous training: At the end of each epoch, the weights are synchronized and the update is computed.

  • Partially asynchronous: The weights of each worker are updated commutatively with the general weights and vice versa.

  • Asynchronous training: A synchronization and update of the weights is done after executing all the epochs or each n specified epochs.

Variables:

  • model_parameters (tensor) – weights and biases of the different layers of the network that is being trained.

  • compss_object (list) – List that contains objects of type EddlDistributedConmutativo, each of the objects in this list makes a small part of the epoch training in parallel to the rest.

  • num_workers (int) – Number of parallel trainings existing.

build(net, optimizer, loss, metric, num_gpu=0, num_nodes=0)[source]#

Builds the model to obtain the initial training parameters and prepares a copy in each worker, ready to start the training.

Parameters:

  • net (eddl.Model) – Neural network model to be used during the parallel training.

  • optimizer (dict) – Dictionary containing the optimizer to be used and its parameters.

  • loss (str) – String specifying the loss to be used during the training.

  • metric (str) – Metric to be used during the training.

  • num_gpu (int) – Number of GPUs to use during the training.

  • num_nodes (int) – Number of nodes available during the training.

Return type:

(void)

fit_asynchronous_n_epochs_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, n_epocs_sync=0, shuffle_blocks=True, shuffle_block_data=True)[source]#

Training of the neural network performing an asynchronous update of the weights every n epochs, shuffling the tensors and locally shuffling the elements inside each tensor

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • n_epocs_sync (int) – Number of epochs to train before performing an asynchronous update of the weights and between the following updates

  • shuffle_blocks (boolean) – Variable specifying to shuffle the blocks of the ds_tensor or not

  • shuffle_block_data (boolean) – Variable specifying whether to shuffle the elements inside each tensor locally or not

Returns:

model_parameters

Return type:

np.array

fit_asynchronous_shuffle_n_epochs_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, n_epocs_sync=0)[source]#

Training of the neural network performing an asynchronous update of the weights every n epochs, performing a total shuffle of the dataset

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • n_epocs_sync (int) – Number of epochs to train before performing an asynchronous update of the weights and between the following updates

Returns:

model_parameters

Return type:

np.array

fit_asynchronous_shuffle_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs)[source]#

Training of the neural network performing an asynchronous update of the weights every epoch, performing a total shuffle of the dataset

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

Returns:

model_parameters

Return type:

np.array

fit_asynchronous_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, shuffle_blocks=True, shuffle_block_data=True)[source]#

Training of the neural network performing an asynchronous update of the weights every epoch, shuffling the tensors of the ds_tensor and locally shuffling the elements inside each tensor

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • shuffle_blocks (boolean) – Variable specifying to shuffle the blocks of the ds_tensor or not

  • shuffle_block_data (boolean) – Variable specifying whether to shuffle the elements inside each tensor locally or not

Returns:

model_parameters

Return type:

np.array

fit_synchronous_every_n_epochs_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, n_epocs_sync=1, shuffle_blocks=True, shuffle_block_data=True)[source]#

Training of the neural network performing a synchronization every n specified epochs, shuffling the tensors of the ds_tensor and the elements inside each tensor

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • n_epocs_sync (int) – Number of epochs to train before performing a synchronization and between synchronizations

  • shuffle_blocks (boolean) – Variable specifying to shuffle the blocks of the ds_tensor or not

  • shuffle_block_data (boolean) – Variable specifying whether to shuffle the elements inside each tensor locally or not

Returns:

model_parameters

Return type:

np.array

fit_synchronous_shuffle_every_n_epochs_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, n_epocs_sync=1)[source]#

Training of the neural network performing a synchronization every n specified epochs, performing a total shuffle of the dataset.

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • n_epocs_sync (int) – Number of epochs to train before performing a synchronization and between synchronizations

Returns:

model_parameters

Return type:

np.array

fit_synchronous_shuffle_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs)[source]#

Training of the neural network performing a synchronization of the weights every epoch, performing a total shuffle of the dataset

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

Returns:

model_parameters

Return type:

np.array

fit_synchronous_with_GPU(x_train, y_train, num_batches_per_worker, num_epochs, shuffle_blocks=True, shuffle_block_data=True)[source]#

Training of the neural network performing a synchronization of the weights at the end of each epoch, shuffling the tensors of the ds_tensor and the elements inside each tensor

Parameters:

  • x_train (ds_tensor) – samples and features of the training dataset

  • y_train (ds_tensor) – classes or values of the samples of the training dataset

  • num_batches_per_worker (int) – Number of batches that each worker will be trained with every piece of the dataset

  • num_epochs (int) – Total number of epochs to train the model

  • shuffle_blocks (boolean) – Variable specifying to shuffle the blocks of the ds_tensor or not

  • shuffle_block_data (boolean) – Variable specifying whether to shuffle the elements inside each tensor locally or not

Returns:

model_parameters

Return type:

np.array

get_parameters()[source]#

Returns the parameters (weights) of the neural network :returns: model_parameters :rtype: np.array