HybridQRBM package

Submodules

HybridQRBM.callbacks module

Some callbacks to pass to the RBM’s fit() method.

class HybridQRBM.callbacks.Sample(visible, visible_prob, hidden, hidden_prob)

Bases: tuple

hidden

Alias for field number 2

hidden_prob

Alias for field number 3

visible

Alias for field number 0

visible_prob

Alias for field number 1

HybridQRBM.callbacks.get_auc_calculator(features: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], labels: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → callable

Return a function that calculates the AUC of a model using the given dataset (features) and labels. Using this function ensures that these are not overwritten by the locals() passed to the callback. This functionality allows to pass AUC calculating functions with different datasets, e.g. training and validation.

Parameters

• features (numpy.ndarray) – The features of the dataset.

• labels (numpy.ndarray) – The labels of the dataset.

Returns

A function (callback) that calculates the AUC of a model.

Return type

callable

HybridQRBM.callbacks.produce_logger(message: str) → callable

Produce a logger callback. The function will receive the locals of either the fit() or the _fit_batch() methods. These can be used in the message.

Parameters

message (str) – The message to log. For example, “Epoch {epoch}” or “{self.name}”. In the second case, self refers to the RBM instance.

Returns

log_ – The logger callback.

Return type

callable

HybridQRBM.callbacks.save_particles(self, negative_sample: numpy.ndarray[Any, numpy.dtype[numpy.int64]], epoch: int, batch_num: int, **kwargs) → None

Save a visualization of the particles produced from the model distribution.

The callback also saves some additional information to inspect how the training is going.

Parameters

• self – The RBM instance that calls the function.

• negative_sample (numpy.ndarray) – The particles produced by the sampler in the negative phase of the training.

• epoch (int) –

• batch_num (int) –

• **kwargs – locals() passed from the method that calls the callback.

HybridQRBM.optimizers module

Optimizer to update the weights of the RBM.

class HybridQRBM.optimizers.Parameters(weights, biases_visible, biases_hidden)

Bases: tuple

biases_hidden

Alias for field number 2

biases_visible

Alias for field number 1

weights

Alias for field number 0

class HybridQRBM.optimizers.RBMOptimizer(learning_rate: float = 0.05, momentum: float = 0.9, decay_factor: float = 1.0005, regularizers: tuple = ())

Bases: object

Optimizer to update the weights of the RBM.

calculate_update(positive_sample, negative_sample)

Calculate the update for the weights and biases.

Parameters

• positive_sample (tuple) – The values and probabilities of the visible and hidden layers produced in the positive phase of the training. The tuple should contain four elements: (visible_values, visible_probabilities, hidden_values, hidden_probabilities).

• negative_sample (tuple) – The values and probabilities of the visible and hidden layers produced in the negative phase of the training. The tuple should contain four elements: (visible_values, visible_probabilities, hidden_values, hidden_probabilities).

Returns

The delta values for the update of the weights and biases.

Return type

namedtuple

HybridQRBM.pytorchdnx module

Pytorch Dynex Neuromporhic Layer.

class HybridQRBM.pytorchdnx.dnx(num_hidden: int, steps_per_epoch: int, sampler: Sampler, optimizer: RBMOptimizer, rnd: numpy.random.mtrand.RandomState = None, name: str = None, num_gibbs_updates=1, mainnet=False, logging=False, num_reads=100, annealing_time=1000, debugging=False, minimum_stepsize=0.002)

Bases: torch.nn.modules.module.Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

to_qubo_matrix(data: Optional[numpy.ndarray] = None, clamp_strength: float = 2.0)

Generates a QUBO matrix from the RBM weights and biases.

The resulting matrix is of shape (num_visible + num_hidden, num_visible + num_hidden) with weights in the upper right (num_visible, num_hidden) block and biases on the the diagonal.

Parameters

• data (np.ndarray) – 2D binary array of shape (num_samples, num_features). If data is passed, the first num_features columns are fixed to the given values.

• clamp_strength (float) – If data array is passed, the corresponding visible unit bias values are overriden and set to +/- clamp_strength * (maximum weight or bias value). If the value is set too low, the returned samples may differ from the data. If it is too high, it might not be possible to embed to QUBO matrix on the D-Wave annealer without scaling the other values too much.

training: bool

class HybridQRBM.pytorchdnx.dnx_experimental(n_visible, n_hidden, batch_size=1, lr=0.1, lr_decay=0.1, mainnet=False, logging=False, num_reads=4096, annealing_time=1000)

Bases: torch.nn.modules.module.Module

Dynex QRBM Layer

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

sample_opposite_layer_pyqubo(v, layer, weights, opposite_layer)

training: bool

HybridQRBM.rbm module

“Restricted Boltzmann Machine

class HybridQRBM.rbm.Callbacks(on_fit_start, on_epoch_start, on_batch_start, on_batch_end, on_epoch_end, on_fit_end)

Bases: tuple

on_batch_end

Alias for field number 3

on_batch_start

Alias for field number 2

on_epoch_end

Alias for field number 4

on_epoch_start

Alias for field number 1

on_fit_end

Alias for field number 5

on_fit_start

Alias for field number 0

class HybridQRBM.rbm.RBM(num_hidden: int, sampler: Sampler, optimizer: RBMOptimizer, name: str = None, rnd: numpy.random.mtrand.RandomState = None, debug=False)

Bases: object

Restricted Boltzmann Machine

To run, the RBM requires two additional components — a sampler and an optimizer.

energy(visible: numpy.ndarray[Any, numpy.dtype[numpy.int64]], hidden: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → numpy.ndarray[Any, numpy.dtype[float]]

Calculates the energy of a given state of the RBM.

Parameters

• visible (np.ndarray(dtype=np.int0)) – 2D array of shape (num_samples, num_visible).

• hidden (np.ndarray(dtype=np.int0)) – 2D array of shape (num_samples, num_hidden).

Returns

1D array of shape (num_samples,).

Return type

np.ndarray(dtype=np.float)

fit(features: numpy.ndarray[typing.Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], labels: typing.Optional[numpy.ndarray[typing.Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]]] = None, epochs: int = 20, batch_size: int = 100, callbacks: HybridQRBM.rbm.Callbacks = Callbacks(on_fit_start=[], on_epoch_start=[], on_batch_start=[], on_batch_end=[], on_epoch_end=[], on_fit_end=[<function produce_logger.<locals>.log_>])) → float

Fit the RBM to the data.

If labels are given, these are combined with the features to form the visible layer.

Parameters

• features (npt.NDArray) – 2D binary or float array of shape (num_samples, num_features). If the values are floats, they are binarized in each batch update.

• labels (Optional[npt.NDArray]) – Optional 2D binary or float array of shape (num_samples, num_labels). Generaly binary values are expected, but if floats are passed, they are binarized in each batch update.

• epochs (int) – The number of epochs to train for. Alternatively, the number of times each datapoint is used in weight update calculation.

• batch_size (int) – The number of datapoints to use in each weight update calculation.

• callbacks (Callbacks) – Namedtuple of lists of callback functions. The following hooks are provided: on_fit_start, on_epoch_start, on_batch_start, on_batch_end, on_epoch_end, on_fit_end. The default callback offers some basic logging. If you pass any custom callbacks, you also need to supply your own callbacks for logging.

Returns

The average reconstruction error of the last epoch. Reconstruction error represents the similarity between the input data and visible layer after sampling with input data as the initial state. It does not fully represent the quality of the model, but is a good proxy during training.

Return type

float

fit_range(features: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]])

For some models, the range of the predictions are much more limited than the range [0, 1]. This function learns the range of the predictions for each class. This information can be used to rescale the predictions to the range [0, 1].

Parameters

features (npt.NDArray) – 2D binary or float array of shape (num_samples, num_features).

Returns

Return an array of shape (2, num_labels), where the first rows are the minima of the predictions, and second row — the maxima.

Return type

np.ndarray

predict(features: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], num_particles: int = 100, rescale: bool = False, **kwargs) → numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]]

Predicts the labels of the given features.

Parameters

• features (npt.NDArray) – 2D array of shape (num_samples, num_features).

• rescale (bool) – Rescale the predictions to the range [0, 1].

• num_particles – If this method is used with DWaveInferenceSampler, an important parameter is the num_particles, which determines the number of reads to do on the D-Wave annealer.

Returns

2D array of shape (num_samples, num_labels).

Return type

npt.NDArray

to_qubo_matrix(data: Optional[numpy.ndarray] = None, clamp_strength: float = 2.0)

Generates a QUBO matrix from the RBM weights and biases.

The resulting matrix is of shape (num_visible + num_hidden, num_visible + num_hidden) with weights in the upper right (num_visible, num_hidden) block and biases on the the diagonal.

Parameters

• data (np.ndarray) – 2D binary array of shape (num_samples, num_features). If data is passed, the first num_features columns are fixed to the given values.

• clamp_strength (float) – If data array is passed, the corresponding visible unit bias values are overriden and set to +/- clamp_strength * (maximum weight or bias value). If the value is set too low, the returned samples may differ from the data. If it is too high, it might not be possible to embed to QUBO matrix on the D-Wave annealer without scaling the other values too much.

HybridQRBM.samplers module

Samplers for the RBM training.

class HybridQRBM.samplers.ContrastiveDivergenceSampler(num_gibbs_updates: int)

Bases: HybridQRBM.samplers.Sampler

Implements sampling for contrastive divergence training.

sample(visible: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], **kwargs) → tuple

Implements sampling for contrastive divergence training.

The expected use is to start with feature values for the visible layer and then perform a number (defined in the constructor) of Gibbs updates. If the number of Gibbs updates is 1, this implements the standard CD-1 training.

Parameters

visible (numpy.ndarray) – The visible layer values. Shape (num_samples, num_visible).

Returns

visible, prob_visible, hidden, prob_hidden – See gibbs_updates() method for details.

Return type

tuple

class HybridQRBM.samplers.DWaveInferenceSampler(dwave_sampler, num_spin_reversal_transforms, temp=1.0, num_gibbs_updates=0, chain_strength=None, dwave_params=None, **kwargs)

Bases: HybridQRBM.samplers.DWaveSampler

Extends DWaveSampler with inference / prediction.

predict(features: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], num_particles: int = 100, num_gibbs_updates: Optional[int] = None, use_cache=False, **kwargs) → numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]]

Predict the labels for the given feature values.

Parameters

• features (numpy.ndarray[np.int0]) – The feature values to predict the labels for. Shape (num_samples, num_features).

• num_particles (int) – Number of particles to use for the sampling, i.e. how may times to run the label sampling process for each sample.

• num_gibbs_updates (int, optional) – Number of Gibbs updates to perform for each particle. If not provided, self.num_gibbs_updates will be used.

• use_cache (bool) – If set to True, and a cached response is available, it will be used.

Returns

labels – The predicted labels. Shape (num_samples, num_label_classes).

Return type

numpy.ndarray[np.float]

class HybridQRBM.samplers.DWaveSampler(dwave_sampler, num_spin_reversal_transforms, temp=1.0, num_gibbs_updates=0, chain_strength=None, dwave_params=None, **kwargs)

Bases: HybridQRBM.samplers.Sampler

Implements sampling for the D-Wave machine.

sample(visible, **kwargs)

Implements sampling for the D-Wave machine.

Parameters

visible (numpy.ndarray) – The visible layer values are always passed from the RBM. When sampling from the model distribution, it is only used to determine the shape of the sample.

Returns

visible, prob_visible, hidden, prob_hidden – visible and hidden are the samples returned by the DWave sampler. prob_visible and prob_hidden are calculated based on the DWave sample values.

Return type

tuple

class HybridQRBM.samplers.DynexSampler(num_reads=100, annealing_time=300, mainnet=False, minimum_stepsize=6e-08, logging=True, debugging=False, num_gibbs_updates=0, **kwargs)

Bases: HybridQRBM.samplers.Sampler

sample(visible, **kwargs)

Implements sampling for the Dynex Neuromorphic Platform.

Parameters

visible (numpy.ndarray) – The visible layer values are always passed from the RBM. When sampling from the model distribution, it is only used to determine the shape of the sample.

Returns

visible, prob_visible, hidden, prob_hidden – visible and hidden are the samples returned by the Dynex sampler. prob_visible and prob_hidden are calculated based on the Dynex sample values.

Return type

tuple

class HybridQRBM.samplers.NaiveSampler(num_gibbs_updates: int)

Bases: HybridQRBM.samplers.Sampler

Implements sampling for naive training.

sample(visible, **kwargs)

Implements sampling for naive training.

The particles (Markov chains) for the negative phase are initialized to random values at the beginning of each batch update. If the Markov chains are properly burnt in, the sampled particles represent the model distribution better than those from either contrastive divergence or persistent contrastive divergence; howevere the required number of Gibbs updates to burn in the Markov chains is unknown and likely high.

Parameters

visible (numpy.ndarray) – The visible layer values are always passed from the RBM. For this sampler it is only used to determine the shape of the sample.

Returns

visible, prob_visible, hidden, prob_hidden – See gibbs_updates() method for details.

Return type

tuple

class HybridQRBM.samplers.PersistentContrastiveDivergenceSampler(*args, **kwargs)

Bases: HybridQRBM.samplers.Sampler

Implements sampling for persistent contrastive divergence training.

sample(visible, **kwargs) → tuple

Performs a number of Gibbs updates, starting from the state of the visible layer after the previous update.

Parameters

visible (numpy.ndarray) – The visible layer values are always passed from the RBM. For this sampler it is only used to determine the shape of the sample.

Returns

visible, prob_visible, hidden, prob_hidden – See gibbs_updates() method for details.

Return type

tuple

class HybridQRBM.samplers.Sampler(num_gibbs_updates: int)

Bases: object

Base class for samplers.

generate(hidden: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]])

Generate the visible layer given the hidden layer.

Parameters

hidden (numpy.ndarray) – The hidden layer. Shape (num_samples, num_hidden).

Returns

• visible (numpy.ndarray) – Binary visible layer values sampled from their probability distribution. Shape (num_samples, num_visible).

• prob_visible (numpy.ndarray) – The probability for each unit of the visible layer to be 1. Shape (num_samples, num_visible).

gibbs_updates(visible: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], num_gibbs_updates=None)

Perform Gibbs sampling starting from some visible layer values.

Parameters

• visible (numpy.ndarray) – The initial visible layer values. Shape (num_samples, num_visible).

• num_gibbs_updates (int) – The number of full updates to perform. If None, the default for the sampler is performed.

Returns

• visible (numpy.ndarray) – The visible layer after the Gibbs sampling. Shape (num_samples, num_visible).

• prob_visible (numpy.ndarray) – The probability for each unit of the visible layer to be 1. Shape (num_samples, num_visible).

• hidden (numpy.ndarray) – Binary hidden layer values sampled from their probability distribution. Shape (num_samples, num_hidden).

• prob_hidden (numpy.ndarray) – The probability for each unit of the hidden layer to be 1. Shape (num_samples, num_hidden).

infer(visible: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]])

Infer the hidden layer given the visible layer.

Parameters

visible (numpy.ndarray) – The visible layer. Shape (num_samples, num_visible).

Returns

• hidden (numpy.ndarray) – Binary hidden layer values sampled from their probability distribution. Shape (num_samples, num_hidden).

• prob_hidden (numpy.ndarray) – The probability for each unit of the hidden layer to be 1. Shape (num_samples, num_hidden).

predict(features: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], num_particles: int = 10, num_gibbs_updates: Optional[int] = None, **kwargs) → numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]]

Predict the features and labels for the given feature values.

Parameters

• features (numpy.ndarray[np.int0]) – The feature values to predict the labels for. Shape (num_samples, num_features).

• num_particles (int) – Number of particles to use for the sampling, i.e. how may times to run the label sampling process for each sample.

• num_gibbs_updates (int, optional) – Number of Gibbs updates to perform for each particle. If not provided, self.num_gibbs_updates will be used.

Returns

• labels (numpy.ndarray[np.float]) – The predicted labels. Shape (num_samples, num_labels).

• features (numpy.ndarray[np.float]) – The predicted features. Shape (num_samples, num_features).

sample(visible: numpy.ndarray[Any, numpy.dtype[numpy.typing._generic_alias.ScalarType]], **kwargs)

Abstract method for sampling from the model distribution. Possibly with some clamped visible layer values.

HybridQRBM.utils module

class HybridQRBM.utils.CXRDataset_pca(root_dir, dataset_type='train', dataset_file=None, num_features=- 1, include_classes=(0, None))

Bases: object

HybridQRBM.utils.binary(x)

HybridQRBM.utils.binary_three_bits(x)

HybridQRBM.utils.encode(x, func)

HybridQRBM.utils.l2_regularizer(weight_cost=0.0002)

HybridQRBM.utils.load_data(data_root_dir, pca_features, encoding=None)

Module contents