neon.data.image.CIFAR10

class neon.data.image.CIFAR10(path='.', subset_pct=100, normalize=True, contrast_normalize=False, whiten=False, pad_classes=False)[source]

Bases: neon.data.datasets.Dataset

CIFAR10 data set from https://www.cs.toronto.edu/~kriz/cifar.html

Parameters:
  • path (str) – Local path to copy data files.
  • normalize (bool) – Flag to normalize data.
  • whiten (bool) – Flag to apply whitening transform.
  • pad_classes (bool) – Flag to pad out class count to 16 for compatibility with conv layers on GPU.
__init__(path='.', subset_pct=100, normalize=True, contrast_normalize=False, whiten=False, pad_classes=False)[source]

Methods

__init__([path, subset_pct, normalize, …])
fetch_dataset(url, sourcefile, destfile, totalsz) Download the file specified by the given URL.
gen_class(pdict)
gen_iterators()
get_description([skip]) Returns a dict that contains all necessary information needed to serialize this object.
get_iterator(setname) Helper method to get the data iterator for specified dataset
global_contrast_normalize(X[, scale, …]) Subtract mean and normalize by vector norm.
load_data() Fetch the CIFAR-10 dataset and load it into memory.
load_zip(filename, size) Helper function for downloading test files
recursive_gen(pdict, key) helper method to check whether the definition
serialize() Generates dictionary with the required parameters to describe this object
zca_whiten(train, test[, cache]) Use train set statistics to apply the ZCA whitening transform to both train and test sets.
be = None
classnm

Returns the class name.

data_dict
fetch_dataset(url, sourcefile, destfile, totalsz)

Download the file specified by the given URL.

Parameters:
  • url (str) – Base URL of the file to be downloaded.
  • sourcefile (str) – Name of the source file.
  • destfile (str) – Path to the destination.
  • totalsz (int) – Size of the file to be downloaded.
gen_class(pdict)
gen_iterators()[source]
get_description(skip=[], **kwargs)

Returns a dict that contains all necessary information needed to serialize this object.

Parameters:skip (list) – Objects to omit from the dictionary.
Returns:Dictionary format for object information.
Return type:(dict)
get_iterator(setname)

Helper method to get the data iterator for specified dataset

Parameters:setname (str) – which iterator to return (e.g. ‘train’, ‘valid’)
static global_contrast_normalize(X, scale=1.0, min_divisor=1e-08)[source]

Subtract mean and normalize by vector norm.

load_data()[source]

Fetch the CIFAR-10 dataset and load it into memory.

Parameters:
  • path (str, optional) – Local directory in which to cache the raw dataset. Defaults to current directory.
  • normalize (bool, optional) – Whether to scale values between 0 and 1. Defaults to True.
Returns:

Both training and test sets are returned.

Return type:

tuple

load_zip(filename, size)

Helper function for downloading test files Will download and unzip the file into the directory self.path

Parameters:
  • filename (str) – name of file to download from self.url
  • size (str) – size of the file in bytes?
Returns:

Path to the downloaded dataset.

Return type:

str

modulenm

Returns the full module path.

recursive_gen(pdict, key)

helper method to check whether the definition dictionary is defining a NervanaObject child, if so it will instantiate that object and replace the dictionary element with an instance of that object

serialize()

Generates dictionary with the required parameters to describe this object

test_iter

Helper method to return test set iterator

train_iter

Helper method to return training set iterator

valid_iter

Helper method to return validation set iterator

static zca_whiten(train, test, cache=None)[source]

Use train set statistics to apply the ZCA whitening transform to both train and test sets.