neon.transforms.cost.CrossEntropyMulti

class neon.transforms.cost.CrossEntropyMulti(scale=1, usebits=False)[source]

Bases: neon.transforms.cost.Cost

Multi-class cross-entropy cost.

The multi-class cross-entropy cost is used when the labels have multiple classes. The cost is computed as \(C = \sum -t*\log(y)\), where \(t\) is the target label and \(y\) is the network output.

The target labels \(t\) are expected to be in an one-hot encoding. By default, the natural logarithm is used, but a cost that returns bits instead (e.g. log base 2) can also be specified with the usebits argument.

Note

The back-propogation assumes that this cost is coupled with an output layer that uses the Softmax() activation function. This allows for a shortcut in the deriviate that saves computation.

__init__(scale=1, usebits=False)[source]
Parameters:
  • scale (float, optional) – scale factor for the backpropagated error (default: 1)
  • usebits (boolean, optional) – Display costs in bits (default: False)

Methods

__init__([scale, usebits])
param scale:scale factor for the backpropagated error (default: 1)
bprop(y, t) Returns the derivative of the multiclass cross entropy cost.
gen_class(pdict)
get_description([skip]) Returns a dict that contains all necessary information needed to serialize this object.
recursive_gen(pdict, key) helper method to check whether the definition
be = None
bprop(y, t)[source]

Returns the derivative of the multiclass cross entropy cost.

Parameters:
  • y (Tensor or OpTree) – Output of previous layer or model
  • t (Tensor or OpTree) – True targets corresponding to y
Returns:

Returns the (mean) shortcut derivative of the multiclass entropy cost function (y - t) / y.shape[1]

Return type:

OpTree

classnm

Returns the class name.

gen_class(pdict)
get_description(skip=[], **kwargs)

Returns a dict that contains all necessary information needed to serialize this object.

Parameters:skip (list) – Objects to omit from the dictionary.
Returns:Dictionary format for object information.
Return type:(dict)
modulenm

Returns the full module path.

recursive_gen(pdict, key)

helper method to check whether the definition dictionary is defining a NervanaObject child, if so it will instantiate that object and replace the dictionary element with an instance of that object