View source on GitHub |
A module containing optimization routines.
Classes
class AGNCustomGetter
: Custom_getter class is used to do:
class AGNOptimizer
: Wrapper that implements the Accumulated GradientNormalization algorithm.
class AdaMaxOptimizer
: Optimizer that implements the AdaMax algorithm.
class AdamGSOptimizer
: Optimizer that implements the Adam algorithm.
class AdamWOptimizer
: Optimizer that implements the Adam algorithm with weight decay.
class AddSignOptimizer
: Optimizer that implements the AddSign update.
class DecoupledWeightDecayExtension
: This class allows to extend optimizers with decoupled weight decay.
class DropStaleGradientOptimizer
: Wrapper optimizer that checks and drops stale gradient.
class ElasticAverageCustomGetter
: Custom_getter class is used to do:
class ElasticAverageOptimizer
: Wrapper optimizer that implements the Elastic Average SGD algorithm.
class ExternalOptimizerInterface
: Base class for interfaces with external optimization algorithms.
class GGTOptimizer
: Optimizer that implements the GGT algorithm.
class LARSOptimizer
: Layer-wise Adaptive Rate Scaling for large batch training.
class LazyAdamGSOptimizer
: Variant of the Adam optimizer that handles sparse updates more efficiently.
class LazyAdamOptimizer
: Variant of the Adam optimizer that handles sparse updates more efficiently.
class ModelAverageCustomGetter
: Custom_getter class is used to do.
class ModelAverageOptimizer
: Wrapper optimizer that implements the Model Average algorithm.
class MomentumWOptimizer
: Optimizer that implements the Momentum algorithm with weight_decay.
class MovingAverageOptimizer
: Optimizer that computes a moving average of the variables.
class MultitaskOptimizerWrapper
: Optimizer wrapper making all-zero gradients harmless.
class NadamOptimizer
: Optimizer that implements the Nadam algorithm.
class PowerSignOptimizer
: Optimizer that implements the PowerSign update.
class RegAdagradOptimizer
: RegAdagrad: Adagrad with updates that optionally skip updating the slots.
class ScipyOptimizerInterface
: Wrapper allowing scipy.optimize.minimize
to operate a tf.compat.v1.Session
.
class ShampooOptimizer
: The Shampoo Optimizer
class VariableClippingOptimizer
: Wrapper optimizer that clips the norm of specified variables after update.
Functions
clip_gradients_by_global_norm(...)
: Clips gradients of a multitask loss by their global norm.
extend_with_decoupled_weight_decay(...)
: Factory function returning an optimizer class with decoupled weight decay.