Perform hybrid quantized dot of float Tensor `lhs` and quantized Tensor `rhs`.
Given float `lhs` and quantized `rhs`, internally performs quantization on `lhs`, and then performs quantized dot on quantized lhs and `rhs`. The internal quantization on `lhs` is a quantization to qint8, dynamic range, per-batch (per-axis along axis 0), asymmetric, and not narrow range (the range is [-128, 127]). `lhs` and `rhs` must be 2D Tensors and the lhs.dim_size(1) must match rhs.dim_size(0). `rhs` must be quantized Tensor, where its data value is quantized using the formula: quantized_data = clip(original_data / scale + zero_point, quantization_min_val, quantization_max_val).
Nested Classes
| class | UniformQuantizedDotHybrid.Options | Optional attributes for UniformQuantizedDotHybrid
|
|
Public Methods
| Output<V> |
asOutput()
Returns the symbolic handle of a tensor.
|
| static <V extends Number, T extends Number, U> UniformQuantizedDotHybrid<V> | |
| Output<V> |
output()
The output 2D Tensor of Tout, whose shape is (lhs.dim_size(0), rhs.dim_size(1)).
|
| static UniformQuantizedDotHybrid.Options |
rhsQuantizationAxis(Long rhsQuantizationAxis)
|
Inherited Methods
Public Methods
public Output<V> asOutput ()
Returns the symbolic handle of a tensor.
Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.
public static UniformQuantizedDotHybrid<V> create (Scope scope, Operand<T> lhs, Operand<U> rhs, Operand<Float> rhsScales, Operand<Integer> rhsZeroPoints, Class<V> Tout, Long rhsQuantizationMinVal, Long rhsQuantizationMaxVal, Options... options)
Factory method to create a class wrapping a new UniformQuantizedDotHybrid operation.
Parameters
| scope | current scope |
|---|---|
| lhs | Must be a 2D Tensor of Tlhs. |
| rhs | Must be a 2D Tensor of Trhs. |
| rhsScales | The float value(s) used as scale when quantizing original data that rhs represents. Must be a scalar Tensor (per-tensor quantization) or 1D Tensor of size (rhs.dim_size(1),) (per-channel quantization). |
| rhsZeroPoints | The int32 value(s) used as zero_point when quantizing original data that rhs represents. Same shape condition as rhs_scales. |
| Tout | The type of output Tensor. |
| rhsQuantizationMinVal | The min value of the quantized data stored in rhs. For example, if Trhs is qint8, this must be set to -127 if narrow range quantized or -128 if not. |
| rhsQuantizationMaxVal | The max value of the quantized data stored in rhs. For example, if Trhs is qint8, this must be set to 127. |
| options | carries optional attributes values |
Returns
- a new instance of UniformQuantizedDotHybrid
public Output<V> output ()
The output 2D Tensor of Tout, whose shape is (lhs.dim_size(0), rhs.dim_size(1)). The output data is the original output data itself (Not quantized).
public static UniformQuantizedDotHybrid.Options rhsQuantizationAxis (Long rhsQuantizationAxis)
Parameters
| rhsQuantizationAxis | Indicates the dimension index of the tensor where per-axis quantization is applied for the slices along that dimension. If set to -1 (default), this indicates per-tensor quantization. For dot op rhs, only per-tensor quantization or per-channel quantization along dimension 1 is supported. Thus, this attribute must be set to -1 or 1. Other values are rejected. |
|---|