tfp.experimental.linalg.LinearOperatorInterpolatedPSDKernel

View source on GitHub

Structured interpolation to approximate a large matrix.

tfp.experimental.linalg.LinearOperatorInterpolatedPSDKernel(
    kernel,
    bounds_min,
    bounds_max,
    num_interpolation_points,
    x1,
    x2=None,
    interp_fn=tfp.math.batch_interp_regular_nd_grid,
    diag_shift=None,
    is_non_singular=None,
    is_self_adjoint=None,
    is_positive_definite=None,
    is_square=None,
    name=None
)

This implements a component of the Structured Kernel Interpolation [1] algorithm. We approximate the pairwise kernel values for two inputs x1, x2 with the following product:

k(x1, x2) = r(x1) @ k(u, u) @ r(x2)^T

where u is a set of points regularly spaced on grid and r is an interpolation matrix. In short, instead of evaluating kernel on x1 and x2, we evaluate it on u, which is chosen to contain fewer points than x1 and x2 and the interpolate using the interpolation matrix.

This construction lets us compute matrix products efficiently. If x1/x2 are of shape O(n) and u is of shape O(m), this reduces the compute cost and memory to O(n + m^2) from O(n^2).

In practice, the interpolation matrix is implicitly defined using interp_fn, of which tfp.math.batch_interp_regular_nd_grid linear interpolation is a prototypical example. When x1 == x2 we can relatively cheaply compute the diagonal of that matrix exactly, to preserve positive-semi-definiteness.

Since we rely on a dense grid u, this method works best when the the kernel inputs are low dimensional (2 or 3).

References

[1]: Wilson, A. G., & Nickisch, H. Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP). 2005. arXiv. http://arxiv.org/abs/1503.01057

a = tf.Module()
b = tf.Module()
c = tf.Module()
a.b = b
b.c = c
list(a.submodules) == [b, c]
True
list(b.submodules) == [c]
True
list(c.submodules) == []
True

Methods

add_to_tensor

add_to_tensor(
    x, name='add_to_tensor'
)

Add matrix represented by this operator to x. Equivalent to A + x.

adjoint

adjoint(
    name: str = 'adjoint'
) -> 'LinearOperator'

Returns the adjoint of the current LinearOperator.

Given A representing this LinearOperator, return A*. Note that calling self.adjoint() and self.H are equivalent.

assert_non_singular

assert_non_singular(
    name='assert_non_singular'
)

Returns an Op that asserts this operator is non singular.

This operator is considered non-singular if

ConditionNumber < max{100, range_dimension, domain_dimension} * eps,
eps := np.finfo(self.dtype.as_numpy_dtype).eps

assert_positive_definite

assert_positive_definite(
    name='assert_positive_definite'
)

Returns an Op that asserts this operator is positive definite.

Here, positive definite means that the quadratic form x^H A x has positive real part for all nonzero x. Note that we do not require the operator to be self-adjoint to be positive definite.

assert_self_adjoint

assert_self_adjoint(
    name='assert_self_adjoint'
)

Returns an Op that asserts this operator is self-adjoint.

Here we check that this operator is exactly equal to its hermitian transpose.

batch_shape_tensor

batch_shape_tensor(
    name='batch_shape_tensor'
)

Shape of batch dimensions of this operator, determined at runtime.

If this operator acts like the batch matrix A with A.shape = [B1,...,Bb, M, N], then this returns a Tensor holding [B1,...,Bb].

cholesky

cholesky(
    name: str = 'cholesky'
) -> 'LinearOperator'

Returns a Cholesky factor as a LinearOperator.

Given A representing this LinearOperator, if A is positive definite self-adjoint, return L, where A = L L^T, i.e. the cholesky decomposition.

cond

cond(
    name='cond'
)

Returns the condition number of this linear operator.

determinant

determinant(
    name='det'
)

Determinant for every batch member.

diag_part

diag_part(
    name='diag_part'
)

Efficiently get the [batch] diagonal part of this operator.

If this operator has shape [B1,...,Bb, M, N], this returns a Tensor diagonal, of shape [B1,...,Bb, min(M, N)], where diagonal[b1,...,bb, i] = self.to_dense()[b1,...,bb, i, i].

my_operator = LinearOperatorDiag([1., 2.])

# Efficiently get the diagonal
my_operator.diag_part()
==> [1., 2.]

# Equivalent, but inefficient method
tf.linalg.diag_part(my_operator.to_dense())
==> [1., 2.]

domain_dimension_tensor

domain_dimension_tensor(
    name='domain_dimension_tensor'
)

Dimension (in the sense of vector spaces) of the domain of this operator.

Determined at runtime.

If this operator acts like the batch matrix A with A.shape = [B1,...,Bb, M, N], then this returns N.

eigvals

eigvals(
    name='eigvals'
)

Returns the eigenvalues of this linear operator.

If the operator is marked as self-adjoint (via is_self_adjoint) this computation can be more efficient.

inverse

inverse(
    name: str = 'inverse'
) -> 'LinearOperator'

Returns the Inverse of this LinearOperator.

Given A representing this LinearOperator, return a LinearOperator representing A^-1.

log_abs_determinant

log_abs_determinant(
    name='log_abs_det'
)

Log absolute value of determinant for every batch member.

matmul

matmul(
    x, adjoint=False, adjoint_arg=False, name='matmul'
)

Transform [batch] matrix x with left multiplication: x --> Ax.

# Make an operator acting like batch matrix A.  Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]

X = ... # shape [..., N, R], batch matrix, R > 0.

Y = operator.matmul(X)
Y.shape
==> [..., M, R]

Y[..., :, r] = sum_j A[..., :, j] X[j, r]

matvec

matvec(
    x, adjoint=False, name='matvec'
)

Transform [batch] vector x with left multiplication: x --> Ax.

# Make an operator acting like batch matrix A.  Assume A.shape = [..., M, N]
operator = LinearOperator(...)

X = ... # shape [..., N], batch vector

Y = operator.matvec(X)
Y.shape
==> [..., M]

Y[..., :] = sum_j A[..., :, j] X[..., j]

range_dimension_tensor

range_dimension_tensor(
    name='range_dimension_tensor'
)

Dimension (in the sense of vector spaces) of the range of this operator.

Determined at runtime.

If this operator acts like the batch matrix A with A.shape = [B1,...,Bb, M, N], then this returns M.

row

View source

row(
    index
)

Gets a row from the dense operator.

shape_tensor

shape_tensor(
    name='shape_tensor'
)

Shape of this LinearOperator, determined at runtime.

If this operator acts like the batch matrix A with A.shape = [B1,...,Bb, M, N], then this returns a Tensor holding [B1,...,Bb, M, N], equivalent to tf.shape(A).

solve

solve(
    rhs, adjoint=False, adjoint_arg=False, name='solve'
)

Solve (exact or approx) R (batch) systems of equations: A X = rhs.

The returned Tensor will be close to an exact solution if A is well conditioned. Otherwise closeness will vary. See class docstring for details.

Examples:

# Make an operator acting like batch matrix A.  Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]

# Solve R > 0 linear systems for every member of the batch.
RHS = ... # shape [..., M, R]

X = operator.solve(RHS)
# X[..., :, r] is the solution to the r'th linear system
# sum_j A[..., :, j] X[..., j, r] = RHS[..., :, r]

operator.matmul(X)
==> RHS

solvevec

solvevec(
    rhs, adjoint=False, name='solve'
)

Solve single equation with best effort: A X = rhs.

The returned Tensor will be close to an exact solution if A is well conditioned. Otherwise closeness will vary. See class docstring for details.

Examples:

# Make an operator acting like batch matrix A.  Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]

# Solve one linear system for every member of the batch.
RHS = ... # shape [..., M]

X = operator.solvevec(RHS)
# X is the solution to the linear system
# sum_j A[..., :, j] X[..., j] = RHS[..., :]

operator.matvec(X)
==> RHS

tensor_rank_tensor

tensor_rank_tensor(
    name='tensor_rank_tensor'
)

Rank (in the sense of tensors) of matrix corresponding to this operator.

If this operator acts like the batch matrix A with A.shape = [B1,...,Bb, M, N], then this returns b + 2.

to_dense

to_dense(
    name='to_dense'
)

Return a dense (batch) matrix representing this operator.

trace

trace(
    name='trace'
)

Trace of the linear operator, equal to sum of self.diag_part().

If the operator is square, this is also the sum of the eigenvalues.

with_name_scope

@classmethod
with_name_scope(
    method
)

Decorator to automatically enter the module name scope.

class MyModule(tf.Module):
  @tf.Module.with_name_scope
  def __call__(self, x):
    if not hasattr(self, 'w'):
      self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
    return tf.matmul(x, self.w)

Using the above module would produce tf.Variables and tf.Tensors whose names included the module name:

mod = MyModule()
mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

__getitem__

__getitem__(
    slices
)

__matmul__

__matmul__(
    other
)