descramble.m

Generates a weight matrix descrambler for a particular layer in a neural network using Tikhonov smoothness criterion. The particulars are described in https://www.pnas.org/doi/10.1073/pnas.2016917118

Syntax

    P=descramble(S,n_iter,guess)

Parameters

   S       - a matrix containing, in its columns, the outputs
             of the preceding layers of the neural network for
             a (preferably large) number of reasonable inputs

   n_iter  - maximum number of Newton-Raphson interations, 400
             is generally sufficient

   guess   - [optional] the initial guess for the descrambling
             transform generator (lower triangle is used), a
             reasonable choice is a zero matrix (default)

Outputs

   P       - descrambling matrix. In the case when the network 
             is wiretapped before the activation function, i.e.

                            S = Wf(W...f(Wf(WX)))

             matrix P descrambles the output dimension of the
             left-most W. In the case when the network is wire-
             tapped after the activation function, i.e.

                           S = f(Wf(W...f(Wf(WX))))

             matrix inv(P) descrambles the input dimension of
             the weight matrix of the subsequent layer.

See also

left_diag.m, Neural network module, Built-in_experiments

Version 2.8, authors: Jake Amey, Ilya Kuprov, Tajwar Choudhury