Difference between revisions of "Deer lib gen.m"

From Spinach Documentation Wiki
Jump to: navigation, search
(Update function See also links and function index membership)
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
{{DISPLAYTITLE:deer_lib_gen.m}}
+
{{DISPLAYTITLE:deer_lib_gen.m}} __NOTOC__
−
Generates a library of distance distributions and corresponding simulated DEER traces using the parameters supplied, in a 4 step process:
+
Generates a library of distance distributions and corresponding DEER or RIDME traces for use in neural network training. Full details are given in our papers on the subject:
  
−
#The batch of simulated spin label distributions are generated as a randomly selected number of skew normal distributions.
+
                https://doi.org/10.1126/sciadv.aat5218
−
#:<math>p(r)=\frac{2}{\sigma \sqrt{2 \pi}}e^{- \frac{(r-r_0)^2}{2 \sigma ^2}} \int_{- \infty}^{ \alpha \left (\frac{(r-r_0)}{\sigma} \right )} e^{- \frac{t^2}{2}}dt  </math>
+
                https://doi.org/10.1073/pnas.2016917118
−
#The DEER form factor ''d''(''t'') is computed from the distributions using the kernel for DEER in the presence of exchange coupling (Equation 2 in our [https://dx.doi.org/10.1126/sciadv.aat5218 paper]).
+
                https://doi.org/10.1016/j.jmr.2022.107186
−
#:<math>\gamma(r,t)=\sqrt{\frac{\pi}{6Dt}} \left [ cos [(D+J)t]FrC \left [ \sqrt{\frac{6Dt}{\pi}} \right ] + sin [(D+J)t]FrS \left [ \sqrt{\frac{6Dt}{\pi}} \right ] \right ] </math>
 
−
#Simulated background, ''b''(''t'') and additive noise tracks ''n''(''t'') are mixed with the DEER form factor;
 
−
#:<math>s(t)=[1-\lambda+\lambda d (t)]b(t)+n(t) </math>
 
−
#*the background signal is generated as a stretched exponential function,
 
−
#:<math>b(t)=exp \left [ -(kt)^{n/3} \right ]  </math>
 
−
#*the noise track is uncorrelated, representing the instrumental noise expected during the indirect acquisition in the DEER method.
 
−
#The distance distributions and DEER traces are then scaled to the neural network activation range:
 
−
#*distributions are uniformly scaled to 0.75,
 
−
#*DEER trace are scaled and shifted to make the first and last points equal to 1 and 0 respectively.
 
−
 
 
−
For each pair in the training set the variables controlling the shape of the distance distribution; the amount of exchange coupling to include; the form of the background contribution to the signal; and the level of noise are randomly selected from the ranges provided in the [[Neural network module#Training parameters structure|parameters]] structure.
 
  
 
==Syntax==
 
==Syntax==
  
−
     deer_lib_gen(file_name,parameters)
+
     library=deer_lib_gen(file_name,parameters)
  
−
    [output arguments]=deer_lib_gen(file_name,parameters)
+
==Parameters==
−
 
+
Required fields of the parameters.* structure:
−
==Arguments==
+
−
 
+
    max_time          - DEER trace duration, seconds
−
    file_name      - name of output *.mat file, include
+
−
    a full path to specify the output
+
    max_exch          - maximum exchange coupling, fraction of
−
      location.      
+
                        the maximum frequency representable on
−
 
+
                        the current time discretisation grid
−
    parameters     - training set parameters, with fields
+
−
    described [[Neural network module#Training parameters structure|here]].
+
    max_exch          - minimum exchange coupling, fraction of
 +
                        the maximum frequency representable on
 +
                        the current time discretisation grid
 +
 +
    ntraces          - number of traces you wish to generate
 +
 +
    ndistmax          - maximum number of skewed gaussians
 +
                        in the distance distribution
 +
 +
    np_time          - number of digitisation points in the
 +
                        DEER trace
 +
 +
    np_dist          - number of digitisation points in the
 +
                        distance distribution
 +
 +
    npt_acq          - number of digitization points actually
 +
                        acquired in a sparsely sampled dataset;
 +
                        points are distributed randomly with
 +
                        uniform sampling probability
 +
 +
    noise_lvl        - maximum RMS noise level as a fraction
 +
                        of the modulation depth (min is zero)
 +
 +
    min_fwhm          - minimum FWHM for a gaussian in the dis-
 +
                        tance distribution, fraction of distance
 +
 +
    max_fwhm          - maximum FWHM for a gaussian in the dis-
 +
                        tance distribution, fraction of distance
 +
                        range
 +
 +
    max_mdep          - minimum DEER modulation depth
 +
 +
    min_mdep          - maximum DEER modulation depth
 +
 +
    expt              - background model, 'deer' or 'ridme'
 +
 +
    max_brate        - maximum background signal decay
 +
                        rate, s^-1 (DEER backgrounds only)
 +
 +
    min_brate        - minimum background signal decay
 +
                        rate, s^-1 (DEER backgrounds only)
 +
 +
    min_bdim          - minimum background dimensionality
 +
                        (DEER backgrounds only)
 +
 +
    max_bdim          - maximum background dimensionality
 +
                        (DEER backgrounds only)
 +
 +
     max_tshift        - maximum number of time discretisation
 +
                        points to shift the trace by, either
 +
                        forward or backward
  
 
==Outputs==
 
==Outputs==
−
The function output arguments are listed below, in the order expected:
+
The function returns library.* structure with the following fields:
  
−
    time_grid     - the time axis for the DEER traces, in
+
    time_grid         - time grid (seconds) as a row vector
−
    seconds. A row vector.
 
 
   
 
   
−
    dist_grid     - distance grid for the distributions,
+
    dist_grid         - distance grid (Angsrom) as a row vector
−
    in Angstroms. A row vetor.
 
 
   
 
   
−
    dist_distr_lib - all distance distributions, a horizontal
+
    dist_distr_lib   - all distance distributions as a horizon-
−
    array of column vectors.
+
                        tal stack of column vectors
 
   
 
   
−
    deer_ffact_lib - all DEER form factors, a horizontal
+
    background_lib    - all background signals as a horizonal
−
    array of column vectors.
+
                        stack of column vectors, shifted and
 +
                        scaled to match DEER/RIDME traces
 
   
 
   
−
    background_lib - all background signals, a horizontal
+
    deer_noisy_lib    - all complete DEER/RIDME traces as a  
−
    array of column vectors.
+
                        horizonal stack of column vectors
 
   
 
   
−
    deer_trace_lib - all primary DEER traces, a horizontal
+
    deer_clean_lib    - DEER traces as they would come out, but
−
    array of column vectors.
+
                        without the noise; horizonal stack of  
−
         
+
                        column vectors
−
    noise_line_lib - all noise tracks, a horizontal array
 
−
                      of column vectors.
 
 
   
 
   
−
     exchange_lib  - exchange coupling scalar used to in
+
    exchange_lib     - exchange interaction (MHz), a row vector
−
    generating each trace, a row vector.
+
                        containing the value for each example
 
   
 
   
−
    parameters     - the parameters structure, unchanged.
+
    parameters       - parameters array as received
−
 
 
−
The function may be called with a specified file name (including a full path), in which case the output arguments are also saved in a .mat file at that location. If the file name input is left empty then the database is not saved.           
 
  
−
        file_name=[];
+
If a file name is provided, these variables are written into that file.
  
 
==Examples==
 
==Examples==
−
The example below first loads the netset parameters from the ensemble optimised for all peak widths, and then generates a library of 1000 trace/distribution pairs. The example may be run from the '''examples/deernet/''' directory.
+
The example below generates a library of 1000 DEER traces.
  
−
% Load the training set parameters
+
        % Load default parameters
−
run('net_set_any_peaks/netset_params.m');  
+
        parameters=library_dd;
 +
 +
        % Number of traces
 +
        parameters.ntraces=1000;
 
   
 
   
−
% Specify number of traces to produce
+
        % Time and distance point counts
−
parameters.ntraces=1000;
+
        parameters.np_time=512;
 +
        parameters.np_dist=512;
 
   
 
   
−
% Set the training database name
+
        % PDS experiment type
−
file_name='dlg_example_set.mat';
+
        parameters.expt='deer';
 
   
 
   
−
% Generate the training library
+
        % Generate the training library
−
[time_grid,dist_grid,dist_distr_lib,...
+
        library=deer_lib_gen([],parameters);
−
  deer_ffact_lib,background_lib,deer_trace_lib,...
 
−
  noise_line_lib,exchange_lib,parameters]=deer_lib_gen(file_name,parameters);
 
  
−
This example will add the output libraries to the MATLAB workspace, as well as saving them in the working directory as '''"dlg_example_set.mat"'''
+
One of the resulting DEER traces is shown below.
  
 
<gallery widths=400 heights=300>
 
<gallery widths=400 heights=300>
−
File:dlg_distr_example.png|frame|Example distance distribution.
+
File:dlg_distr_example.png|frame|Distance distribution.
−
File:dlg_trace_example.png|frame|DEER trace and components for example distribution.
+
File:dlg_trace_example.png|frame|DEER trace and its components.
 
</gallery>
 
</gallery>
  
 
==Notes==
 
==Notes==
−
*As the dipolar modulation frequency is a cubic function of the inter-spin distance, a scaling relationship exists betweent the distance range and the duration of the DEER signal.
+
Multiple caveats exist in the training process. Please read our papers carefully before training your own networks.
−
 
 
−
:<math>\frac{t_A}{r_A^3}=\frac{t_B}{r_B^3} </math>
 
−
 
 
−
*An important factor when generating data for training is the dynamic range - the ratio between the longest and shortest distances represented in the training set.
 
−
 
 
−
*The training set DEER traces should be sufficiently discretised ([[Neural network module#Training parameters structure|parameters.npoints]]) to reproduce all frequencies present.  
 
  
 
==See also==
 
==See also==
−
[[netset_curate.m]], [[train_one_net.m]]
+
[[deernet.m]], [[dist_range.m]], [[elexsys2deernet.m]], [[process_using.m]], [[train_one_net.m]], [[Neural network module]], [[Built-in_experiments]]
−
 
 
  
−
''Version 2.2, authors: [[Ilya Kuprov]], [[Steve Worswick]], [[Gunnar Jeschke]]''
+
''Version 2.8, authors: [[Ilya Kuprov]], [[Steve Worswick]], [[Jake Keeley]], [[Gunnar Jeschke]]''

Latest revision as of 19:35, 6 June 2026

Generates a library of distance distributions and corresponding DEER or RIDME traces for use in neural network training. Full details are given in our papers on the subject: 

               https://doi.org/10.1126/sciadv.aat5218
               https://doi.org/10.1073/pnas.2016917118
               https://doi.org/10.1016/j.jmr.2022.107186

Syntax

    library=deer_lib_gen(file_name,parameters)

Parameters

Required fields of the parameters.* structure: 

   max_time          - DEER trace duration, seconds

   max_exch          - maximum exchange coupling, fraction of
                       the maximum frequency representable on
                       the current time discretisation grid

   max_exch          - minimum exchange coupling, fraction of
                       the maximum frequency representable on
                       the current time discretisation grid

   ntraces           - number of traces you wish to generate

   ndistmax          - maximum number of skewed gaussians 
                       in the distance distribution

   np_time           - number of digitisation points in the
                       DEER trace

   np_dist           - number of digitisation points in the
                       distance distribution

   npt_acq           - number of digitization points actually
                       acquired in a sparsely sampled dataset;
                       points are distributed randomly with
                       uniform sampling probability

   noise_lvl         - maximum RMS noise level as a fraction
                       of the modulation depth (min is zero)

   min_fwhm          - minimum FWHM for a gaussian in the dis-
                       tance distribution, fraction of distance

   max_fwhm          - maximum FWHM for a gaussian in the dis-
                       tance distribution, fraction of distance
                       range

   max_mdep          - minimum DEER modulation depth

   min_mdep          - maximum DEER modulation depth

   expt              - background model, 'deer' or 'ridme'

   max_brate         - maximum background signal decay
                       rate, s^-1 (DEER backgrounds only)

   min_brate         - minimum background signal decay 
                       rate, s^-1 (DEER backgrounds only)

   min_bdim          - minimum background dimensionality
                       (DEER backgrounds only)

   max_bdim          - maximum background dimensionality
                       (DEER backgrounds only)

   max_tshift        - maximum number of time discretisation
                       points to shift the trace by, either
                       forward or backward

Outputs

The function returns library.* structure with the following fields: 

   time_grid         - time grid (seconds) as a row vector

   dist_grid         - distance grid (Angsrom) as a row vector

   dist_distr_lib    - all distance distributions as a horizon-
                       tal stack of column vectors

   background_lib    - all background signals as a horizonal 
                       stack of column vectors, shifted and 
                       scaled to match DEER/RIDME traces

   deer_noisy_lib    - all complete DEER/RIDME traces as a 
                       horizonal stack of column vectors

   deer_clean_lib    - DEER traces as they would come out, but
                       without the noise; horizonal stack of 
                       column vectors

   exchange_lib      - exchange interaction (MHz), a row vector
                       containing the value for each example

   parameters        - parameters array as received

If a file name is provided, these variables are written into that file.

Examples

The example below generates a library of 1000 DEER traces. 

       % Load default parameters
       parameters=library_dd; 

       % Number of traces
       parameters.ntraces=1000;

       % Time and distance point counts
       parameters.np_time=512;
       parameters.np_dist=512;

       % PDS experiment type
       parameters.expt='deer';

       % Generate the training library
       library=deer_lib_gen([],parameters);

One of the resulting DEER traces is shown below.

  • Distance distribution.

  • DEER trace and its components.

Notes

Multiple caveats exist in the training process. Please read our papers carefully before training your own networks.

See also

deernet.m, dist_range.m, elexsys2deernet.m, process_using.m, train_one_net.m, Neural network module, Built-in_experiments

Version 2.8, authors: Ilya Kuprov, Steve Worswick, Jake Keeley, Gunnar Jeschke

Retrieved from "https://spindynamics.org/wiki/index.php?title=Deer_lib_gen.m&oldid=9075"