Exporting data from Spinach to other softwares

[tmduque]

Dear Spinach team,

I was simulating some spectra and I would like to compare them to other results I got experimentally. Is there any function that exports spectra to MestreNova or TopSpin? If not, what's the easiest way to compare both experimental and simulated spectra?

Thank you very much in advance

[kuprov]

This is something we specifically stay out of: import and export. Trouble is, the number of formats (and bugs in those formats) on either side is infinite and we simply do not have the manpower.

You do get a Matlab vector or a matrix at the end of every simulation, and Matlab itself can save in a variety of formats, from ASCII text to complicated binary data types to XML. So my suggestion would be to find out what your format expects and make an export script in Matlab that writes that format.

[ptg20]

Hi folks,

I have done this, in a way that works, but needs some topspin fiddling, for both 1D and for some 2D data.
This is a long post but a large chunk is the functions themselves. Once you have created a suitable topspin dataset for a given case you can just copy it and you only need to modify parameters again if something is changed in the simulation,
I think this is a workable workflow overall. Copy the functions into files fid2bruker.m / fid2bruker2d.m in a suitable place in the spinach tree (eg I made a topspin folder in interfaces folder).

Actual experimental raw data files in topspin are either 32 bit integers (historically), 32 bit floats (theoretically, but I think never in practice), or 64 bit doubles (for AV Neo spectrometers, and some 1D data from older spectrometers).
Which of these is the case, is indicated by the status parameter dtypa which can be int, float or double (== 0, 1 or 2 in the actual parameter file). When writing files we can just use doubles, and this will be readable by recent topspin versions (4, and later versions of 3 I guess).


For 1D you can write a bruker "fid" file, with the following function -basically just writing out the real and imaginary parts of the raw data interleaved into a single file, as doubles. For convenience here the first point is multiplied by 0.5, as topspin isn't keen on doing that in the direct dimension (in principle it's controlled by the FCOR parameter in topspin, but this actually only does anything in the indirect dimension). Without this first point correction the processed data will have a dc offset.

This function creates a file fid in the working directory that you can copy to a suitable topspin dataset (some comments at the end...)

function fid2bruker(fid)

% Check consistency
grumble(fid)

% Decide data dimensions
if isvector(fid)

% Open the file for writing
file_id=fopen('fid','w');
% write real and imaginary interleaved
% write first point out multiplied by 0.5 because topspin doesn't want
% to do that itself
fwrite(file_id,0.5*real(fid (1)),'double');
fwrite(file_id,0.5*imag(fid (1)),'double');
%then all the other points
for n=1:numel(fid)
fwrite(file_id,real(fid (n)),'double');
fwrite(file_id,imag(fid (n)),'double');
end

else

% Complain and bomb out
error('unsupported data dimensionality.');

% Close the file
fclose(file_id);

end

% Consistency enforcement
function grumble(fid)
if ~isnumeric(fid), error('fid must be numeric.'); end
end

You can call the function for example just after the fid is created in the spinach file, eg you can have:

% Simulation
fid=liquid(spin_system,@acquire,parameters,'nmr');

fid2bruker(fid);

For 2D we just need to loop over the other dimension as well; first point correction in the indirect dimension is handled OK by topspin so just the direct dimension is corrected here. We write for example a cosine modulated fid followed by a sin modulated fid (or a echo followed by an antiecho fid for such data)- how to deal with the FT in the indirect dimension is handled by settings in the topspin dataset. The below function writes a serial file:

function fid2bruker2d(posfid, negfid)

% Check consistency
grumble(posfid)

% Open the file for writing
file_id=fopen('ser','w');
% posfid=fid.pos;
% negfid=fid.neg;
% Write out the fid
for k=1:size(posfid,2)

% Interleave real and imaginary parts in one fid
% write first poitn, scaled by 0.5
fwrite(file_id,(0.5*real(posfid (1,k))),'double');
fwrite(file_id,(0.5*imag(posfid (1,k))),'double');
for n=2:size(posfid,1)
fwrite(file_id,real(posfid (n,k)),'double');
fwrite(file_id,imag(posfid (n,k)),'double');
end
% then write out the other fid
fwrite(file_id,(0.5*real(negfid (1,k))),'double');
fwrite(file_id,(0.5*imag(negfid (1,k))),'double');
for n=2:size(negfid,1)
fwrite(file_id,real(negfid (n,k)),'double');
fwrite(file_id,imag(negfid (n,k)),'double');
end

end



% Close the file
fclose(file_id);

end

% Consistency enforcement
function grumble(fid)
if ~isnumeric(fid), error('fid must be numeric.'); end
end

For 2D in your main spinach file you need to convert the data struct fid which contains fid.pos / fid.neg (for echo/antiecho data) or fid.cos/fid.sin (for hypercomplex), into single data arrays to pass to fid2bruker. So in a NOESY for example you could have:

% Simulation
fid=liquid(spin_system,@noesy,parameters,'nmr');

cosfid=fid.cos;
sinfid=fid.sin;
fid2bruker2d(cosfid, sinfid);

or in an HSQC:

% Simulation
fid=liquid(spin_system,@hsqcZ,parameters,'nmr');

posfid=fid.pos;
negfid=fid.neg;

fid2bruker2d(posfid, negfid);

fid.cos etc are the structures containing the final raw data, created by the pulse program part of spinach, so eg in NOESY they are created from the 4 fids by:

% Axial peak elimination
fid.cos=fids{1}-fids{3}; fid.sin=fids{2}-fids{4};

This however is only the start - you have to manually set some parameters in the topspin dataset for everything to work. Start from a basically similar dataset, and copy the parameters eg with iexpno, put the fid or ser in the new expno directory. Then you can open the dataset in topspin.

In order for everything to work you will need to set at least the following parameters - type the name to set, and either select from dropdown list or enter values; many are status parameters so where indicated you do need the "s <parameter>" rather than just "<parameter>"

pknl : false (disables the digital filter correction)

s dtypa : double

s td : raw data size should be set to twice the numbers numbers of points set in the Spinach file as the Bruker numbers are real not complex indices

s digtyp : may need to be set to whatever the current topspin version is happy with, ie DRX for TS4 or DRU for TS3

s swh : The spectral widths in Hz should be set to what was used in the spinach calculation.

s o1 : Likewise offsets

for 2D only:

s fnmode : This should be set according to how the data was "acquired"; Typically the examples are States for homonuclear experiments, Echo-Antiecho for HSQC etc. You can play with this until xfb gives the correct result!

1 fcor : (ie fcor in the F1 dimension): usually 0.5, assuming the first datapoint in the indirect dimension corresponds to t1=0.

1 reverse : May need to be true ; check whether spectrum is the right way up after processing

and all the usual processing parameters, window functions etc as desired. Phase constants in F1 should usually be zero.

Then you can process as normal with efp/xfb or whatever. At this point the data should also be readable by mestrenova.

In spinach the "destreak" function is usually used before plotting - I guess this is a consequence of lack of first point correction of the data. See for example:
http://dx.doi.org/10.1002/cmr.a.21250
for some discussion of this. If all is set correctly no baseline correction should be needed.

Hopefully this helps someone - I will see responses here so can answer questions

Pete

[kuprov]

Thanks, Pete - we can probably start institutionalising these things after I settle down at Weizmann after Christmas: undergraduate reaching would not be getting in the way.

[tmduque]

Thank you very much Pete and Prof. Kuprov. I'll take a look into it as soon as I can!