Parallel computing using SpinDynamica 2.6.0

[JyrkiRantaharju]

Hi all,

I think the problem in the 2.6.1 is as Andreas's described

The reason is that DistributeDefinitions and ParallelNeeds will automatically be evaluated on a newly launched subkernel

In my previous messages I proposed a workaround to this problem.

 ParalleEvaluate@Needs will not evaluate automatically so changing ParallelNeeds to  ParalleEvaluate@Needs  will prevent SpinDynamica to be loaded automatically to the newly launched sub-kernels,. After this change the new sub-kernels started to complain that the SpinSystem and Basis are not set so I decided to distribute them in the PrepareParallel routine.

DistributedContexts->All all option of the individual  Parallelize[]  routines will probably make SpinDynamica distribute stuff automatically to the new sub-kernels. The DistributedContexts->All option will cause error message

Code: [Select]

EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} in the subsequent calculations. I decided to try distribution code like

Code: [Select]

stufftodistribute = {tmin, tmax, \[Delta]t, events, evopts,
   varsubrules, opbasis, basis,  opts};
DistributeDefinitions[stufftodistribute]; instead of the  DistributedContexts->All option. This fixed the problem. There probably is a solution to this problem which does not include complex changes to the individual routines. To find this solution we need to investigate better what causes the 

Code: [Select]

EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages.

Jyrki

[JyrkiRantaharju]

Hi all,

DistributedContexts->"Global`" would only distribute definitions on context Global`. Using this in the Parallelize instead of DistributedContexts->All would be another fix to the problem which causes the

Code: [Select]

EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages.
I think this would be the only change needed in the individual routines. Other things that need to be distributed could be distributed in PrepareParallel routine.


Jyrki

[JyrkiRantaharju]

Hi all,

It seems that the DistributedContexts->"Global`" option does not distribute anything to the subkernles.  The definitions defined in the EnsembleAverage modules are in the "NMR`SpinDynamics`Private`" context.
Also the definitions that cause the

Code: [Select]

EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages, in subsequent calculations, seem to be in the "NMR`SpinDynamics`Private`" context.

 However it seems that the newly  launched sub kernels print the above message right after ParallelEvaluate[Needs["SpinDynamica`"]] is evaluated in the PrepareParallel[] routine and all the kernels work normally after the messages.

[JyrkiRantaharju]

Hi Malcolm,

I tested SDv2.6.2b2 with  trajectory simulations similar to what I usually perform and it seems to work fine.

It seems that my previous message to this thread is false.  DistributedContexts->"Global`" does distribute stuff to sub-kernels.

Jyrki.

[MalcolmHLevitt]

OK good, thanks Jyrki. Does anyone else have any problem to report with parallelization in v2.6.2b2?

[MalcolmHLevitt]

v2.6.2b3 contains Jyrkis recoding of parallel and also addresses the slow execution of Trajectory reported in the other thread. It is available here:

https://www.dropbox.com/sh/x26heihcib5mbqu/1MN3HBANk5

[Andreas Brinkmann]

Dear all,

I tested SDv2.6.2b2 and it works great for me (I am using mainly Signal1D). Thanks Jyrki!

Andreas

[Andreas Brinkmann]

One thing I noticed though:

If I use Signal1D[..., MaxSteps->xxx] the option is not passed on to NDSolve[]. I am using SignalCalculationMethod -> "COMPUTE" in this example.

Andreas

[MalcolmHLevitt]

Hi Andreas,
 is this problem specific to Parallel->True or does it also arise for Parallel->False?
malcolm

[Andreas Brinkmann]

Hi Malcolm,

this problem occurs both for Parallel->True and Parallel->False. If I specify a Preparation sequence, it does not occur for the calculation of that one, but only for the Compute part. I notice that in the code of Compute the call to NPropagationOperator[] does not pass any additional options? Should I better open a new thread since this is more Compute related than a parallel problem?

Andreas

[JyrkiRantaharju]

Hi Andreas,

it would be great if you have time to test SDv2.6.2b4 which is available here

https://www.dropbox.com/sh/ptfd6fcwedr3lcv/EjpwCr1yJU

I made small modifications to Signal1D of it in-order to fix the memory overshoot issue.

Jyrki.

[MalcolmHLevitt]

Hi all,
 I hope that these problems have been addressed in the new release SDv2.7.1. I have incorporated most of Jyrki's and your suggestions.

 So far, the implementation of Parallel has worked fine in the tests I have done.

Good luck
malcolm

[MalcolmHLevitt]

As far as I know, all of these concerns have been fixed in SDv2.8.1. Please report any further problems!
malcolm