Author Topic: Parallel computing using SpinDynamica 2.6.0  (Read 11418 times)

JyrkiRantaharju

  • Member
  • *
  • Posts: 36
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #15 on: May 15, 2013, 10:39:24 AM »
Hi all,

I think the problem in the 2.6.1 is as Andreas's described
Quote
The reason is that DistributeDefinitions and ParallelNeeds will automatically be evaluated on a newly launched subkernel
In my previous messages I proposed a workaround to this problem.

 ParalleEvaluate@Needs will not evaluate automatically so changing ParallelNeeds to  ParalleEvaluate@Needs  will prevent SpinDynamica to be loaded automatically to the newly launched sub-kernels,. After this change the new sub-kernels started to complain that the SpinSystem and Basis are not set so I decided to distribute them in the PrepareParallel routine.

DistributedContexts->All all option of the individual  Parallelize[]  routines will probably make SpinDynamica distribute stuff automatically to the new sub-kernels. The DistributedContexts->All option will cause error message
Code: [Select]
EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} in the subsequent calculations. I decided to try distribution code like
Code: [Select]
stufftodistribute = {tmin, tmax, \[Delta]t, events, evopts,
   varsubrules, opbasis, basis,  opts};
DistributeDefinitions[stufftodistribute];
instead of the  DistributedContexts->All option. This fixed the problem. There probably is a solution to this problem which does not include complex changes to the individual routines. To find this solution we need to investigate better what causes the 
Code: [Select]
EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages.

Jyrki
« Last Edit: May 15, 2013, 10:42:46 AM by Jyrki Rantaharju »

JyrkiRantaharju

  • Member
  • *
  • Posts: 36
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #16 on: May 15, 2013, 02:52:41 PM »
Hi all,

DistributedContexts->"Global`" would only distribute definitions on context Global`. Using this in the Parallelize instead of DistributedContexts->All would be another fix to the problem which causes the
Code: [Select]
EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages.
I think this would be the only change needed in the individual routines. Other things that need to be distributed could be distributed in PrepareParallel routine.


Jyrki
« Last Edit: May 15, 2013, 02:57:45 PM by Jyrki Rantaharju »

JyrkiRantaharju

  • Member
  • *
  • Posts: 36
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #17 on: May 24, 2013, 06:32:13 PM »
Hi all,

It seems that the DistributedContexts->"Global`" option does not distribute anything to the subkernles.  The definitions defined in the EnsembleAverage modules are in the "NMR`SpinDynamics`Private`" context.
Also the definitions that cause the
Code: [Select]
EventList::cantdo: cannot interpret event specifications: {events:{_Event..},evlistmore:(Except[_Rule]..),evlistopts___Rule} messages, in subsequent calculations, seem to be in the "NMR`SpinDynamics`Private`" context.

 However it seems that the newly  launched sub kernels print the above message right after ParallelEvaluate[Needs["SpinDynamica`"]] is evaluated in the PrepareParallel[] routine and all the kernels work normally after the messages.

JyrkiRantaharju

  • Member
  • *
  • Posts: 36
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #18 on: June 12, 2013, 03:29:55 PM »
Hi Malcolm,

I tested SDv2.6.2b2 with  trajectory simulations similar to what I usually perform and it seems to work fine.

It seems that my previous message to this thread is false.  DistributedContexts->"Global`" does distribute stuff to sub-kernels.

Jyrki.

MalcolmHLevitt

  • Administrator
  • Member
  • *****
  • Posts: 108
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #19 on: June 13, 2013, 02:28:33 PM »
OK good, thanks Jyrki. Does anyone else have any problem to report with parallelization in v2.6.2b2?

MalcolmHLevitt

  • Administrator
  • Member
  • *****
  • Posts: 108
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #20 on: June 13, 2013, 06:46:30 PM »
v2.6.2b3 contains Jyrkis recoding of parallel and also addresses the slow execution of Trajectory reported in the other thread. It is available here:

https://www.dropbox.com/sh/x26heihcib5mbqu/1MN3HBANk5

Andreas Brinkmann

  • Member
  • *
  • Posts: 13
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #21 on: June 13, 2013, 07:04:15 PM »
Dear all,

I tested SDv2.6.2b2 and it works great for me (I am using mainly Signal1D). Thanks Jyrki!

Andreas

Andreas Brinkmann

  • Member
  • *
  • Posts: 13
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #22 on: June 13, 2013, 09:00:02 PM »
One thing I noticed though:

If I use Signal1D[..., MaxSteps->xxx] the option is not passed on to NDSolve[]. I am using SignalCalculationMethod -> "COMPUTE" in this example.

Andreas

MalcolmHLevitt

  • Administrator
  • Member
  • *****
  • Posts: 108
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #23 on: June 14, 2013, 10:18:23 AM »
Hi Andreas,
 is this problem specific to Parallel->True or does it also arise for Parallel->False?
malcolm

Andreas Brinkmann

  • Member
  • *
  • Posts: 13
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #24 on: June 14, 2013, 03:37:26 PM »
Hi Malcolm,

this problem occurs both for Parallel->True and Parallel->False. If I specify a Preparation sequence, it does not occur for the calculation of that one, but only for the Compute part. I notice that in the code of Compute the call to NPropagationOperator[] does not pass any additional options? Should I better open a new thread since this is more Compute related than a parallel problem?

Andreas
« Last Edit: June 14, 2013, 04:28:21 PM by Andreas Brinkmann »

JyrkiRantaharju

  • Member
  • *
  • Posts: 36
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #25 on: June 14, 2013, 04:25:26 PM »
Hi Andreas,

it would be great if you have time to test SDv2.6.2b4 which is available here

https://www.dropbox.com/sh/ptfd6fcwedr3lcv/EjpwCr1yJU

I made small modifications to Signal1D of it in-order to fix the memory overshoot issue.

Jyrki.

MalcolmHLevitt

  • Administrator
  • Member
  • *****
  • Posts: 108
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #26 on: July 24, 2013, 11:11:27 PM »
Hi all,
 I hope that these problems have been addressed in the new release SDv2.7.1. I have incorporated most of Jyrki's and your suggestions.

 So far, the implementation of Parallel has worked fine in the tests I have done.

Good luck
malcolm

MalcolmHLevitt

  • Administrator
  • Member
  • *****
  • Posts: 108
    • View Profile
Re: Parallel computing using SpinDynamica 2.6.0
« Reply #27 on: September 13, 2013, 06:04:30 PM »
As far as I know, all of these concerns have been fixed in SDv2.8.1. Please report any further problems!
malcolm