particles_mpicomm_blocks_partition
Module
Description
$Id: particles_mpicomm_blocks.f90 17983 2011-12-11 02:04:17Z Bourdin.KIS $
This module deals with communication of particles between processors.
This version is for block domain decomposition of particles. See particles_map_blocks.f90 for documentation.
Quick access
- Variables:
communicate_fpbuf,cumulate_counts,dfb,dist_particles_evenly_procs,dvol1xb,dvol1xbrick,dvol1yb,dvol1ybrick,dvol1zb,dvol1zbrick,dx1b,dx1brick,dy1b,dy1brick,dz1b,dz1brick,fb,get_brick_index,ibrick_parent_block,inearblock,input_blocks_dist,input_blocks_mpi,iproc_foster_brick,iproc_foster_brick_part,iproc_foster_list,iproc_parent_block,iproc_parent_list,it1_loadbalance,k1_iblock,k2_iblock,l1b,l2b,ladopt_own_light_bricks,lbrick_partition,lfill_blocks_density,lfill_blocks_gpotself,lfill_blocks_velocity,lfill_bricks_velocity,lreblock_particles_run,m1b,m2b,migrate_particles_block_to_proc,migrate_particles_proc_to_block,migrate_particles_proc_to_proc,mxb,myb,mzb,n1b,n2b,nblock_loc,nbrick_foster,nbricks,nbx,nby,nbz,nghostb,npar_iblock,npart_brick_max,nproc_foster,nproc_parent,nxb,nyb,nzb,output_blocks_dist,output_blocks_mpi,report_missing_particles,xb,xbrick,xref_par,yb,ybrick,yref_par,zb,zbrick,zref_par- Routines:
communicate_fpbuf(),get_brick_index(),initialize_particles_mpicomm(),input_blocks(),load_balance_particles(),migrate_particles(),output_blocks(),sort_blocks()
Needed modules
cdata: $Id$general(keep_compiler_quiet()): $Id$messages: $Id$mpicomm: $Id$particles_cdata: $Id$
Variables
- particles_mpicomm/communicate_fpbuf [public]
- particles_mpicomm/dist_particles_evenly_procs [private]
- particles_mpicomm/find_index_by_bisection [private]
- particles_mpicomm/get_brick_index [public]
- particles_mpicomm/ibrick_parent_block (nblockmax) [integer,public]
- particles_mpicomm/inearblock (mpar_loc) [integer,public]
- particles_mpicomm/iproc_foster_brick_part (*) [integer,private/allocatable]
- particles_mpicomm/iproc_parent_block (nblockmax) [integer,public]
- particles_mpicomm/it1_loadbalance [integer,public/optional/default=100]
- particles_mpicomm/k1_iblock (nblockmax) [integer,public]
- particles_mpicomm/k2_iblock (nblockmax) [integer,public]
- particles_mpicomm/l1b [integer,parameter=2]
- particles_mpicomm/ladopt_own_light_bricks [public]
- particles_mpicomm/lbrick_partition [logical,public/optional/default=.false.]
- particles_mpicomm/lfill_blocks_density [logical,public/optional/default=.false.]
- particles_mpicomm/lfill_blocks_gpotself [logical,public/optional/default=.false.]
- particles_mpicomm/lfill_blocks_velocity [logical,public/optional/default=.false.]
- particles_mpicomm/lfill_bricks_velocity [logical,public/optional/default=.false.]
- particles_mpicomm/lreblock_particles_run [logical,public/optional/default=.false.]
- particles_mpicomm/m1b [integer,parameter=2]
- particles_mpicomm/migrate_particles_block_to_proc [private]
- particles_mpicomm/migrate_particles_proc_to_block [private]
- particles_mpicomm/migrate_particles_proc_to_proc [private]
- particles_mpicomm/n1b [integer,parameter=2]
- particles_mpicomm/nblock_loc [integer,public]
- particles_mpicomm/nbrick_foster [integer,public]
- particles_mpicomm/nghostb [integer,parameter=1]
- particles_mpicomm/npar_iblock (nblockmax) [integer,public]
- particles_mpicomm/npart_brick_max [integer,private/optional/default=1000]
- particles_mpicomm/nproc_foster [integer,public]
- particles_mpicomm/nproc_parent [integer,public]
- particles_mpicomm/report_missing_particles [private]
- particles_mpicomm/xref_par [real,private/optional/default=0.0]
- particles_mpicomm/yref_par [real,private/optional/default=0.0]
- particles_mpicomm/zref_par [real,private/optional/default=0.0]
Subroutines and functions
- subroutine particles_mpicomm/initialize_particles_mpicomm(f)
Perform any post-parameter-read initialization i.e. calculate derived parameters.
31-oct-09/anders: coded
- Parameters:
- Call to:
fatal_error_local(),fatal_error_local_collect(),fatal_error(),input_blocks(),warning(),sort_blocks()
- subroutine particles_mpicomm/migrate_particles(fp, ipar[, dfp[, linsert]])
Migrate particles between processors.
Migration is divided into three steps for block domain decomposition:
Particles that are no longer in any blocks adopted by the considered processor are migrated to their parent processor.
The parent processor either keeps the particle, if it is really the parent, or migrates the particle to the actual parent. The actual parent may differ from the registred parent if the particle has moved away from the latter.
The parent processor either keeps the particle, if the particle is in one of its blocks, or migrates the particle to the foster parent.
The three-step division is necessary to have directed communication between processors, because it is impractical at high processor numbers to allow all processors to exchange particles.
31-oct-09/anders: coded
- Parameters:
fp (mpar_loc,mpvar) [real]
ipar (mpar_loc) [integer]
dfp (mpar_loc,mpvar) [real]
linsert [logical]
- Call to:
fatal_error(),fatal_error_local(),fatal_error_local_collect(),warning(),sort_blocks()
- subroutine particles_mpicomm/load_balance_particles(f, fp, ipar)
This subroutine counts particles in the bricks at the local processor and distributes the bricks in such a away that there is approximately equal number of particles per processor.
- Alternative method:
Integrate np on processors (should be possible to do MPI integral)
Each processor finds out if it has the boundary between two procs
Use MPI_ANY_SOURCE to communicate boundary to correct processor
To avoid stacking light bricks at a single processor, this can be done first for heavy bricks and then for light bricks.
12-oct-09/anders: coded
- Parameters:
- Call to:
fatal_error_local(),fatal_error_local_collect(),fatal_error(),sort_blocks()
- subroutine particles_mpicomm/output_blocks(filename)
Write block domain decomposition to file.
04-nov-09/anders: coded
- Parameters:
filename [character,in]
- Call to:
- subroutine particles_mpicomm/input_blocks(filename)
Read block domain decomposition from file.
04-nov-09/anders: coded
- Parameters:
filename [character,in]
- Call to:
- subroutine particles_mpicomm/sort_blocks()
Sort the blocks by parent processor and by parent brick index.
04-nov-09/anders: coded
- Call to: