Domain decomposition for parallel computing. More...

#include "utils.h"
#include <limits.h>
#include "communication.h"
#include "errorhandling.h"

Include dependency graph for grid.h:

Defines
#define	PERIODIC(coord) (periodic & (1L << coord))
	Macro that tests for a coordinate being periodic or not.
Functions
Exported Functions
void	init_node_grid ()
	Make sure that the node grid is set, eventually determine one automatically.
int	node_grid_is_set ()
	return wether node grid was set.
MDINLINE void	map_node_array (int node, int pos[3])
	node mapping: array -> node.
MDINLINE int	map_array_node (int pos[3])
	node mapping: node -> array.
int	map_position_node_array (double pos[3])
	map a spatial position to the node grid
void	calc_node_neighbors (int node)
	fill neighbor lists of node.
void	grid_changed_n_nodes ()
	called from mpi_bcast_parameter .
void	grid_changed_box_l ()
	called from mpi_bcast_parameter .
void	calc_minimal_box_dimensions ()
	Calculates the smallest box and local box dimensions for periodic directions.
void	calc_2d_grid (int n, int grid[3])
	calculate most square 2d grid.
int	map_3don2d_grid (int g3d[3], int g2d[3], int mult[3])
	calculate 'best' mapping between a 2d and 3d grid.
void	rescale_boxl (int dir, double d_new)
	rescales the box in dimension 'dir' to the new value 'd_new', and rescales the particles accordingly
MDINLINE void	get_mi_vector (double res[3], double a[3], double b[3])
	get the minimal distance vector of two vectors in the current bc.
MDINLINE void	fold_coordinate (double pos[3], int image_box[3], int dir)
	fold a coordinate to primary simulation box.
MDINLINE void	fold_position (double pos[3], int image_box[3])
	fold particle coordinates to primary simulation box.
MDINLINE void	unfold_position (double pos[3], int image_box[3])
	unfold coordinates to physical position.
Variables
Exported Variables
int	node_grid [3]
	The number of nodes in each spatial dimension.
int	node_pos [3]
	position of node in node grid
int	node_neighbors [6]
	the six nearest neighbors of a node in the node grid.
int	boundary [6]
	where to fold particles that leave local box in direction i.
int	periodic
	Flags for all three dimensions wether pbc are applied (default).
double	box_l [3]
	Simulation box dimensions.
double	box_l_i [3]
	1 / box dimensions.
double	min_box_l
	Smallest simulation box dimension (box_l).
double	local_box_l [3]
	Dimensions of the box a single node is responsible for.
double	min_local_box_l
	Smallest local simulation box dimension (local_box_l).
double	my_left [3]
	Left (bottom, front) corner of this nodes local box.
double	my_right [3]
	Right (top, back) corner of this nodes local box.

Detailed Description

Domain decomposition for parallel computing.

The primary simulation box is divided into orthogonal rectangular subboxes which are assigned to the different nodes (or processes or threads if you want). This grid is described in node_grid. Each node has a number this_node and a position node_pos in that grid. Each node has also 6 nearest neighbors node_neighbors which are necessary for the communication between the nodes (see also ghosts.c and p3m.c for more details about the communication.

For the 6 directions we have the following convention:

Convention for the order of the directions

The Figure illustrates the direction convetion used for arrays with 6 (e.g. node_neighbors, boundary) and 3 entries (e.g node_grid, box_l , my_left,...).

For more information on the domain decomposition, see grid.c.

Definition in file grid.h.

Define Documentation

#define PERIODIC ( coord ) (periodic & (1L << coord))

Macro that tests for a coordinate being periodic or not.

Definition at line 52 of file grid.h.

Referenced by add_ljangle_pair_force(), analyze_fold_molecules(), check_particle_consistency(), check_particles(), comfixed_set_params(), dd_append_particles(), dd_exchange_and_sort_particles(), dd_position_to_cell(), dd_prepare_comm(), dd_save_position_to_cell(), dd_update_communicators_w_boxl(), dp3m_sanity_checks(), ELC_sanity_checks(), fold_coordinate(), get_mi_vector(), layered_get_mi_vector(), layered_topology_init(), lb_check_halo_regions(), ljangle_pair_energy(), maggs_sanity_checks(), magnetic_dipolar_direct_sum_calculations(), mdlc_sanity_checks(), MMM1D_sanity_checks(), MMM2D_sanity_checks(), p3m_sanity_checks(), prepare_halo_communication(), and update_mol_pos_particle().

Function Documentation

void calc_2d_grid	(	int	n,
		int	grid[3]
	)

calculate most square 2d grid.

Definition at line 186 of file grid.c.

References i.

Referenced by dfft_init(), and fft_init().

void calc_minimal_box_dimensions ( )

Calculates the smallest box and local box dimensions for periodic directions.

This is needed to check if the interaction ranges are compatible with the box dimensions and the node grid. see also box_l, local_box_l, min_box_l and min_local_box_l. Remark: In the apreiodic case min_box_l is set to 2 * MAX_INTERACTION_RANGE .

Definition at line 175 of file grid.c.

References box_l, dmin(), i, local_box_l, MAX_INTERACTION_RANGE, min_box_l, and min_local_box_l.

Referenced by grid_changed_box_l().

void calc_node_neighbors ( int node )

fill neighbor lists of node.

Calculates the numbers of the 6 nearest neighbors for a node and stors them in node_neighbors.

Parameters:

node	number of the node.

Definition at line 96 of file grid.c.

References boundary, comm_cart, GRID_TRACE, map_node_array(), MPI_Cart_shift(), node_grid, node_neighbors, node_pos, and this_node.

Referenced by grid_changed_n_nodes().

MDINLINE void fold_coordinate	(	double	pos[3],
		int	image_box[3],
		int	dir
	)

fold a coordinate to primary simulation box.

Parameters:

pos	the position...
image_box	and the box
dir	the coordinate to fold: dir = 0,1,2 for x, y and z coordinate.

Both pos and image_box are I/O, i. e. a previously folded position will be folded correctly.

Definition at line 197 of file grid.h.

References box_l, box_l_i, ERROR_SPRINTF, ES_DOUBLE_SPACE, ES_INTEGER_SPACE, PERIODIC, ROUND_ERROR_PREC, and runtime_error().

Referenced by calc_diffusion_profile(), calc_radial_density_map(), dd_append_particles(), dd_exchange_and_sort_particles(), density_profile_av(), fold_all(), and fold_position().

MDINLINE void fold_position	(	double	pos[3],
		int	image_box[3]
	)

fold particle coordinates to primary simulation box.

Parameters:

pos	the position...
image_box	and the box

Both pos and image_box are I/O, i. e. a previously folded position will be folded correctly.

Definition at line 223 of file grid.h.

References fold_coordinate(), and i.

Referenced by add_area_global_force(), add_constraints_energy(), add_constraints_forces(), add_volume_force(), calc_angledist_param(), calc_area_force_local(), calc_area_global(), calc_bending_force(), calc_local_mol_info(), calc_pwdist(), calc_scaling(), calc_scaling2(), calc_volume(), constraint_collision(), cuda_mpi_get_particles(), cuda_mpi_get_particles_slave(), layered_append_particles(), layered_exchange_and_sort_particles(), local_place_particle(), magnetic_dipolar_direct_sum_calculations(), map_position_node_array(), map_position_to_lattice_global(), observable_density_profile(), observable_flux_density_profile(), observable_radial_density_profile(), observable_radial_flux_density_profile(), reflect_particle(), tclcommand_analyze_parse_get_folded_positions(), tclcommand_imd_parse_pos(), tclcommand_part_print_folded_position(), and tclcommand_system_CMS().

MDINLINE void get_mi_vector	(	double	res[3],
		double	a[3],
		double	b[3]
	)

get the minimal distance vector of two vectors in the current bc.

Parameters:

a	the vector to subtract from
b	the vector to subtract
res	where to store the result

Definition at line 176 of file grid.h.

References box_l, box_l_i, dround(), i, and PERIODIC.

Referenced by add_bonded_energy(), add_bonded_force(), add_bonded_virials(), add_ljangle_pair_force(), add_three_body_bonded_stress(), angle_cosine_energy(), angle_cossquare_energy(), angle_energy(), angle_harmonic_energy(), angledist_energy(), bilayer_density_profile_sphere(), calc_angle_3body_forces(), calc_angle_3body_tabulated_forces(), calc_angle_cosine_3body_forces(), calc_angle_cosine_force(), calc_angle_cossquare_3body_forces(), calc_angle_cossquare_force(), calc_angle_force(), calc_angle_harmonic_3body_forces(), calc_angle_harmonic_force(), calc_angledist_force(), calc_angledist_param(), calc_area_force_local_complicated(), calc_dihedral_angle(), calc_dipole_dipole_ia(), calc_dipole_of_molecule(), calc_endangledist_pair_force(), calc_force_between_mol(), calc_mol_pos(), calc_overlap_angle_force(), calc_pwangle(), calc_scaling(), calc_scaling2(), calc_tab_angle_force(), calc_wallbondyz(), calc_wallrdfyz(), compute_pos_corr_vec(), compute_vel_corr_vec(), distribute_mol_force(), distribute_tensors(), distto(), does_line_go_through_cube(), ELC_P3M_dielectric_layers_energy_contribution(), ELC_P3M_dielectric_layers_energy_self(), ELC_P3M_dielectric_layers_force_contribution(), ELC_P3M_self_forces(), get_mol_dist(), get_mol_dist_vector_from_molid_cfg(), get_nonbonded_interaction(), incubewithskin(), integrate_reaction(), ljangle_pair_energy(), local_stress_tensor_calc(), min_distance2(), nbhood(), nsq_calculate_energies(), nsq_calculate_ia(), nsq_calculate_ia_iccp3m(), nsq_calculate_virials(), observable_interacts_with(), overlap_angle_energy(), print_bond_len(), reducepos(), tab_angle_energy(), test_mesh_elements(), and vs_relate_to().

void grid_changed_box_l ( )

called from mpi_bcast_parameter .

Definition at line 124 of file grid.c.

References box_l, box_l_i, calc_minimal_box_dimensions(), GRID_TRACE, i, local_box_l, my_left, my_right, node_grid, node_pos, and this_node.

Referenced by grid_changed_n_nodes(), on_parameter_change(), and propagate_press_box_pos_and_rescale_npt().

void grid_changed_n_nodes ( )

called from mpi_bcast_parameter .

Definition at line 148 of file grid.c.

References boundary, calc_node_neighbors(), comm_cart, grid_changed_box_l(), GRID_TRACE, MPI_Cart_coords(), MPI_Cart_create(), MPI_Comm_free(), MPI_Comm_rank(), MPI_COMM_WORLD, node_grid, node_neighbors, node_pos, and this_node.

Referenced by init_node_grid(), and on_parameter_change().

void init_node_grid ( )

Make sure that the node grid is set, eventually determine one automatically.

Definition at line 64 of file grid.c.

References CELL_FLAG_GRIDCHANGED, cells_on_geometry_change(), and grid_changed_n_nodes().

Referenced by on_program_start().

int map_3don2d_grid	(	int	g3d[3],
		int	g2d[3],
		int	mult[3]
	)

calculate 'best' mapping between a 2d and 3d grid.

This we need for the communication from 3d domain decomposition to 2d row decomposition. The dimensions of the 2d grid are resorted, if necessary, in a way that they are multiples of the 3d grid dimensions.

Parameters: