From Carpet Workshop

dh.hh declares different data bounding boxes:

  • light_dboxes is a data structure with a simplified description of the domain regions. Its members consist of bounding boxes describing the exterior region (the whole domain including outer and/or ghost zones and/or symmetry boundaries), the owned region, corresponding to the region owned by a particular processor (MPI job) that is actually evolved in time and the interior region that excludes the ghost zones and/or boundaries.
  • local_dboxes, on the other hand, describes information about the processor-local region. It contains information about the buffer zones and the active region (owned region less the buffer zones).
  • full_dboxes contains the complete description of a region:
    • exterior describes the whole region (domain) including outer boundaries.
    • outer_boundaries (ob) is a set of bboxes describing the outer boundaries or symmetry boundaries.
    • communicated region is the exterior less outer_boundaries. The nomenclature may be misleading but this region does represent a region where Carpet "communicates" i.e applies the Berger-Oliger algorithm, and not a region where inter-process communication happens.
    • boundaries region indeed represent the ghost zones (the region where data needs to be filled from other processors data). The nomenclature is different from the Cactus one since its definition of ghost zones may also include the outer_boundaries, while this is not the case for this Carpet region.
    • owned is the region evolved in time, therefore it does not include the boundaries (ghost zone).
    • buffers are regions where the prolongation (interpolation) from coarse to fine grid happens. Its size is usually determined by the number of time iterations the time integrator happens to have and the stencil size. For example, Runge-Kutta 4th order takes 4 mini-steps. If the spatial stencil is also 4th order, then the buffer region would have a width of 8 mesh spacings on each direction.
    • active is the owned region minus the buffers region.
  • fast_dboxes is a data structure defined to take care of the communication of bounding boxes amongst several processors.

To help visualising the several different regions, a few plots were drawn below:

  • This plot shows the whole physical domain (black box dom) and an example of a region on the corner of the physical domain sent to a particular process (MPI job). The red box bounds the exterior region (ext), that is simply the union of the outer boundaries (blue boxes ob) and the communicated region (green box com). Outer boundaries for Carpet may also include the symmetry boundaries. The xfig version of this plot can be found here and its psfrag here.
  • This plot shows a typical domain decomposed region where two exterior regions are sent to two different processors (MPI jobs). The processor owned region is represented by the green boxes. The union of all owned regions covers exactly the physical domain. The region marked by dashed and solid blue lines is the boundary region. The solid blue lines shows explicitly the intersection of the boundary of processor 0 boundary with processor 1 owned region (and vice versa). At last the red box describe the exterior region, the union of the owned and boundary regions for each processor. The xfig version of this plot can be found here and its psfrag here.