DiFfRG::MapScheduler Class Reference

#include <map_scheduler.hh>

Classes
struct	Entry

Public Member Functions
bool	active () const
uint	rank () const
uint	n_ranks () const
MPI_Comm	comm () const
MapSlice	schedule (size_t integrator_id, void *dest, size_t elem_size, size_t grid_size, size_t quadrature_volume, bool splittable, MapTarget target)
	Register one map() call and return this rank's slice of it.
bool	plan_contains (size_t integrator_id) const
bool	has_open_plan () const
void	complete ()
	Exchange every slice registered since the last completion.
void	poison (const char *reason)
void	set_batched (const bool batched)
	Tell the scheduler whether the caller has a deferral scope open.
void	set_quantum (double quantum)
double	quantum () const
	The explicit override, or 0 when the automatic defaults are in force.
void	set_verbose (bool verbose)

Static Public Member Functions
static MapScheduler &	instance ()

Protected Member Functions
	MapScheduler ()
	MapScheduler (uint rank, uint n_ranks)
	Bind to an explicit rank and rank count instead of MPI_COMM_WORLD.
bool	prepare_batch ()
void	finish_batch ()
	Land every other rank's slices out of the receive buffer, then close the batch.

Protected Attributes
std::vector< char >	m_send
std::vector< char >	m_recv
std::vector< int >	m_counts
std::vector< int >	m_displs

Private Member Functions
std::vector< uint >	least_loaded (size_t r, MapResource resource) const
void	reset_load ()
	Zero every resource budget, sized to the rank count.
uint64_t	plan_hash () const
void	log_plan () const

Static Private Member Functions
static size_t	part (size_t grid_size, size_t r, size_t j)
	Canonical partition: slice j of r covers [part(G, r, j), part(G, r, j + 1)).

Private Attributes
MPI_Comm	m_comm
uint	m_rank = 0
uint	m_n_ranks = 1
double	m_quantum_override = 0.
	0 == no override, use the hardware-derived per-space thresholds.
bool	m_quantum_pinned = false
bool	m_batched = false
	Whether a DeferredMaps scope is open; see set_batched().
size_t	m_batch_index = 0
size_t	m_verify_every = 64
bool	m_verbose = false
bool	m_logged = false
const char *	m_poisoned = nullptr
std::vector< Entry >	m_plan
std::array< std::vector< double >, n_map_resources >	m_load
bool	m_simulated = false
	Simulated instances (see the rank/n_ranks constructor) never communicate.

Static Private Attributes
static constexpr size_t	verify_head = 8
	Batches at the start of a run that are always verified, whatever m_verify_every says.

Constructor & Destructor Documentation

MapScheduler() [1/2]

DiFfRG::MapScheduler::MapScheduler ( )

protected

MapScheduler() [2/2]

DiFfRG::MapScheduler::MapScheduler ( uint rank, uint n_ranks )

protected

Bind to an explicit rank and rank count instead of MPI_COMM_WORLD.

For tests that simulate several ranks inside one process: the schedule, the byte accounting and the packing are all pure functions of the call sequence, so they can be exercised without an MPI build. Only the single Allgatherv in between cannot.

Member Function Documentation

active()

bool DiFfRG::MapScheduler::active ( ) const

inline

Whether there is more than one rank to schedule over. False in a serial run, which makes every method below a no-op and the whole feature zero-overhead.

comm()

MPI_Comm DiFfRG::MapScheduler::comm ( ) const

inline

complete()

void DiFfRG::MapScheduler::complete

(

)

Exchange every slice registered since the last completion.

One MPI_Allgatherv for the whole batch. Called by MapCompletion::flush() after the device results have landed in host memory.

finish_batch()

void DiFfRG::MapScheduler::finish_batch ( )

protected

Land every other rank's slices out of the receive buffer, then close the batch.

has_open_plan()

bool DiFfRG::MapScheduler::has_open_plan ( ) const

inline

instance()

static MapScheduler & DiFfRG::MapScheduler::instance ( )

static

least_loaded()

std::vector< uint > DiFfRG::MapScheduler::least_loaded ( size_t r, MapResource resource ) const

private

log_plan()

void DiFfRG::MapScheduler::log_plan ( ) const

private

n_ranks()

uint DiFfRG::MapScheduler::n_ranks ( ) const

inline

part()

static size_t DiFfRG::MapScheduler::part ( size_t grid_size, size_t r, size_t j )

inlinestaticprivate

Canonical partition: slice j of r covers [part(G, r, j), part(G, r, j + 1)).

plan_contains()

bool DiFfRG::MapScheduler::plan_contains

(

size_t

integrator_id

)

const

Whether the open plan already contains a map() from this integrator. Used in place of a rank-local "is a result still staged" test, which ranks would answer differently.

plan_hash()

uint64_t DiFfRG::MapScheduler::plan_hash ( ) const

private

poison()

void DiFfRG::MapScheduler::poison

(

const char *

reason

)

Discard the open plan without communicating and poison the scheduler, so that the next collective aborts with a diagnosis instead of hanging. See DeferredMaps' destructor.

prepare_batch()

bool DiFfRG::MapScheduler::prepare_batch ( )

protected

Verify agreement, compute the byte accounting and pack this rank's slices. False if there is nothing to exchange, in which case the batch is already closed.

quantum()

double DiFfRG::MapScheduler::quantum ( ) const

inline

The explicit override, or 0 when the automatic defaults are in force.

rank()

uint DiFfRG::MapScheduler::rank ( ) const

inline

reset_load()

void DiFfRG::MapScheduler::reset_load ( )

private

Zero every resource budget, sized to the rank count.

schedule()

MapSlice DiFfRG::MapScheduler::schedule	(	size_t	integrator_id,
		void *	dest,
		size_t	elem_size,
		size_t	grid_size,
		size_t	quadrature_volume,
		bool	splittable,
		MapTarget	target )

Register one map() call and return this rank's slice of it.

Called by every rank for every map(), including ranks that end up computing nothing – the entry with count == 0 is what keeps the collective in complete() matched. Deciding from rank-local state instead is the one way to deadlock this design.

Parameters

integrator_id	stable, construction-order id of the calling integrator
dest	base of the caller's destination array (unoffset)
elem_size	sizeof one result element
grid_size	number of external grid points, G
quadrature_volume	points per integral, Q
splittable	whether this coordinate system may be windowed (see class docs). True for every coordinate system today; kept as a parameter so a future one that cannot be windowed has somewhere to say so.
target	the resource the calling integrator's execution space runs on, and the number of evaluations that saturate one rank's share of it, from map_target<ExecutionSpace>(). The threshold sets the split width; the resource selects which budget the resulting slices are charged to. An explicit user override (DIFFRG_MAP_QUANTUM or /integration/map_quantum) replaces the threshold, but never the resource.

set_batched()

void DiFfRG::MapScheduler::set_batched ( const bool batched )

inline

Tell the scheduler whether the caller has a deferral scope open.

Only the split width depends on it, and only through which threshold schedule() measures against – see internal::launch_threshold. Pushed in by MapCompletion::set_deferral() rather than read back out of MapCompletion, so the dependency between the two stays one-way.

Safe for the plan hash: DeferredMaps is constructed by replicated model code, so this flag holds the same value on every rank at every schedule() call and the plan remains a pure function of the call sequence.

set_quantum()

void DiFfRG::MapScheduler::set_quantum

(

double

quantum

)

Override the per-execution-space fill thresholds with one explicit value. 0 restores the automatic, hardware-derived defaults.

set_verbose()

void DiFfRG::MapScheduler::set_verbose ( bool verbose )

inline

Member Data Documentation

m_batch_index

size_t DiFfRG::MapScheduler::m_batch_index = 0

private

Completed batches, used to space out the plan-agreement collective. Only ever incremented on the path where every rank agrees there is a batch, so it cannot drift between ranks.

m_batched

bool DiFfRG::MapScheduler::m_batched = false

private

Whether a DeferredMaps scope is open; see set_batched().

m_comm

MPI_Comm DiFfRG::MapScheduler::m_comm

private

m_counts

std::vector<int> DiFfRG::MapScheduler::m_counts

protected

m_displs

std::vector<int> DiFfRG::MapScheduler::m_displs

protected

m_load

std::array<std::vector<double>, n_map_resources> DiFfRG::MapScheduler::m_load

private

Running kernel-evaluation count per rank within the open plan, one budget per resource; reset by complete(). Never summed across resources – see the class docs.

m_logged

bool DiFfRG::MapScheduler::m_logged = false

private

m_n_ranks

uint DiFfRG::MapScheduler::m_n_ranks = 1

private

m_plan

std::vector<Entry> DiFfRG::MapScheduler::m_plan

private

m_poisoned

const char* DiFfRG::MapScheduler::m_poisoned = nullptr

private

m_quantum_override

double DiFfRG::MapScheduler::m_quantum_override = 0.

private

0 == no override, use the hardware-derived per-space thresholds.

m_quantum_pinned

bool DiFfRG::MapScheduler::m_quantum_pinned = false

private

m_rank

uint DiFfRG::MapScheduler::m_rank = 0

private

m_recv

std::vector<char> DiFfRG::MapScheduler::m_recv

protected

m_send

std::vector<char> DiFfRG::MapScheduler::m_send

protected

m_simulated

bool DiFfRG::MapScheduler::m_simulated = false

private

Simulated instances (see the rank/n_ranks constructor) never communicate.

m_verbose

bool DiFfRG::MapScheduler::m_verbose = false

private

m_verify_every

size_t DiFfRG::MapScheduler::m_verify_every = 64

private

Verify the plan hash on batch 1..verify_head and then every m_verify_every-th batch. 0 == verify every batch (DIFFRG_MAP_VERIFY_EVERY).

verify_head

size_t DiFfRG::MapScheduler::verify_head = 8

staticconstexprprivate

Batches at the start of a run that are always verified, whatever m_verify_every says.

---

The documentation for this class was generated from the following file:

• /home/runner/work/DiFfRG_current/DiFfRG_current/DiFfRG/include/DiFfRG/physics/integration/map_scheduler.hh