Python API#
import dyng gives the whole public Python API: the names of the dyng package (the graph,
and from 0.3 the hypergraph: dyng.Hypergraph, dyng.HyperedgeBatch,
dyng.HypergraphProperties, dyng.CapacityPolicy, dyng.HyperApplySummary;
Use hypergraphs from Python) and the modules dyng.sssp,
dyng.cycle_count, dyng.mosp (0.2), the tutorial algorithms dyng.dynamic_bfs and
dyng.triangle_delta (0.2), dyng.io, dyng.generators and dyng.testing. The
names mirror the C++ API (C++ API): dyng::sssp::compute is dyng.sssp.compute,
dyng::sssp::options is dyng.sssp.Options, and option keywords are the C++ field names
(max_length=, delta=). Names with a leading underscore (dyng._core, the native module) are
private and may change in any release. What the C++ library has and the Python
package does not yet bind is listed in Python gaps.
The pages below the reference are generated from the docstrings and signatures of
python/dyng (and the committed stubs of the native module). The rest of this page explains the
rules that hold across the whole package (ADR 0011).
Resources and backends#
A dyng.Resources chooses the backend: dyng.Resources.sequential(),
dyng.Resources.openmp(num_threads=0) (0 is the OpenMP default, which honours
OMP_NUM_THREADS) or dyng.Resources("openmp"). The CUDA backend comes with a CUDA plugin wheel
(pip install "dyng[cu13]"; Install):
dyng.Resources.cuda(device=0, stream=None) (stream: an integer handle, a CuPy or PyTorch
stream, or any object with __cuda_stream__; a stream object is kept alive by the resources and
by what is built with them). Without a usable plugin dyng.Resources.cuda() raises
dyng.NotSupportedError with a message that names the plugin wheels and, when plugins
are installed, why none is used. Every function takes resources=None as a keyword; None means
the resources the graph was built with for every call that takes a graph (compute, update,
Graph.apply), and the process-wide default (dyng.get_default_resources(), set with
dyng.set_default_resources()) for the rest (building graphs, the readers). All backends
return the same results bit for bit.
The native module: the CPU module or a CUDA plugin#
import dyng loads no native module. The first use of dynG (the first graph, resources, reader
or dyng.__version__) chooses one for the process (ADR 0031): dyng._core (sequential and
OpenMP), or the module of an installed CUDA plugin (dyng_cu12 / dyng_cu13: every backend,
CUDA included; its default backend is then cuda). Of the installed plugins of the same version
as dyng that can run here (an NVIDIA driver of the plugin’s CUDA major or newer and a visible
device), the one of the driver’s CUDA major wins. When plugins are installed but none can be
used, dynG runs on dyng._core and issues one dyng.BackendWarning (a UserWarning)
that names each plugin and the reason. DYNG_CPU_ONLY=1 in the environment (also true,
yes, on; 0, false, no, off or empty mean unset; other values are ignored with a
warning), or
dyng.use_cpu_only() before the first use, chooses the CPU module without looking at the
plugins (and without the warning). dyng.show_config() prints the active module, why it
was chosen and every installed plugin’s state; dyng.config() returns the same as a dict
(native_module, selection, plugins).
Dtype dispatch#
The compiled library holds five graph types, (vertex id, edge offset, weight) =
(int32, int32, int32), (int32, int64, int32), (int64, int64, int32),
(int32, int32, unweighted) and (int32, int64, unweighted). The Python layer picks one from the
inputs:
The vertex id type comes from the declared dtype of the id arrays: int32 selects int32 ids and int64 selects int64 ids; narrower integer types widen to int32 and uint32 to int64. An int64 array never selects int32 ids on its own; pass
vertex_dtype="int32"to ask for them, and every value is then checked. Inputs without a dtype (Python lists, ranges) take int32 when every value fits, int64 otherwise.The edge offset type is int32 whenever the stored edge count fits (ADR 0009), int64 with int64 ids or when
row_ptris declared int64.Where the type is already fixed (a batch, a tree or parents for an existing graph), ids are converted to the graph’s type with a range check: a value that does not fit raises
dyng.InvalidArgumentErrorinstead of wrapping. So anEdgeBatchof int64 NumPy arrays updates an int32 graph.Weights are int32 integers (range-checked); floating-point and boolean weights are rejected.
An unsupported combination raises
dyng.NotSupportedErrorlisting the supported ones.Hypergraphs (0.3) have int32 vertex ids, with or without int32 weights: member arrays that declare int64 raise
dyng.NotSupportedErrorunlessvertex_dtype="int32"is given (then every value is checked). Hyperedge ids, offsets andinserted_idsare int64 and never narrowed.Inputs in CUDA device memory (CuPy, PyTorch, JAX, a
dyng.Arrayof the CUDA backend) need a CUDA plugin’s module and are copied to the host once (graphs and batches are built from host arrays in this release), after the producer’s stream (__dlpack__(stream=...)).
Result arrays#
Result arrays (tree.distances, tree.parents, hist.counts, paths.path_costs) are
dyng.Array objects (1-D, except the (n, K) path costs of dyng.mosp): read-only, zero-copy views with __dlpack__, __dlpack_device__ and
__array_interface__, so np.asarray(a), np.from_dlpack(a) and torch.from_dlpack(a) see the
library’s memory without a copy, and every view keeps that memory alive. An Array belongs to one
state of its result: once the result is updated, using it raises
dyng.StaleResultError; read the property again. a.to_numpy() returns a copy (the
default), a.to_numpy(copy=False) the read-only view. Views that NumPy or another library made
keep showing the state they were made from, unchanged: while such a view is alive, an update
copies the result’s state first and changes the copy (copy-on-write), so exported memory is
never changed or freed under a view. That costs one copy of the result’s arrays per update while
a view is alive; drop views you no longer need to update in place. DLPack consumers that ask for
DLPack >= 1.0 get a read-only zero-copy capsule; the unversioned capsule of older consumers,
which cannot be marked read-only, is a copy. PyTorch and CuPy have no read-only arrays: a tensor
from a.to_torch() or torch.from_dlpack(a) aliases the result and must not be written.
copy.copy / copy.deepcopy of a result or a graph are clone(); batches, resources and arrays
pickle, graphs and results explain how to send them (to_csr(), Result.from_arrays()).
Results of the CUDA backend live in device memory (a.device == "cuda:0",
a.__dlpack_device__() == (2, 0); mosp’s path_costs and cycle_count’s counts are host
arrays on every backend). Such an Array has __cuda_array_interface__ (version 3, read-only,
with the stream of the call that last wrote the result) instead of __array_interface__, so
cupy.asarray(a), cupy.from_dlpack(a) and torch.from_dlpack(a) view it on the device
without a copy. A DLPack consumer passes its stream (__dlpack__(stream=...)) and the Array
orders that stream after the result’s with a CUDA event (PLAN Section 5.4, rule 4); the host
does not wait. shape, dtype, size and ndim read no element; a.to_numpy() copies to the
host; a.to_numpy(copy=None) returns a read-only host copy that the Array keeps (for host
memory: the read-only view), and a.to_numpy(copy=False) raises for device memory. Indexing,
iteration, comparisons, repr and np.asarray(a) read through that kept copy.
Results, versions and stale results#
A graph has a version that grows by one with every applied batch, and every result remembers the
version it matches (Results and versions). dyng.sssp.update(g, batch, tree) applies the batch to g and brings tree up to date; another result on the same graph is
then stale, and using it raises dyng.StaleResultError. To keep several results
current, update them together: dyng.update(g, batch, tree, hist) applies the batch once and
returns one Stats per result (any mix of dyng.sssp, dyng.cycle_count and dyng.mosp
results).
Exceptions#
Every error of the library is a dyng.Error, and each class is also the built-in
exception a Python user expects:
C++ |
Python |
Also a |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
One warning class: dyng.BackendWarning (a UserWarning), issued once per process
when a CUDA plugin is installed but cannot be used and dynG falls back to the CPU module.
A call that raises leaves the graph and its results as they were, except where the C++
documentation says otherwise (a result poisoned by a failed update raises StaleResultError
until it is recomputed).
Threads and the GIL#
The GIL is released around every native call that runs an algorithm or I/O, so several Python
threads can run dynG at once. Graphs and results are not thread-safe in C++; the binding keeps
them memory-safe with a reader/writer lock per object: calls that change a graph or a result
(apply, the updates) are serialized against every other call on it, and reads run in parallel.
Setting Resources.copy_policy and entering or leaving dyng.profile() change the shared
resources handle, so they wait until no native call runs in any thread. A batch reads its arrays
at every use: do not change them while a call that uses the batch runs.
Profiling#
with dyng.profile() as p: attaches a profiler to the default resources (or to the resources
given) for the calls inside the block; p.stages lists the stages (sssp.update,
sssp.identify_affected, …), p.to_csv() and p.to_json() export them, and p.to_dataframe()
returns a pandas DataFrame when pandas is installed (Profile an update).
Citing#
dyng.citation("sssp") returns the BibTeX entries of an algorithm’s papers (and
dyng.citation() the entry of dynG itself); dyng.algorithms() lists the algorithms of the
installed package with their maturity, backends and cite keys.