Package¶
-
class
pydtmc.
MarkovChain
(p, states=None, order=None)[source]¶ Bases:
object
Defines a Markov chain with given transition matrix and state names.
- Parameters
- Raises
ValidationError – if any input argument is not compliant.
-
property
absorbing_states
[source]¶ A property representing the absorbing states of the Markov chain.
-
property
absorption_probabilities
[source]¶ A property representing the absorption probabilities of the Markov chain. If the Markov chain is not absorbing and has no transient states, then None is returned.
-
property
absorption_times
[source]¶ A property representing the absorption times of the Markov chain. If the Markov chain is not absorbing, then None is returned.
-
property
accessibility_matrix
[source]¶ A property representing the accessibility matrix of the Markov chain.
- Return type
-
property
adjacency_matrix
[source]¶ A property representing the adjacency matrix of the Markov chain.
- Return type
-
static
approximation
(size, approximation_type, alpha, sigma, rho, k=None)[source]¶ The method approximates the Markov chain associated with the discretized version of the following first-order autoregressive process:
\(y_t = (1 - \rho) \alpha + \rho y_{t-1} + \varepsilon_t\)with \(\varepsilon_t \overset{i.i.d}{\sim} \mathcal{N}(0, \sigma_{\varepsilon}^{2})\)- Parameters
size (
int
) – the size of the Markov chain.approximation_type (
str
) – the approximation type to use (one of adda-cooper, rouwenhorst, tauchen or tauchen-hussey).alpha (
float
) – the constant term \(\alpha\), representing the unconditional mean of the process.sigma (
float
) – the standard deviation of the innovation term \(\varepsilon\).rho (
float
) – the autocorrelation coefficient \(\rho\), representing the persistence of the process across periods.in the Tauchen approximation, the number of standard deviations to approximate out to (if omitted, the value is set to 3);
in the Tauchen-Hussey approximation, the standard deviation used for the gaussian quadrature (if omitted, the value is set to an optimal default).
- Return type
- Returns
a tuple whose first element is a Markov chain and whose second element is a vector of nodes.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the approximation type is neither Tauchen nor Tauchen-Hussey and k is not equal to None or if the gaussian quadrature fails to converge in the Tauchen-Hussey approach.
-
are_communicating
(state1, state2)[source]¶ The method verifies whether the given states of the Markov chain are communicating.
-
backward_committor
(states1, states2)¶ Alias of backward_committor_probabilities.
-
backward_committor_probabilities
(states1, states2)[source]¶ The method computes the backward committor probabilities between the given subsets of the state space defined by the Markov chain.
Aliases: backward_committor- Parameters
- Return type
- Returns
the backward committor probabilities if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the two sets are not disjoint.
-
static
birth_death
(p, q, states=None)[source]¶ The method generates a birth-death Markov chain of given size and from given probabilities.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if q and p have a different size or if the vector resulting from the sum of q and p contains any value greater than 1.
-
closest_reversible
(distribution, weighted=False)[source]¶ The method computes the closest reversible of the Markov chain.
Notes: the algorithm is described in Computing the nearest reversible Markov Chain (Nielsen & Weber, 2015).- Parameters
- Return type
Optional
[~MarkovChain]- Returns
a Markov chain if the algorithm finds a solution, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if a weighted Frobenius norm is used and the distribution contains zero-valued probabilities.
-
property
communicating_classes
[source]¶ A property representing the communicating classes of the Markov chain.
-
conditional_probabilities
(state)[source]¶ The method computes the probabilities, for all the states of the Markov chain, conditioned on the process being at a given state.
Aliases: conditional_distribution- Parameters
- Return type
- Returns
the conditional probabilities of the Markov chain states.
- Raises
ValidationError – if any input argument is not compliant.
-
property
determinant
[source]¶ A property representing the determinant the transition matrix of the Markov chain.
- Return type
-
property
entropy_rate
[source]¶ A property representing the entropy rate of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
property
entropy_rate_normalized
[source]¶ A property representing the entropy rate, normalized between 0 and 1, of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
expected_rewards
(steps, rewards)[source]¶ The method computes the expected rewards of the Markov chain after N steps, given the reward value of each state.
- Parameters
- Return type
- Returns
the expected rewards of each state of the Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
expected_transitions
(steps, initial_distribution=None)[source]¶ The method computes the expected number of transitions performed by the Markov chain after N steps, given the initial distribution of the states.
- Parameters
- Return type
- Returns
the expected number of transitions on each state of the Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
static
fit_function
(f, possible_states, quadrature_type='newton-cotes', quadrature_interval=None)[source]¶ The method fits a Markov chain using the given transition function and with the given quadrature type.
- Parameters
f (
Callable
[[float
,float
],float
]) – the transition function of the process.possible_states (
List
[str
]) – the possible states of the process.quadrature_type (
str
) – the quadrature type to use for the computation of nodes and weights (one of gauss-chebyshev, gauss-legendre, neiderreiter, newton-cotes, simpson-rule or trapezoid-rule; by default, newton-cotes).quadrature_interval (
Optional
[Tuple
[Union
[float
,int
],Union
[float
,int
]]]) – the quadrature interval to use for the computation of nodes and weights (by default, the interval [0, 1]).
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the Gauss-Legendre quadrature fails to converge or if the Simpson rule quadrature is attempted on an even number of possible states.
-
static
fit_standard
(possible_states, walk, fitting_type, k=None)[source]¶ The method fits a Markov chain using either the maximum a posteriori approach (MAP) or the maximum likelihood approach (MLE).
- Parameters
possible_states (
List
[str
]) – the possible states of the process.walk (
Union
[List
[int
],List
[str
]]) – the observed sequence of states.fitting_type (
str
) – the type of fitting to use (either map or mle).in the maximum a posteriori approach, the matrix for the a priori distribution (if omitted, a default value of 1 is assigned to each parameter);
in the maximum likelihood approach, a boolean indicating whether to apply a Laplace smoothing to compensate for the unseen transition combinations (if omitted, the value is set to False).
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
forward_committor
(states1, states2)¶ Alias of forward_committor_probabilities.
-
forward_committor_probabilities
(states1, states2)[source]¶ The method computes the forward committor probabilities between the given subsets of the state space defined by the Markov chain.
Aliases: forward_committor- Parameters
- Return type
- Returns
the forward committor probabilities if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the two sets are not disjoint.
-
static
from_dictionary
(d)[source]¶ The method generates a Markov chain from the given dictionary, whose keys represent state pairs and whose values represent transition probabilities.
- Parameters
d (
Dict
[Tuple
[str
,str
],Union
[float
,int
]]) – the dictionary to transform into the transition matrix.- Return type
- Returns
a Markov chain.
- Raises
ValueError – if the transition matrix defined by the dictionary is not valid.
ValidationError – if any input argument is not compliant.
-
static
from_file
(file_path, file_type='json')[source]¶ The method reads a Markov chain from the given file.
In the json format, data must be structured as an array of objects with the following properties:state_from (string)state_to (string)probability (float or int)In the text format, every line of the file must have the following format:<state_from> <state_to> <probability>- Parameters
- Return type
- Returns
a Markov chain.
- Raises
FileNotFoundError – if the file does not exist.
OSError – if the file cannot be read or is empty.
ValidationError – if any input argument is not compliant.
ValueError – if the file contains invalid data.
-
static
from_matrix
(m, states=None)[source]¶ The method generates a Markov chain with the given state names, whose transition matrix is obtained through the normalization of the given matrix.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
property
fundamental_matrix
[source]¶ A property representing the fundamental matrix of the Markov chain. If the Markov chain has no transient states, then None is returned.
-
hitting_probabilities
(states=None)[source]¶ The method computes the hitting probability, for all the states of the Markov chain, to the given set of states.
-
static
identity
(size, states=None)[source]¶ The method generates a Markov chain of given size based on an identity transition matrix.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_absorbing
[source]¶ A property indicating whether the Markov chain is absorbing.
- Return type
-
is_absorbing_state
(state)[source]¶ The method verifies whether the given state of the Markov chain is absorbing.
- Parameters
- Return type
- Returns
True if the state is absorbing, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
is_accessible
(state_target, state_origin)[source]¶ The method verifies whether the given target state is reachable from the given origin state.
-
property
is_aperiodic
[source]¶ A property indicating whether the Markov chain is aperiodic.
- Return type
-
property
is_canonical
[source]¶ A property indicating whether the Markov chain has a canonical form.
- Return type
-
is_cyclic_state
(state)[source]¶ The method verifies whether the given state is cyclic.
- Parameters
- Return type
- Returns
True if the state is cyclic, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_ergodic
[source]¶ A property indicating whether the Markov chain is ergodic or not.
- Return type
-
property
is_irreducible
[source]¶ A property indicating whether the Markov chain is irreducible.
- Return type
-
is_recurrent_state
(state)[source]¶ The method verifies whether the given state is recurrent.
- Parameters
- Return type
- Returns
True if the state is recurrent, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_reversible
[source]¶ A property indicating whether the Markov chain is reversible.
- Return type
-
property
is_symmetric
[source]¶ A property indicating whether the Markov chain is symmetric.
- Return type
-
is_transient_state
(state)[source]¶ The method verifies whether the given state is transient.
- Parameters
- Return type
- Returns
True if the state is transient, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
kemeny_constant
[source]¶ A property representing the Kemeny’s constant of the fundamental matrix of the Markov chain. If the Markov chain is not absorbing, then None is returned.
-
property
mean_first_passage_times
[source]¶ A property representing the mean first passage times of the Markov chain. If the Markov chain is not ergodic, then None is returned.
Aliases: mfpt
-
mean_first_passage_times_between
(states_target, states_origin)[source]¶ The method computes the mean first passage times between the given subsets of the state space.
Aliases: mfpt_between- Parameters
- Return type
- Returns
the mean first passage times between the given subsets if the Markov chain is irreducible, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
mean_first_passage_times_to
(states=None)[source]¶ The method computes the mean first passage times, for all the states, to the given set of states.
Aliases: mfpt_to
-
mfpt_between
(states_target, states_origin)¶ Alias of mean_first_passage_times_between.
-
property
mixing_rate
[source]¶ A property representing the mixing rate of the Markov chain. If the SLEM (second largest eigenvalue modulus) cannot be computed, then None is returned.
-
mixing_time
(initial_distribution=None, jump=1, cutoff_type='natural')[source]¶ The method computes the mixing time of the Markov chain, given the initial distribution of the states.
- Parameters
initial_distribution (
Optional
[Iterable
]) – the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).jump (
int
) – the number of steps in each iteration (by default, 1).cutoff_type (
str
) – the type of cutoff to use (either natural or traditional; by default, natural).
- Return type
- Returns
the mixing time if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
periods
[source]¶ A property representing the period of each communicating class defined by the Markov chain.
-
property
pi
[source]¶ A property representing the stationary distributions of the Markov chain.
Aliases: stationary_distributions, steady_states
-
predict
(steps, initial_state=None, include_initial=False, output_indices=False, seed=None)[source]¶ The method simulates the most probable outcome of a random walk of N steps.
Notes: in case of probability tie, the subsequent state is chosen uniformly at random among all the equiprobable states.- Parameters
steps (
int
) – the number of steps.initial_state (
Union
[int
,str
,None
]) – the initial state of the prediction (if omitted, it is chosen uniformly at random).include_initial (
bool
) – a boolean indicating whether to include the initial state in the output sequence (by default, False).output_indices (
bool
) – a boolean indicating whether to the output the state indices (by default, False).seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
the sequence of states produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
prior_probabilities
(hyperparameter=None)[source]¶ The method computes the prior probabilities, in logarithmic form, of the Markov chain.
- Parameters
hyperparameter (
Optional
[Iterable
]) – the matrix for the a priori distribution (if omitted, a default value of 1 is assigned to each parameter).- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
static
random
(size, states=None, zeros=0, mask=None, seed=None)[source]¶ The method generates a Markov chain of given size with random transition probabilities.
- Parameters
size (
int
) – the size of the Markov chain.states (
Optional
[List
[str
]]) – the name of each state (if omitted, an increasing sequence of integers starting at 1).zeros (
int
) – the number of zero-valued transition probabilities (by default, 0).mask (
Optional
[Iterable
]) – a matrix representing the locations and values of fixed transition probabilities.seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the number of zero-valued transition probabilities exceeds the maximum threshold.
-
property
rank
[source]¶ A property representing the rank of the transition matrix of the Markov chain.
- Return type
-
property
recurrence_times
[source]¶ A property representing the recurrence times of the Markov chain. If the Markov chain has no recurrent states, then None is returned.
-
property
recurrent_classes
[source]¶ A property representing the recurrent classes defined by the Markov chain.
-
property
recurrent_states
[source]¶ A property representing the recurrent states of the Markov chain.
-
redistribute
(steps, initial_status=None, include_initial=False, output_last=True)[source]¶ The method simulates a redistribution of states of N steps.
- Parameters
steps (
int
) – the number of steps.initial_status (
Union
[int
,Iterable
,None
]) – the initial state or the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).include_initial (
bool
) – a boolean indicating whether to include the initial distribution in the output sequence (by default, False).output_last (
bool
) – a boolean indicating whether to the output only the last distributions (by default, True).
- Return type
- Returns
the sequence of redistributions produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
property
relaxation_rate
[source]¶ A property representing the relaxation rate of the Markov chain. If the SLEM (second largest eigenvalue modulus) cannot be computed, then None is returned.
-
sensitivity
(state)[source]¶ The method computes the sensitivity matrix of the stationary distribution with respect to a given state.
-
property
spectral_gap
[source]¶ A property representing the spectral gap of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
to_canonical
()¶ Alias of to_canonical_form.
- Return type
-
to_canonical_form
()[source]¶ The method returns the canonical form of the Markov chain.
Aliases: to_canonical- Return type
- Returns
a Markov chain.
-
to_directed_graph
(multi=True)[source]¶ The method returns a directed graph representing the Markov chain.
Aliases: to_digraph- Parameters
multi (
bool
) – a boolean indicating whether the graph is allowed to define multiple edges between two nodes (by default, True).- Return type
DiGraph
- Returns
a directed graph.
- Raises
ValidationError – if any input argument is not compliant.
-
to_file
(file_path, file_type='json')[source]¶ The method writes a Markov chain to the given file.
- Parameters
- Raises
OSError – if the file cannot be written.
ValidationError – if any input argument is not compliant.
-
to_graph
(multi=True)¶ Alias of to_directed_graph.
- Return type
DiGraph
-
to_lazy
(inertial_weights=0.5)¶ Alias of to_lazy_chain.
- Return type
-
to_lazy_chain
(inertial_weights=0.5)[source]¶ The method returns a lazy chain by adjusting the state inertia of the original process.
- Parameters
inertial_weights (
Union
[float
,int
,Iterable
]) – the inertial weights to apply for the transformation (by default, 0.5).- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
to_subchain
(states)[source]¶ The method returns a subchain containing all the given states plus all the states reachable from them.
-
property
topological_entropy
[source]¶ A property representing the topological entropy of the Markov chain.
- Return type
-
property
transient_classes
[source]¶ A property representing the transient classes defined by the Markov chain.
-
property
transient_states
[source]¶ A property representing the transient states of the Markov chain.
-
transition_probability
(state_target, state_origin)[source]¶ The method computes the probability of a given state, conditioned on the process being at a given specific state.
-
walk
(steps, initial_state=None, final_state=None, include_initial=False, output_indices=False, seed=None)[source]¶ The method simulates a random walk of N steps.
- Parameters
steps (
int
) – the number of steps.initial_state (
Union
[int
,str
,None
]) – the initial state of the walk (if omitted, it is chosen uniformly at random).final_state (
Union
[int
,str
,None
]) – the final state of the walk (if specified, the simulation stops as soon as it is reached even if not all the steps have been performed).include_initial (
bool
) – a boolean indicating whether to include the initial state in the output sequence (by default, False).output_indices (
bool
) – a boolean indicating whether to the output the state indices (by default, False).seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
the sequence of states produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_eigenvalues
(mc, dpi=100)[source]¶ The function plots the eigenvalues of the Markov chain on the complex plane.
- Parameters
mc (~MarkovChain) – the target Markov chain.
dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_graph
(mc, nodes_color=True, nodes_type=True, edges_color=True, edges_value=True, dpi=100)[source]¶ The function plots the directed graph of the Markov chain.
Notes: Graphviz and Pydot are not required, but they provide access to extended graphs with additional features.- Parameters
mc (~MarkovChain) – the target Markov chain.
nodes_color (
bool
) – a boolean indicating whether to display colored nodes based on communicating classes (by default, True).nodes_type (
bool
) – a boolean indicating whether to use a different shape for every node type (by default, True).edges_color (
bool
) – a boolean indicating whether to display edges using a gradient based on transition probabilities, valid only for extended graphs (by default, True).edges_value (
bool
) – a boolean indicating whether to display the transition probability of every edge (by default, True).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_redistributions
(mc, distributions, initial_status=None, plot_type='projection', dpi=100)[source]¶ The function plots a redistribution of states on the given Markov chain.
- Parameters
mc (~MarkovChain) – the target Markov chain.
distributions (
Union
[int
,List
[ndarray
]]) – a sequence of redistributions or the number of redistributions to perform.initial_status (
Union
[int
,Iterable
,None
]) – the initial state or the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).plot_type (
str
) – the type of plot to display (either heatmap or projection; projection by default).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValueError – if the “distributions” parameter represents a sequence of redistributions and the “initial_status” parameter does not match its first element.
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_walk
(mc, walk, initial_state=None, plot_type='histogram', dpi=100)[source]¶ The function plots a random walk on the given Markov chain.
- Parameters
mc (~MarkovChain) – the target Markov chain.
walk (
Union
[int
,List
[int
],List
[str
]]) – a sequence of states or the number of simulations to perform.initial_state (
Union
[int
,str
,None
]) – the initial state of the walk (if omitted, it is chosen uniformly at random).plot_type (
str
) – the type of plot to display (one of histogram, sequence and transitions; histogram by default).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValueError – if the “walk” parameter represents a sequence of states and the “initial_state” parameter does not match its first element.
ValidationError – if any input argument is not compliant.