csle_tolerance.util package

Submodules

csle_tolerance.util.general_util module

class csle_tolerance.util.general_util.GeneralUtil[source]

Bases: object

Class with general utility functions related to csle-tolerance

static inverse_sigmoid(y) → float[source]

The inverse sigmoid function

Parameters

y – sigmoid(x)

Returns

sigmoid(x)^(-1)

static register_envs() → None[source]

Utility method for registering Gymnasium environments

Returns

None

static sample_next_state(transition_tensor: List[List[List[float]]], s: int, a: int, states: List[int]) → int[source]

Samples the next state of a MDP or POMDP

Parameters

  • transition_tensor – the transition tensor

  • s – the current state

  • a – the current action

  • states – the list of states

Returns

the next state

static sigmoid(x) → float[source]

The sigmoid function

Parameters

x – the input

Returns

sigmoid(x)

static threshold_probability(b1: float, threshold: float, k=- 20) → float[source]

Returns the probability of taking an action given a belief and a threshold

Parameters

  • b1 – the belief

  • threshold – the threshold

Returns

the stopping probability

csle_tolerance.util.intrusion_recovery_pomdp_util module

class csle_tolerance.util.intrusion_recovery_pomdp_util.IntrusionRecoveryPomdpUtil[source]

Bases: object

Class with utility functions for the intrusion-recovery POMDP

static action_space() → List[int][source]

Gets the action space of the POMDP

Returns

a list with the actions

static bayes_filter(s_prime: int, o: int, a: int, b: List[float], states: List[int], observations: List[int], observation_tensor: List[List[float]], transition_tensor: List[List[List[float]]]) → float[source]

A Bayesian filter to compute b[s_prime] of the POMDP

Parameters

  • s_prime – the state to compute the belief for

  • o – the latest observation

  • a – the latest action

  • b – the current belief

  • states – the list of states

  • observations – the list of observations

  • observation_tensor – the observation tensor

  • transition_tensor – the transition tensor of the POMDP

Returns

b[s_prime]

static cost_function(s: int, a: int, eta: float, negate: bool = False) → float[source]

Cost function of the POMDP

Parameters

  • s – the state

  • a – the action

  • eta – the scaling factor

  • negate – boolean flag, if true then return negated version of the cost (the reward)

Returns

the cost or reward

static cost_tensor(eta: float, states: List[int], actions: List[int], negate: bool = False) → List[List[float]][source]

Creates a |A|x|S| tensor with the costs (or rewards) of the POMDP

Parameters

  • eta – the cost scaling factor

  • states – the list of states

  • actions – the list of actions

  • negate – a boolean flag indicating whether the cost should be negated as a reward or not

Returns

A tensor with the costs (or rewards)

static generate_os_posg_game_file(game_config: csle_tolerance.dao.intrusion_recovery_game_config.IntrusionRecoveryGameConfig) → str[source]

Generates the POSG game file for HSVI

Parameters

game_config – the game configuration

Returns

a string with the contents of the config file

static generate_rewards(game_config: csle_tolerance.dao.intrusion_recovery_game_config.IntrusionRecoveryGameConfig) → List[str][source]

Generates the reward rows of the POSG config file of HSVI

Parameters

game_config – the game configuration

Returns

list of reward rows

static generate_transitions(game_config: csle_tolerance.dao.intrusion_recovery_game_config.IntrusionRecoveryGameConfig) → List[str][source]

Generates the transition rows of the POSG config file of HSVI

Parameters

game_config – the game configuration

Returns

list of transition rows

static initial_belief(p_a: float) → List[float][source]

Gets the initial belief state of the POMDP

Parameters

p_a – the attack probability

Returns

the initial belief state

static next_belief(o: int, a: int, b: List[float], states: List[int], observations: List[int], observation_tensor: List[List[float]], transition_tensor: List[List[List[float]]]) → List[float][source]

Computes the next belief using a Bayesian filter

Parameters

  • o – the latest observation

  • a – the latest action of player 1

  • b – the current belief

  • states – the list of states

  • observations – the list of observations

  • observation_tensor – the observation tensor

  • transition_tensor – the transition tensor

Returns

the new belief

static observation_function(s: int, o: int, num_observations: int) → float[source]

The observation function of the POMDP

Parameters

  • s – the state

  • o – the observation

  • num_observations – the total number of observations

Returns

the probability P(o | s)

static observation_space(num_observations: int) → List[int][source]

Gets the observation space of the POMDP

Parameters

num_observations – the number of observations

Returns

a list with the actions

static observation_tensor(states: List[int], observations: List[int]) → List[List[float]][source]

Creates a |S|x|O| tensor with the observation probabilities

Parameters

  • states – the list of states

  • observations – the list of observations

Returns

the observation tensor

static p_o_given_b_a1_a2(o: int, b: List[float], a: int, states: List[int], transition_tensor: List[List[List[float]]], observation_tensor: List[List[float]]) → float[source]

Computes P[o|a,b] of the POMDP

Parameters

  • o – the observation

  • b – the belief point

  • a – the action

  • states – the list of states

  • transition_tensor – the transition tensor

  • observation_tensor – the observation tensor

Returns

the probability of observing o when taking action a in belief point b

static pomdp_solver_file(config: csle_tolerance.dao.intrusion_recovery_pomdp_config.IntrusionRecoveryPomdpConfig) → str[source]

Gets the POMDP environment specification based on the format at http://www.pomdp.org/code/index.html, for the defender’s local problem against a static attacker

Parameters

config – the POMDP config

Returns

the file content as a string

static sample_initial_state(b1: List[float]) → int[source]

Samples the initial state

Parameters

b1 – the initial belief

Returns

the initial state

static sample_next_observation(observation_tensor: List[List[float]], s_prime: int, observations: List[int]) → int[source]

Samples the next observation

Parameters

  • s_prime – the new state

  • observations – the observation space

  • observation_tensor – the observation tensor

Returns

the next observation o

static sample_next_state_game(transition_tensor: List[List[List[List[float]]]], s: int, a1: int, a2: int) → int[source]

Samples the next observation

Parameters

  • s – the current state

  • a1 – the defender action

  • a2 – the attacker action

  • transition_tensor – the transition tensor

Returns

the next state a

static state_space() → List[int][source]

Gets the state space of the POMDP

Returns

a list with the states

static transition_function(s: int, s_prime: int, a: int, p_a: float, p_c_1: float, p_u: float, p_c_2: float) → float[source]

The transition function of the POMDP

Parameters

  • s – the state

  • s_prime – the next state

  • a – the action

  • p_a – the intrusion probability

  • p_c_1 – the crash probability in the healthy state

  • p_c_2 – the crash probability in the compromised state

  • p_u – the upgrade probability

Returns

P(s_prime | s, a)

static transition_function_game(s: int, s_prime: int, a1: int, a2: int, p_a: float, p_c_1: float) → float[source]

The transition function of the POSG

Parameters

  • s – the state

  • s_prime – the next state

  • a1 – the defender action

  • a2 – the attacker action

  • p_a – the intrusion probability

  • p_c_1 – the crash probability

Returns

P(s_prime | s, a1, a2)

static transition_tensor(states: List[int], actions: List[int], p_a: float, p_c_1: float, p_c_2: float, p_u: float) → List[List[List[float]]][source]

Creates a |A|x|S|x|S| tensor with the transition probabilities of the POMDP

Parameters

  • states – the list of states

  • actions – the list of actions

  • p_a – the intrusion probability

  • p_c_1 – the crash probability in the healthy state

  • p_c_2 – the crash probability in the compromised state

  • p_u – the upgrade probability

Returns

the transition tensor

static transition_tensor_game(states: List[int], defender_actions: List[int], attacker_actions: List[int], p_a: float, p_c_1: float) → List[List[List[List[float]]]][source]

Creates a |A|x|A|x|S|x|S| tensor with the transition probabilities of the POSG

Parameters

  • states – the list of states

  • defender_actions – the list of defender actions

  • attacker_actions – the list of attacker actions

  • p_a – the intrusion probability

  • p_c_1 – the crash probability

Returns

the transition tensor

csle_tolerance.util.intrusion_response_cmdp_util module

class csle_tolerance.util.intrusion_response_cmdp_util.IntrusionResponseCmdpUtil[source]

Bases: object

Class with utility functions for the intrusion-response CMDP

static action_space() → List[int][source]

Gets the action space of the CMDP

Returns

a list with the actions

static constraint_cost_function(s: int, f: int) → float[source]

Constraint cost function of the CMDP

Parameters

  • s – the state

  • f – the tolerance threshold

Returns

the cost

static constraint_cost_tensor(states: List[int], f: int) → List[float][source]

Creates a |S| tensor with the constraint costs of the CMDP

Parameters

  • states – the list of states

  • f – the tolerance threshold

Returns

A tensor with the costs (or rewards)

static cost_function(s: int, negate: bool = False) → float[source]

Cost function of the CMDP

Parameters

  • negate – boolean flag, if true then return negated version of the cost (the reward)

  • s – the state

Returns

the cost

static cost_tensor(states: List[int], negate: bool = False) → List[float][source]

Creates a |S| tensor with the costs (or rewards) of the CMDP

Parameters

  • states – the list of states

  • negate – a boolean flag indicating whether the cost should be negated as a reward or not

Returns

A tensor with the costs (or rewards)

static delta_function(s: int, p_a: float, p_c: float, p_u: float, delta: int, s_max: int) → float[source]

The delta function that gives the probability of the change in the number of healthy nodes

Parameters

  • s – the current state

  • p_a – the intrusion probability

  • p_c – the crash probability

  • p_u – the recovery probability

  • delta – the delta value

  • s_max – the maximum number of nodes

Returns

static state_space(s_max) → List[int][source]

Gets the state space of the CMDP

Returns

a list with the states

static transition_function(s: int, s_prime: int, a: int, p_a: float, p_c: float, p_u: float, s_max: int) → float[source]

The transition function of the CMDP

Parameters

  • s – the state

  • s_prime – the next state

  • a – the action

  • p_a – the intrusion probability

  • p_c – the crash probability

  • p_u – the recovery probability

  • s_max – the maximum state

Returns

P(s_prime | s, a)

static transition_tensor(states: List[int], actions: List[int], p_a: float, p_c: float, p_u: float, s_max: int) → List[List[List[float]]][source]

Creates a |A|x|S|x|S| tensor with the transition probabilities of the CMDP

Parameters

  • states – the list of states

  • actions – the list of actions

  • p_a – the intrusion probability

  • p_c – the failure probability

  • p_u – the upgrade probability

Returns

the transition tensor

csle_tolerance.util.pomdp_solve_parser module

class csle_tolerance.util.pomdp_solve_parser.PomdpSolveParser[source]

Bases: object

Parser of the results of the pomdp-solver by Cassandra et. al. (See http://www.pomdp.org/)

static optimal_avg_value(file_path_to_alpha_vectors: str, initial_belief: float, btr: int) → float[source]

Computes the optimal average value given a set of alpha vectors

Parameters

  • file_path_to_alpha_vectors –

  • initial_belief – the belief to compute the value for

  • btr – the BTR constraint/horizon

Returns

the optimal average value

static parse_alpha_vectors(file_path: str) → List[Tuple[int, List[float]]][source]

Parses alpha vectors from a given file location

Parameters

file_path – path to the file where the alpha vectors are solved

Returns

a list with (action, alphavector)

Module contents