gym_csle_intrusion_response_game.util package

Submodules

gym_csle_intrusion_response_game.util.intrusion_response_game_util module

class gym_csle_intrusion_response_game.util.intrusion_response_game_util.IntrusionResponseGameUtil[source]

Bases: object

Class with utility functions for the intrusion response game environment

static are_local_attack_states_equal(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]], s_prime: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if two local attack states are equal

Parameters

  • s – the first local state to check

  • s_prime – the second local state to check

Returns

True if equal, otherwise False

static are_local_defense_states_equal(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]], s_prime: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if two local defense states are equal

Parameters

  • s – the first local state to check

  • s_prime – the second local state to check

Returns

True if equal, otherwise False

static are_local_states_equal(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]], s_prime: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if two local states are equal

Parameters

  • s – the first local state to check

  • s_prime – the second local state to check

Returns

True if equal, otherwise False

static bayes_filter_attacker_belief(s_a_prime: int, o: int, a2: int, a_b: numpy.ndarray[Any, numpy.dtype[Any]], pi1: numpy.ndarray[Any, numpy.dtype[Any]], config: gym_csle_intrusion_response_game.dao.local_intrusion_response_game_config.LocalIntrusionResponseGameConfig, s_d_prime: int, s_a: int) → float[source]

A Bayesian filter to compute the belief of player 2 of being in s_prime when observing o after taking action a in belief b given that the opponent follows strategy pi1

Parameters

  • s_a_prime – the current attacker state

  • s_a_prime – the previous attacker state

  • o – the observation

  • a2 – the action of player 2

  • a_b – the current attacker belief point

  • pi1 – the policy of player 2

  • s_d_prime – the defender state to compute the belief of

Returns

b_prime(s_prime_d)

static bayes_filter_defender_belief(s_a_prime: int, o: int, a1: int, d_b: numpy.ndarray[Any, numpy.dtype[Any]], pi2: numpy.ndarray[Any, numpy.dtype[Any]], config: gym_csle_intrusion_response_game.dao.local_intrusion_response_game_config.LocalIntrusionResponseGameConfig, s_d_prime: int, s_d: int) → float[source]

A Bayesian filter to compute the belief of player 1 of being in s_prime when observing o after taking action a in belief b given that the opponent follows strategy pi2

Parameters

  • s_a_prime – the attacker state to compute the belief of

  • o – the observation

  • a1 – the action of player 1

  • d_b – the current defender belief point

  • pi2 – the policy of player 2

  • s_d_prime – the defender state

  • s_d – the previous defender state

Returns

b_prime(s_prime)

static constant_defender_action_costs(A1: numpy.ndarray[Any, numpy.dtype[numpy.int64]], constant_cost: float) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Returns a vector with the local defender action costs where each action has the same constant cost

Parameters

  • A1 – a vector with the actions of the defender in the local game

  • constant_cost – the constant action cost

Returns

a vector with the action costs

static constant_zone_detection_probabilities(zones: numpy.ndarray[Any, numpy.dtype[numpy.int64]], constant_detection_prob: float) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Returns a vector with the zone detection probabilities where each zone as the same uniform detection probability

Parameters

  • zones – the vector with zones

  • constant_detection_prob – the constant detection probability

Returns

the vector with zone detection probabilities

static constant_zone_utilities(zones: numpy.ndarray[Any, numpy.dtype[numpy.int64]], constant_utility: float) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Returns a vector with the zone utilities where each zone has the same constant utility

Parameters

  • zones – the vector with zones

  • constant_utility – the constant utility of a zone

Returns

the vector with zone utilities

static get_local_defender_pomdp_solver_file(S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], O: numpy.ndarray[Any, numpy.dtype[Any]], R: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]], Z: numpy.ndarray[Any, numpy.dtype[Any]], static_attacker_strategy: csle_common.dao.training.policy.Policy, s_1_idx: int, discount_factor: float = 0.99) → str[source]

Gets the POMDP environment specification based on the format at http://www.pomdp.org/code/index.html, for the defender’s local problem against a static attacker

Parameters

  • S – the state spaec

  • A1 – the defender’s local action space

  • A2 – the attacker’s local action space

  • O – the observation space

  • R – the reward tensor

  • T – the transition tensor

  • static_attacker_strategy – the static attacker opponent strategy

  • s_1_idx – the initial state index

  • discount_factor – the discount factor

Returns

the file content string

static is_local_state_compromised(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if a local state has been compromised

Parameters

s – the local state to check

Returns

True if compromised, otherwise False

static is_local_state_healthy(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if a local state is healthy and not discovered nor compromised

Parameters

s – the local state to check

Returns

True if compromised, otherwise False

static is_local_state_in_zone(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]], zone: int) → bool[source]

Utility function for checking if a local state is in a given zone or not

Parameters

  • s – the local state to check

  • zone – the zone to check

Returns

True if the state is in the given zone, false otherwise

static is_local_state_recon(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if a local state has been discovered by the attacker

Parameters

s – the local state to check

Returns

True if compromised, otherwise False

static is_local_state_shutdown_or_redirect(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if a local node is in shutdown or redirect state

Parameters

  • s – the local state to check

  • zone – the zone to check

Returns

True if the node is in shutdown or redirect state, otherwise fasle

static is_local_state_terminal(s: numpy.ndarray[Any, numpy.dtype[numpy.int64]]) → bool[source]

Utility function for checking if a local state is terminal or not

Parameters

s – the local state to check

Returns

True if terminal, otherwise False

static local_attack_success_probabilities_uniform(p: float, A2: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Returns a vector with the success probabilities of the attacker actions for the local version of the game where the attacks have the same constant success probability

Parameters

A2 – the local action space of the attacker

Returns

a vector with the attack success probabilities

static local_attacker_actions() → numpy.ndarray[Any, numpy.dtype[numpy.int64]][source]

Gets the attacker’s action space in the local version of the game

Returns

a vector with the actions of the defender

static local_attacker_state_space() → numpy.ndarray[Any, numpy.dtype[numpy.int64]][source]

Gets the attacker state space of the local version of the game

Returns

the local attacker state space

static local_defender_actions(number_of_zones: int) → numpy.ndarray[Any, numpy.dtype[numpy.int64]][source]

Gets the defender’s action space in the local version of the game

Parameters

number_of_zones – the number of zones in the game

Returns

a vector with the actions of the defender

static local_defender_state_space(number_of_zones: int) → numpy.ndarray[Any, numpy.dtype[numpy.int64]][source]

Gets the defender state space of the local version of the game

Parameters

number_of_zones – the number of zones in the network

Returns

the local defender state space

static local_defender_utility_function(s: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, eta: float, reachable: bool, initial_zone: int, beta: float, C_D: numpy.ndarray[Any, numpy.dtype[Any]], Z_U: numpy.ndarray[Any, numpy.dtype[Any]], topology_cost: float = 0) → Any[source]

The local utility function of the defender

Parameters

  • s – the current state of the local game

  • a1 – the defender action

  • eta – a scaling parameter to balance QoS and security

  • reachable – a boolean flag indicating whether the node is reachable from the public gateway or not

  • initial_zone – the initial zone of the node in the local game

  • beta – a scaling parameter indicating the importance of the workflow of the local game

  • C_D – the vector with the costs of the local defender actions

  • Z_U – the utilities of the zones in the network

  • topology_cost – extra topology cost

Returns

the utility of the defender

static local_initial_attacker_belief(S_D: numpy.ndarray[Any, numpy.dtype[Any]], initial_zone) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Gets the initial attacker belief for a local version of the game

Parameters

  • S_D – the defender’s state space of the local game

  • initial_zone – the initial zone of the node

Returns

the initial defender belief

static local_initial_defender_belief(S_A: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Gets the initial defender belief for a local version of the game

Parameters

S_A – the attacker’s state space of the local game

Returns

the initial defender belief

static local_initial_state(initial_zone: int, S: numpy.ndarray[Any, numpy.dtype[Any]]) → Any[source]

Gets the initial state for a local version of the game

Parameters

  • initial_zone – the initial zone of the local node

  • S – the state space of the local game

Returns

the initial state belief

static local_initial_state_distribution(initial_state_idx, S: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[numpy.float64]][source]

Gets the initial state distribution

Parameters

  • initial_state_idx – the initial state index

  • S – the state space

Returns

the initial state distribution

static local_initial_state_idx(initial_zone: int, S: numpy.ndarray[Any, numpy.dtype[Any]]) → int[source]

Gets the initial state for a local version of the game

Parameters

  • initial_zone – the initial zone of the local node

  • S – the state space of the local game

Returns

the initial state belief

static local_intrusion_cost(a1: int, D_C: numpy.ndarray[Any, numpy.dtype[Any]], reachable: bool, s: numpy.ndarray[Any, numpy.dtype[Any]], Z_U: numpy.ndarray[Any, numpy.dtype[Any]]) → Any[source]

The defender’s local cost function for intrusions

Parameters

  • a1 – the defender action

  • D_C – the vector with the costs of the defender actions

  • reachable – a boolean flag indicating whether the node is reachable

  • s – the current state

  • Z_U – the vector with zone utilities

Returns

the intrusion cost

static local_observation_space(X_max: int) → numpy.ndarray[Any, numpy.dtype[numpy.int64]][source]

Gets the observation space of the local version of the game

Parameters

X_max – the maximum observation

Returns

a vector with all possible observations

static local_observation_tensor_betabinom(S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], O: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the observation tensor of the local game where the observations follow beta-binomial distributions

Parameters

  • S – the local state space of the game

  • A1 – the local action space of the defender

  • A2 – the local action space of the attacker

  • O – the local observation space

Returns

a (A1)(A2)(S)(O) tensor

static local_reward_tensor(eta: float, C_D: numpy.ndarray[Any, numpy.dtype[Any]], reachable: bool, Z_U: numpy.ndarray[Any, numpy.dtype[Any]], initial_zone: int, beta: float, S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], topology_cost: float = 0.0) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the defender’s utility tensor of the local version of the game

Parameters

  • eta – a scaling parameter for the local rewards

  • C_D – the vector with the costs of the local defender actions

  • reachable – a boolean flag indicating whether the node of the local game is reachable or not

  • Z_U – the vector with utilities of the different zones in the network

  • initial_zone – the initial zone of the node

  • beta – a scaling parameter for the workflow utility

  • S – the local state space of the game

  • A1 – the local action space of the defender

  • A2 – the local action space of the attacker

  • topology_cost – extra topology costs

Returns

a (A1)(A2)(S) tensor giving the rewards of the state transitions in the local game

static local_state_space(number_of_zones: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the state space of the local version of the game

Parameters

number_of_zones – the number of zones in the network

Returns

the state space

static local_stopping_mdp_reward_tensor(S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], R: numpy.ndarray[Any, numpy.dtype[Any]], S_D: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the local stopping MDP reward tensor for the attacker

Parameters

  • S – the state space of the local game

  • A1 – the action space of the defender in the local game

  • A2 – the action space of the attacker in the local game

  • R – the reward tensor of the local game

  • S_D – the state space of the defender in the local game

Returns

the reward tensor of the attacker in the stopping MDP

static local_stopping_mdp_transition_tensor(S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]], S_D: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the transition tensor for the local MDP of the stopping decomposition in the temporal domain

Parameters

  • S – the full state space of the local problem

  • A1 – the defender’s action space in the local problem

  • A2 – the attacker’s action space in the local problem

  • T – the full transition tensor of the local problem

  • S_D – the defender’s state spce

Returns

the transition tensor for the local MDP of the stopping formulation

static local_stopping_pomdp_observation_tensor(S: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], S_A: numpy.ndarray[Any, numpy.dtype[Any]], Z: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, zone: int, O: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the local observation tensor for a stopping POMDP

Parameters

  • S – the state space of the local game

  • A2 – the action space of the attacker

  • S_A – the state space of the attacker

  • Z – the observation tensor of the game

  • a1 – the defender stop action

  • zone – the zone of the local game

  • O – the observation space

Returns

the observation tensor

static local_stopping_pomdp_reward_tensor(S: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], R: numpy.ndarray[Any, numpy.dtype[Any]], S_A: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, zone: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the local reward tensor of the stopping POMDP

Parameters

  • S – the state space of the local game

  • A2 – the action space of the attacker

  • R – the reward tensor of the local game

  • S_A – the state space of the attacker in the local game

  • a1 – the stopping action of the defender

  • zone – the zone of the local game

Returns

the reward tensor

static local_stopping_pomdp_transition_tensor(S: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]], S_A: numpy.ndarray[Any, numpy.dtype[Any]], a1: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the transition tensor for the local POMDP of the stopping decomposition in the temporal domain

Parameters

  • S – the full state space of the local problem

  • A1 – the defender’s action space in the local problem

  • A2 – the attacker’s action space in the local problem

  • T – the full transition tensor of the local problem

  • S_D – the defender’s state spce

Returns

the transition tensor for the local MDP of the stopping formulation

static local_transition_probability(s: numpy.ndarray[Any, numpy.dtype[Any]], s_prime: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, a2: int, Z_D_P: numpy.ndarray[Any, numpy.dtype[Any]], A_P: numpy.ndarray[Any, numpy.dtype[Any]]) → float[source]

Gets the probability of a local state transition

Parameters

  • s – the current state

  • s_prime – the next state

  • a1 – the defender action

  • a2 – the attacker action

  • Z_D_P – the zone detection probabilities

  • A_P – the attack success probabilities

Returns

the transition probabilitiy

static local_transition_tensor(Z_D: numpy.ndarray[Any, numpy.dtype[Any]], A_P: numpy.ndarray[Any, numpy.dtype[Any]], S: numpy.ndarray[Any, numpy.dtype[Any]], A1: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]]) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the transition tensor of the local game

Parameters

  • Z_D – the zone detection probabilities

  • A_P – the attack success probabilities

  • S – the local state space

  • A1 – the local action space of the defender

  • A2 – the local action space of the attacker

Returns

a (A1)(A2)(S)(S) tensor

static local_workflow_utility(beta: float, reachable: bool, s: numpy.ndarray[Any, numpy.dtype[Any]], initial_zone: int) → float[source]

The local utility function for workflow QoS

Parameters

  • beta – a scaling parameter indicating how important the workflow is

  • reachable – a boolean flag indicating whether the node is reachable from the public gateway or not

  • s – the local state vector

  • initial_zone – the initial zone of the node in the local version of the game

Returns

the workflow utility

static next_local_attacker_belief(o: int, a1: int, a_b: numpy.ndarray[Any, numpy.dtype[Any]], pi1: numpy.ndarray[Any, numpy.dtype[Any]], config: gym_csle_intrusion_response_game.dao.local_intrusion_response_game_config.LocalIntrusionResponseGameConfig, a2: int, s_d: int, s_a_prime: int, s_a: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Computes the next local belief of the attacker using a Bayesian filter

Parameters

  • o – the latest observation

  • a1 – the latest action of player 1 (for debugging, should be consistent with pi1)

  • a_b – the current attacker belief

  • pi1 – the policy of player 1

  • config – the game config

  • a2 – the attacker action

  • s_d – the true current defender state (for debugging)

  • s_a_prime – the new attacker state

  • s_a – the previous attacker state

Returns

the new belief

static next_local_defender_belief(o: int, a1: int, d_b: numpy.ndarray[Any, numpy.dtype[Any]], pi2: numpy.ndarray[Any, numpy.dtype[Any]], config: gym_csle_intrusion_response_game.dao.local_intrusion_response_game_config.LocalIntrusionResponseGameConfig, a2: int, s_a: int, s_d_prime: int, s_d: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Computes the next local belief of the defender using a Bayesian filter

Parameters

  • o – the latest observation

  • a1 – the latest action of player 1

  • d_b – the current defender belief

  • pi2 – the policy of player 2

  • config – the game config

  • a2 – the attacker action (for debugging, should be consistent with pi2)

  • s_a – the true current attacker state (for debugging)

  • s_d – the previous defender state

  • s_d_prime – the new defender state

Returns

the new belief

static next_stopping_belief(o: int, a1: int, b: numpy.ndarray[Any, numpy.dtype[Any]], pi2: numpy.ndarray[Any, numpy.dtype[Any]], S: numpy.ndarray[Any, numpy.dtype[Any]], Z: numpy.ndarray[Any, numpy.dtype[Any]], O: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], a2: int = 0, s: int = 0) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Computes the next belief using a Bayesian filter

Parameters

  • o – the latest observation

  • a1 – the latest action of player 1

  • b – the current belief

  • pi2 – the policy of player 2

  • config – the game config

  • a2 – the attacker action (for debugging, should be consistent with pi2)

  • s – the true state (for debugging)

Returns

the new belief

static sample_attacker_action(pi2: numpy.ndarray[Any, numpy.dtype[Any]], s: int) → int[source]

Samples the attacker action

Parameters

  • pi2 – the attacker action

  • s – the game state

Returns

a2 (the attacker action

static sample_defender_action(pi1: numpy.ndarray[Any, numpy.dtype[Any]], s: int) → int[source]

Samples the defender action

Parameters

  • pi1 – the attacker action

  • s – the game state

Returns

a1 (the defender action

static sample_next_observation(Z: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, a2: int, s_prime_idx: int, O: numpy.ndarray[Any, numpy.dtype[Any]]) → int[source]

Samples the next observation

static sample_next_state(T: numpy.ndarray[Any, numpy.dtype[Any]], s_idx: int, a1: int, a2: int, S: numpy.ndarray[Any, numpy.dtype[Any]]) → int[source]

Samples the next state

Parameters

  • T – the transition operator

  • s_idx – the current state index

  • a1 – the defender action

  • a2 – the attacker action

  • S – the state space

  • l – the number of stops remaining

Returns

s’

static stopping_bayes_filter(s_prime: int, o: int, a1: int, b: numpy.ndarray[Any, numpy.dtype[Any]], pi2: numpy.ndarray[Any, numpy.dtype[Any]], S: numpy.ndarray[Any, numpy.dtype[Any]], Z: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]], A2: numpy.ndarray[Any, numpy.dtype[Any]], O: numpy.ndarray[Any, numpy.dtype[Any]]) → float[source]

A Bayesian filter to compute the belief of player 1 of being in s_prime when observing o after taking action a in belief b given that the opponent follows strategy pi2

Parameters

  • s_prime – the state to compute the belief of

  • o – the observation

  • a1 – the action of player 1

  • b – the current belief point

  • pi2 – the policy of player 2

  • l – stops remaining

Returns

b_prime(s_prime)

static stopping_p_o_given_b_a1_a2(o: int, b: numpy.ndarray[Any, numpy.dtype[Any]], a1: int, a2: int, S: numpy.ndarray[Any, numpy.dtype[Any]], Z: numpy.ndarray[Any, numpy.dtype[Any]], T: numpy.ndarray[Any, numpy.dtype[Any]]) → float[source]

Computes P[o|a,b]

Parameters

  • o – the observation

  • b – the belief point

  • a1 – the action of player 1

  • a2 – the action of player 2

  • config – the game config

Returns

the probability of observing o when taking action a in belief point b

static zones(num_zones: int) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Gets the vector with the network zones

Parameters

num_zones – the number of zones

Returns

a vector with the zones

Module contents