gym_csle_intrusion_response_game.envs package
Submodules
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_attacker module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_attacker.IntrusionResponseGameLocalPOMDPAttackerEnv(config: IntrusionResponseGameLocalPOMDPAttackerConfig)[source]
Bases:
BaseEnvOpenAI Gym Env for the POMDP of the attacker when facing a static defender.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the defender strategy is fixed)
- get_attack_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of the “always attacker” attacker strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of a random attacker strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
- set_state(state: IntrusionResponseGameStateLocal) None[source]
Sets the state. Allows to simulate samples from specific states
- Parameters
state – the state
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_defender.IntrusionResponseGameLocalPOMDPDefenderEnv(config: IntrusionResponseGameLocalPOMDPDefenderConfig)[source]
Bases:
BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- get_local_stopping_mdp_reward_tensor() ndarray[Any, dtype[Any]][source]
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_mdp_transition_tensor() ndarray[Any, dtype[Any]][source]
- Returns
the local MDP transition tensor of the optimal stopping formulation
- get_local_stopping_pomdp_obs_tensor(a1: int, zone: int) ndarray[Any, dtype[Any]][source]
- Parameters
a1 – the defender action
zone – the zone
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_pomdp_reward_tensor(a1: int, zone: int) ndarray[Any, dtype[Any]][source]
- Parameters
a1 – the defender action
zone – the zone
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_pomdp_transition_tensor(a1: int) ndarray[Any, dtype[Any]][source]
- Parameters
a1 – the defender action
- Returns
the local POMDP transition tensor of the optimal stopping formulation
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of a random defender strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- pomdp_solver_file() str[source]
Gets the POMDP environment specification based on the format at http://www.pomdp.org/code/index.html, for the defender’s local problem against a static attacker
- Returns
the file content string
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
- set_state(state: IntrusionResponseGameStateLocal) None[source]
Sets the state. Allows to simulate samples from specific states
- Parameters
state – the state
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_stopping_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_stopping_pomdp_defender.IntrusionResponseGameLocalStoppingPOMDPDefenderEnv(config: IntrusionResponseGameLocalPOMDPDefenderConfig)[source]
Bases:
BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker.
(A Partially observed Dynkin game where the attacker strategy is fixed)
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_attacker module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_attacker.IntrusionResponseGameWorkflowPOMDPAttackerEnv(config: WorkflowIntrusionResponsePOMDPAttackerConfig)[source]
Bases:
BaseEnvOpenAI Gym Env for the POMDP of the attacker when facing a static defender in the workflow game.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_defender.IntrusionResponseGameWorkflowPOMDPDefenderEnv(config: WorkflowIntrusionResponsePOMDPDefenderConfig)[source]
Bases:
BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker in the workflow game.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy method for getting the average return of a random strategy
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- reachable(node: int) bool[source]
Checks if a node is reachable from the gw
- Parameters
node – the node to check
- Returns
True if reachable otherwise False
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None