gym_csle_intrusion_response_game.envs package
Submodules
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_attacker module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_attacker.IntrusionResponseGameLocalPOMDPAttackerEnv(config: gym_csle_intrusion_response_game.dao.intrusion_response_game_local_pomdp_attacker_config.IntrusionResponseGameLocalPOMDPAttackerConfig)[source]
Bases:
csle_common.dao.simulation_config.base_env.BaseEnvOpenAI Gym Env for the POMDP of the attacker when facing a static defender.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the defender strategy is fixed)
- action_space: spaces.Space[ActType]
- get_attack_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of the “always attacker” attacker strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of a random attacker strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_traces() List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
- Returns
the list of simulation traces
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- observation_space: spaces.Space[ObsType]
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
- set_state(state: gym_csle_intrusion_response_game.dao.intrusion_response_game_state_local.IntrusionResponseGameStateLocal) None[source]
Sets the state. Allows to simulate samples from specific states
- Parameters
state – the state
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_pomdp_defender.IntrusionResponseGameLocalPOMDPDefenderEnv(config: gym_csle_intrusion_response_game.dao.intrusion_response_game_local_pomdp_defender_config.IntrusionResponseGameLocalPOMDPDefenderConfig)[source]
Bases:
csle_common.dao.simulation_config.base_env.BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- action_space: spaces.Space[ActType]
- get_local_stopping_mdp_reward_tensor() numpy.ndarray[Any, numpy.dtype[Any]][source]
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_mdp_transition_tensor() numpy.ndarray[Any, numpy.dtype[Any]][source]
- Returns
the local MDP transition tensor of the optimal stopping formulation
- get_local_stopping_pomdp_obs_tensor(a1: int, zone: int) numpy.ndarray[Any, numpy.dtype[Any]][source]
- Parameters
a1 – the defender action
zone – the zone
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_pomdp_reward_tensor(a1: int, zone: int) numpy.ndarray[Any, numpy.dtype[Any]][source]
- Parameters
a1 – the defender action
zone – the zone
- Returns
the local MDP reward tensor of the optimal stopping formulation
- get_local_stopping_pomdp_transition_tensor(a1: int) numpy.ndarray[Any, numpy.dtype[Any]][source]
- Parameters
a1 – the defender action
- Returns
the local POMDP transition tensor of the optimal stopping formulation
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy function for estimating the average return of a random defender strategy
- Parameters
samples – the number of samples to use for estimation
- Returns
the estimated return
- get_traces() List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
- Returns
the list of simulation traces
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- observation_space: spaces.Space[ObsType]
- pomdp_solver_file() str[source]
Gets the POMDP environment specification based on the format at http://www.pomdp.org/code/index.html, for the defender’s local problem against a static attacker
- Returns
the file content string
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
- set_state(state: gym_csle_intrusion_response_game.dao.intrusion_response_game_state_local.IntrusionResponseGameStateLocal) None[source]
Sets the state. Allows to simulate samples from specific states
- Parameters
state – the state
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_local_stopping_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_local_stopping_pomdp_defender.IntrusionResponseGameLocalStoppingPOMDPDefenderEnv(config: gym_csle_intrusion_response_game.dao.intrusion_response_game_local_pomdp_defender_config.IntrusionResponseGameLocalPOMDPDefenderConfig)[source]
Bases:
csle_common.dao.simulation_config.base_env.BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker.
(A Partially observed Dynkin game where the attacker strategy is fixed)
- action_space: spaces.Space[ActType]
- get_traces() List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
- Returns
the list of simulation traces
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- observation_space: spaces.Space[ObsType]
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_attacker module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_attacker.IntrusionResponseGameWorkflowPOMDPAttackerEnv(config: gym_csle_intrusion_response_game.dao.workflow_intrusion_response_pomdp_attacker_config.WorkflowIntrusionResponsePOMDPAttackerConfig)[source]
Bases:
csle_common.dao.simulation_config.base_env.BaseEnvOpenAI Gym Env for the POMDP of the attacker when facing a static defender in the workflow game.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- action_space: spaces.Space[ActType]
- get_traces() List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
- Returns
the list of simulation traces
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- observation_space: spaces.Space[ObsType]
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None
gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_defender module
- class gym_csle_intrusion_response_game.envs.intrusion_response_game_workflow_pomdp_defender.IntrusionResponseGameWorkflowPOMDPDefenderEnv(config: gym_csle_intrusion_response_game.dao.workflow_intrusion_response_pomdp_defender_config.WorkflowIntrusionResponsePOMDPDefenderConfig)[source]
Bases:
csle_common.dao.simulation_config.base_env.BaseEnvOpenAI Gym Env for the POMDP of the defender when facing a static attacker in the workflow game.
(A PO-POSG, i.e a partially observed stochastic game with public observations) where the attacker strategy is fixed)
- action_space: spaces.Space[ActType]
- get_random_baseline_return(samples: int = 100) float[source]
Utiltiy method for getting the average return of a random strategy
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- get_traces() List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
- Returns
the list of simulation traces
- get_upper_bound_return(samples: int = 100) float[source]
Utiltiy method for getting an upper bound on the average return
- Parameters
samples – the number of sample returns to average
- Returns
the estimated upper bound
- is_attack_action_legal(attack_action_id: int) bool[source]
Checks whether an attacker action in the environment is legal or not
- Parameters
attack_action_id – the id of the attacker action
- Returns
True or False
- is_defense_action_legal(defense_action_id: int) bool[source]
Checks whether a defender action in the environment is legal or not
- Parameters
defense_action_id – the id of the action
- Returns
True or False
- observation_space: spaces.Space[ObsType]
- reachable(node: int) bool[source]
Checks if a node is reachable from the gw
- Parameters
node – the node to check
- Returns
True if reachable otherwise False
- render(mode: str = 'human')[source]
Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array
- Parameters
mode – the rendering mode
- Returns
True (if human mode) otherwise an rgb array
- reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]
Resets the environment state, this should be called whenever step() returns <done>
- Parameters
seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters
- Returns
initial observation
- set_model(model) None[source]
Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract
- Parameters
model – the model
- Returns
None