gym_csle_apt_game.envs package

Submodules

gym_csle_apt_game.envs.apt_game_env module

class gym_csle_apt_game.envs.apt_game_env.AptGameEnv(config: gym_csle_apt_game.dao.apt_game_config.AptGameConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the csle-apt-game

action_space: spaces.Space[ActType]

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]

Returns: the list of simulation traces

is_attack_action_legal(attack_action_id: int) → bool[source]

Checks whether an attacker action in the environment is legal or not

Parameters: attack_action_id – the id of the attacker action
Returns: True or False

is_defense_action_legal(defense_action_id: int) → bool[source]

Checks whether a defender action in the environment is legal or not

Parameters: defense_action_id – the id of the action
Returns: True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns: None

mean(prob_vector)[source]

Utility function for getting the mean of a vector

Parameters: prob_vector – the vector to take the mean of
Returns: the mean

observation_space: spaces.Space[ObsType]

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters: mode – the rendering mode
Returns: True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters

Returns

initial observation and info

reset_traces() → None[source]

Resets the list of traces

Returns: None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters: model – the model
Returns: None

set_state(state: Union[gym_csle_apt_game.dao.apt_game_state.AptGameState, int]) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters: state – the state
Returns: None

step(action_profile: Tuple[int, Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int]]) → Tuple[Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int]], Tuple[int, int], bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters: action_profile – the actions to take (both players actions
Returns: (obs, reward, terminated, truncated, info)

gym_csle_apt_game.envs.apt_game_mdp_attacker_env module

class gym_csle_apt_game.envs.apt_game_mdp_attacker_env.AptGameMdpAttackerEnv(config: gym_csle_apt_game.dao.apt_game_attacker_mdp_config.AptGameAttackerMdpConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the MDP of the attacker when facing a static defender

action_space: spaces.Space[ActType]

calculate_stage_policy(o: List[Any], a2: int = 0) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Calculates the stage policy of a given model and observation

Parameters: o – the observation
Returns: the stage policy

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]

Returns: the list of simulation traces

is_attack_action_legal(attack_action_id: int) → bool[source]

Checks whether an attacker action in the environment is legal or not

Parameters: attack_action_id – the id of the attacker action
Returns: True or False

is_defense_action_legal(defense_action_id: int) → bool[source]

Checks whether a defender action in the environment is legal or not

Parameters: defense_action_id – the id of the action
Returns: True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns: None

observation_space: spaces.Space[ObsType]

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters: mode – the rendering mode
Returns: True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns: None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters: model – the model
Returns: None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters: state – the state
Returns: None

step(pi2: Union[numpy.ndarray[Any, numpy.dtype[Any]], int, float, numpy.int64, numpy.float64]) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters: pi2 – attacker stage policy
Returns: (obs, reward, terminated, truncated, info)

gym_csle_apt_game.envs.apt_game_pomdp_defender_env module

class gym_csle_apt_game.envs.apt_game_pomdp_defender_env.AptGamePomdpDefenderEnv(config: gym_csle_apt_game.dao.apt_game_defender_pomdp_config.AptGameDefenderPomdpConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the MDP of the defender when facing a static attacker

action_space: spaces.Space[ActType]

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]

Returns: the list of simulation traces

is_attack_action_legal(attack_action_id: int) → bool[source]

Checks whether an attacker action in the environment is legal or not

Parameters: attack_action_id – the id of the attacker action
Returns: True or False

is_defense_action_legal(defense_action_id: int) → bool[source]

Checks whether a defender action in the environment is legal or not

Parameters: defense_action_id – the id of the action
Returns: True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns: None

observation_space: spaces.Space[ObsType]

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters: mode – the rendering mode
Returns: True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

seed – the random seed
soft – boolean flag indicating whether it is a soft reset or not
options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns: None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters: model – the model
Returns: None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters: state – the state
Returns: None

step(a1: int) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], float, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters: a1 – defender action
Returns: (obs, reward, terminated, truncated, info)

gym_csle_apt_game.envs package

Submodules

gym_csle_apt_game.envs.apt_game_env module

gym_csle_apt_game.envs.apt_game_mdp_attacker_env module

gym_csle_apt_game.envs.apt_game_pomdp_defender_env module

Module contents