gym_csle_apt_game.envs package

Submodules

gym_csle_apt_game.envs.apt_game_env module

class gym_csle_apt_game.envs.apt_game_env.AptGameEnv(config: AptGameConfig)[source]

Bases: BaseEnv

OpenAI Gym Env for the csle-apt-game

get_traces() → List[SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

mean(prob_vector)[source]

Utility function for getting the mean of a vector

Parameters

prob_vector – the vector to take the mean of

Returns

the mean

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[Tuple[ndarray[Any, dtype[Any]], Tuple[ndarray[Any, dtype[Any]], int]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation and info

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Union[AptGameState, int]) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(action_profile: Tuple[int, Tuple[ndarray[Any, dtype[Any]], int]]) → Tuple[Tuple[ndarray[Any, dtype[Any]], Tuple[ndarray[Any, dtype[Any]], int]], Tuple[int, int], bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

action_profile – the actions to take (both players actions

Returns

(obs, reward, terminated, truncated, info)

gym_csle_apt_game.envs.apt_game_mdp_attacker_env module

class gym_csle_apt_game.envs.apt_game_mdp_attacker_env.AptGameMdpAttackerEnv(config: AptGameAttackerMdpConfig)[source]

Bases: BaseEnv

OpenAI Gym Env for the MDP of the attacker when facing a static defender

calculate_stage_policy(o: List[Any], a2: int = 0) → ndarray[Any, dtype[Any]][source]

Calculates the stage policy of a given model and observation

Parameters

o – the observation

Returns

the stage policy

get_traces() → List[SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Optional[int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(pi2: Union[ndarray[Any, dtype[Any]], int, float, int64, float64]) → Tuple[ndarray[Any, dtype[Any]], int, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

pi2 – attacker stage policy

Returns

(obs, reward, terminated, truncated, info)

gym_csle_apt_game.envs.apt_game_pomdp_defender_env module

class gym_csle_apt_game.envs.apt_game_pomdp_defender_env.AptGamePomdpDefenderEnv(config: AptGameDefenderPomdpConfig)[source]

Bases: BaseEnv

OpenAI Gym Env for the MDP of the defender when facing a static attacker

get_traces() → List[SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[ndarray[Any, dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(a1: int) → Tuple[ndarray[Any, dtype[Any]], float, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

a1 – defender action

Returns

(obs, reward, terminated, truncated, info)

Module contents