gym_csle_stopping_game.envs package

Submodules

gym_csle_stopping_game.envs.stopping_game_env module

class gym_csle_stopping_game.envs.stopping_game_env.StoppingGameEnv(config: gym_csle_stopping_game.dao.stopping_game_config.StoppingGameConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the csle-stopping-game

action_space: spaces.Space[ActType]
generate_random_particles(o: int, num_particles: int) → List[int][source]

Generates a random list of state particles from a given observation

Parameters

  • o – the latest observation

  • num_particles – the number of particles to generate

Returns

the list of random particles

get_observation_from_history(history: List[int], pi2: numpy.ndarray[Any, numpy.dtype[Any]], l: int) → List[Any][source]

Utility method to get a hidden observation based on a history

Parameters

  • history – the history to get the observation from

  • pi2 – the attacker stage strategy

  • l – the number of stops remaining

Returns

the observation

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

is_state_terminal(state: Union[gym_csle_stopping_game.dao.stopping_game_state.StoppingGameState, int, Tuple[int, int]]) → bool[source]

Checks whether a given state is terminal or not

Parameters

state – the state

Returns

True if terminal, else false

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

mean(prob_vector)[source]

Utility function for getting the mean of a vector

Parameters

prob_vector – the vector to take the mean of

Returns

the mean

observation_space: spaces.Space[ObsType]
render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[Tuple[numpy.ndarray[Any, numpy.dtype[Any]], numpy.ndarray[Any, numpy.dtype[Any]]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation and info

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Union[gym_csle_stopping_game.dao.stopping_game_state.StoppingGameState, int, Tuple[int, int]]) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(action_profile: Tuple[int, Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int]]) → Tuple[Tuple[numpy.ndarray[Any, numpy.dtype[Any]], numpy.ndarray[Any, numpy.dtype[Any]]], Tuple[int, int], bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

action_profile – the actions to take (both players actions

Returns

(obs, reward, terminated, truncated, info)

weighted_intrusion_prediction_distance(intrusion_start: int, first_stop: int)[source]

Computes the weighted intrusion start time prediction distance (Wang, Hammar, Stadler, 2022)

Parameters

  • intrusion_start – the intrusion start time

  • first_stop – the predicted start time

Returns

the weighted distance

gym_csle_stopping_game.envs.stopping_game_mdp_attacker_env module

class gym_csle_stopping_game.envs.stopping_game_mdp_attacker_env.StoppingGameMdpAttackerEnv(config: gym_csle_stopping_game.dao.stopping_game_attacker_mdp_config.StoppingGameAttackerMdpConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the MDP of the attacker when facing a static defender

action_space: spaces.Space[ActType]
calculate_stage_policy(o: List[Any], a2: int = 0) → numpy.ndarray[Any, numpy.dtype[Any]][source]

Calculates the stage policy of a given model and observation

Parameters

o – the observation

Returns

the stage policy

generate_random_particles(o: int, num_particles: int) → List[int][source]

Generates a random list of state particles from a given observation

Parameters

  • o – the latest observation

  • num_particles – the number of particles to generate

Returns

the list of random particles

get_actions_from_particles(particles: List[int], t: int, observation: int, verbose: bool = False) → List[int][source]

Prunes the set of actiosn based on the current particle set

Parameters

  • particles – the set of particles

  • t – the current time step

  • observation – the latest observation

  • verbose – boolean flag indicating whether logging should be verbose or not

Returns

the list of pruned actions

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

observation_space: spaces.Space[ObsType]
render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(pi2: Union[numpy.ndarray[Any, numpy.dtype[Any]], int, float, numpy.int64, numpy.float64]) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

pi2 – attacker stage policy

Returns

(obs, reward, terminated, truncated, info)

gym_csle_stopping_game.envs.stopping_game_pomdp_defender_env module

class gym_csle_stopping_game.envs.stopping_game_pomdp_defender_env.StoppingGamePomdpDefenderEnv(config: gym_csle_stopping_game.dao.stopping_game_defender_pomdp_config.StoppingGameDefenderPomdpConfig)[source]

Bases: csle_common.dao.simulation_config.base_env.BaseEnv

OpenAI Gym Env for the POMDP of the defender when facing a static attacker

action_space: spaces.Space[ActType]
add_observation_vector(obs_vector: List[Any], obs_id: int) → None[source]

Adds an observation vector to the history

Parameters

  • obs_vector – the observation vector to add

  • obs_id – the id of the observation

Returns

None

generate_random_particles(o: int, num_particles: int) → List[int][source]

Generates a random list of state particles from a given observation

Parameters

  • o – the latest observation

  • num_particles – the number of particles to generate

Returns

the list of random particles

get_actions_from_particles(particles: List[int], t: int, observation: int, verbose: bool = False) → List[int][source]

Prunes the set of actiosn based on the current particle set

Parameters

  • particles – the set of particles

  • t – the current time step

  • observation – the latest observation

  • verbose – boolean flag indicating whether logging should be verbose or not

Returns

the list of pruned actions

get_observation_from_history(history: List[int]) → List[Any][source]

Utiltiy function to get a defender observatin (belief) from a history

Parameters

history – the history to get the observation form

Returns

the observation

get_traces() → List[csle_common.dao.simulation_config.simulation_trace.SimulationTrace][source]
Returns

the list of simulation traces

Checks whether an attacker action in the environment is legal or not

Parameters

attack_action_id – the id of the attacker action

Returns

True or False

Checks whether a defender action in the environment is legal or not

Parameters

defense_action_id – the id of the action

Returns

True or False

is_state_terminal(state: Any) → bool[source]

Utility funciton to check whether a state is terminal or not

Parameters

state – the state

Returns

None

manual_play() → None[source]

An interactive loop to test the environment manually

Returns

None

observation_space: spaces.Space[ObsType]
render(mode: str = 'human')[source]

Renders the environment. Supported rendering modes: (1) human; and (2) rgb_array

Parameters

mode – the rendering mode

Returns

True (if human mode) otherwise an rgb array

reset(seed: Union[None, int] = None, soft: bool = False, options: Optional[Dict[str, Any]] = None) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], Dict[str, Any]][source]

Resets the environment state, this should be called whenever step() returns <done>

Parameters

  • seed – the random seed

  • soft – boolean flag indicating whether it is a soft reset or not

  • options – optional configuration parameters

Returns

initial observation

reset_traces() → None[source]

Resets the list of traces

Returns

None

set_model(model) → None[source]

Sets the model. Useful when using RL frameworks where the stage policy is not easy to extract

Parameters

model – the model

Returns

None

set_state(state: Any) → None[source]

Sets the state. Allows to simulate samples from specific states

Parameters

state – the state

Returns

None

step(a1: int) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int, bool, bool, Dict[str, Any]][source]

Takes a step in the environment by executing the given action

Parameters

a1 – defender action

Returns

(obs, reward, terminated, truncated, info)

step_trace(trace: csle_common.dao.emulation_config.emulation_trace.EmulationTrace, a1: int) → Tuple[numpy.ndarray[Any, numpy.dtype[Any]], int, bool, Dict[str, Any]][source]

Utility method for stopping a pre-recorded trace

Parameters

  • trace – the trace to step

  • a1 – the action to step with

Returns

the result of the step according to the trace

Module contents