csle_common.models package

Submodules

csle_common.models.fnn_w_softmax module

A FNN model with Softmax output defined in PyTorch

class csle_common.models.fnn_w_softmax.FNNwithSoftmax(input_dim: int, output_dim: int, hidden_dim: int, num_hidden_layers: int = 2, hidden_activation: str = 'ReLU')[source]

Bases: torch.nn.modules.module.Module

Implements a FNN with parameterizable number of layers, dimensions, and hidden activations.

Sub-classing the torch.nn.Module to be able to use high-level API for creating the custom network

forward(x)[source]

Forward propagation

Parameters

x – input tensor

Returns

Output prediction

get_hidden_activation()[source]

Interprets the hidden activation

Returns

the hidden activation function

training: bool
csle_common.models.fnn_w_softmax.test() → None[source]

A basic test-case to verify that the model can fit some randomly generated data

Returns

None

csle_common.models.ppo_network module

class csle_common.models.ppo_network.PPONetwork(input_dim: int, output_dim_critic: int, output_dim_action: int, num_hidden_layers: int, hidden_layer_dim: int, std_critic: float = 1.0, std_action: float = 0.01)[source]

Bases: torch.nn.modules.module.Module

Class for instantiating a neural network for PPO training

get_action_and_value(x: torch.Tensor, action: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, float, torch.Tensor, torch.Tensor][source]

Gets the action and the value prediction of the network for a given input tensor x

Parameters

  • x – the input tensor

  • action – (optional) the action; if not specified the action is sampled

Returns

the action, log p(action), the entropy of the action, V(x)

get_pi(x: torch.Tensor) → torch.distributions.categorical.Categorical[source]

Utility function for PPG

Parameters

x – the input vector

Returns

the output action distribution

get_pi_value_and_aux_value(x: torch.Tensor) → Tuple[torch.distributions.categorical.Categorical, torch.Tensor, torch.Tensor][source]

Utility function for PPG

Parameters

x – the input vector

Returns

output distribution, critic value, and auxiliary critic value

get_value(x: torch.Tensor) → torch.Tensor[source]

Computes the value function V(x)

Parameters

x – the input observation

Returns

The value

layer_init(layer: torch.nn.modules.linear.Linear, std: float = 1.4142135623730951, bias_const: float = 0.0) → torch.nn.modules.linear.Linear[source]

Initializes a layer in the neural network

Parameters

  • layer – the layer object

  • std – the standard deviation

  • bias_const – the bias constant

Returns

the initialized layer

static load(path: str) → csle_common.models.ppo_network.PPONetwork[source]

Loads the model from a given path

Parameters

path – the path to load the model from

Returns

None

save(path: str) → None[source]

Saves the model to disk

Parameters

path – the path on disk to save the model

Returns

None

training: bool

csle_common.models.q_network module

class csle_common.models.q_network.QNetwork(input_dim: int, num_hidden_layers: int, hidden_layer_dim: int, agent_type: int, action_space_dim: int = 1, n_atoms: int = 101, start: int = - 100, end: int = 100, steps: int = 101)[source]

Bases: torch.nn.modules.module.Module

Class for instantiating a neural network for DQN training

forward(x: torch.Tensor) → torch.Tensor[source]

Makes a forward pass of the neural network with a given input vector x

Parameters

x – the input vector

Returns

the output of the neural network

get_action(x) → Tuple[torch.Tensor, torch.Tensor][source]

Gets the action and the probability distribution over actions

Parameters

x – the input observation(s)

Returns

the action(s)

static load(path: str) → csle_common.models.q_network.QNetwork[source]

Loads the model from a given path

Parameters

path – the path to load the model from

Returns

None

save(path: str) → None[source]

Saves the model to disk

Parameters

path – the path on disk to save the model

Returns

None

training: bool

Module contents