RLPack
 
Loading...
Searching...
No Matches
rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy Class Reference

This class is a PyTorch Model implementing the MLP based Actor-Critic Policy. More...

+ Inheritance diagram for rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy:
+ Collaboration diagram for rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy:

Public Member Functions

def __init__ (self, int sequence_length, List[int] hidden_sizes, Union[int, List[Union[int, List[int]]]] action_space, Union[Activation, List[Activation]] activation=pytorch.nn.ReLU(), float dropout=0.5)
 Initialize ActorCriticMlpPolicy model. More...
 
def forward (self, x)
 The forwards method of the nn.Module. More...
 

Data Fields

 actor_activation
 
 actor_head
 The final head for actor; creates logits for actions. More...
 
 critic_head
 The final head for critic; creates the state value. More...
 
 flatten
 The object to flatten the output fo feature extractor. More...
 
 mlp_feature_extractor
 The feature extractor instance of rlpack.models._mlp_feature_extractor._MlpFeatureExtractor. More...
 
 value_activation
 

Static Private Member Functions

int _process_action_space (Union[int, List[Union[int, List[int]]]] action_space)
 Processes action_space for use by the model. More...
 
List[Activation] _process_activation (Union[Activation, List[Activation]] activation)
 Processes activation for use by the model. More...
 

Private Attributes

 _apply_actor_activation
 FLag indicating whether to apply activation to output of actor head or not. More...
 
 _apply_critic_activation
 FLag indicating whether to apply activation to output of critic head or not. More...
 

Detailed Description

This class is a PyTorch Model implementing the MLP based Actor-Critic Policy.

Constructor & Destructor Documentation

◆ __init__()

def rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.__init__ (   self,
int  sequence_length,
List[int]  hidden_sizes,
Union[int, List[Union[int, List[int]]]]  action_space,
Union[Activation, List[Activation]]   activation = pytorch.nn.ReLU(),
float   dropout = 0.5 
)

Initialize ActorCriticMlpPolicy model.

Parameters
sequence_lengthint: The sequence length of the expected tensor.
hidden_sizesList[int]: The list of hidden sizes for each layer.
action_spaceUnion[int, List[Union[int, List[int]]]]: The action space of the environment. If discrete action set is used, number of actions can be passed. If continuous action space is used, a list must be passed with first element representing the output features from model, second representing the shape of action to be sampled.
activationUnion[Activation, List[Activation]]: The activation function class(es) for the model. Must be an initialized activation object from PyTorch's nn (torch.nn) module. If a list is passed, List must be of length [1, 3], first activation for feature extractor, second for actor head and third for critic head.
dropoutfloat: The dropout to be used in the final Linear (FC) layer.

Member Function Documentation

◆ _process_action_space()

int rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._process_action_space ( Union[int, List[Union[int, List[int]]]]  action_space)
staticprivate

Processes action_space for use by the model.

If checks are passed, returns the output features for actor head.

Parameters
action_spaceUnion[int, List[Union[int, List[int]]]]: The action space of the environment. If discrete action set is used, number of actions can be passed. If continuous action space is used, a list must be passed with first element representing the output features from model, second representing the shape of action to be sampled.
Returns
: Tuple[Union[int, List[Union[int, List[int]]]], Union[Activation, List[Activation]]]: The corrected values for action_space and activation if required.

◆ _process_activation()

List[Activation] rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._process_activation ( Union[Activation, List[Activation]]   activation)
staticprivate

Processes activation for use by the model.

Parameters
activationUnion[Activation, List[Activation]]: The activation function class(es) for the model. Must be an initialized activation object from PyTorch's nn (torch.nn) module. If a list is passed, List must be of length [1, 3], first activation for feature extractor, second for actor head and third for critic head.

◆ forward()

def rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.forward (   self,
  x 
)

The forwards method of the nn.Module.

Parameters
xpytorch.Tensor: The model input.
Returns
pytorch.Tensor: The model output (logits).

Field Documentation

◆ _apply_actor_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._apply_actor_activation
private

FLag indicating whether to apply activation to output of actor head or not.

◆ _apply_critic_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._apply_critic_activation
private

FLag indicating whether to apply activation to output of critic head or not.

◆ actor_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.actor_activation

◆ actor_head

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.actor_head

The final head for actor; creates logits for actions.

◆ critic_head

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.critic_head

The final head for critic; creates the state value.

◆ flatten

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.flatten

The object to flatten the output fo feature extractor.

◆ mlp_feature_extractor

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.mlp_feature_extractor

◆ value_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.value_activation