RLPack
 
Loading...
Searching...
No Matches
rlpack.environments.environments.Environments Class Reference

This class is a generic class to train any agent in any environment. More...

Public Member Functions

def __init__ (self, Agent agent, Dict[str, Any] config, Optional[Callable[[np.ndarray, Tuple[int,...]], np.ndarray]] reshape_func=None)
 
None evaluate_agent (self)
 Method to evaluate a trained model. More...
 
bool is_eval (self)
 Check if environment is to be run in evaluation mode or not. More...
 
bool is_train (self)
 Check if environment is to be run in training mode or not. More...
 
None train_agent (self, bool render=False, bool load=False, bool plot=False, int verbose=-1, bool distributed_mode=False)
 Method to train the agent in the specified environment. More...
 

Data Fields

 agent
 The input RLPack agent to be run. More...
 
 config
 The input config for setup. More...
 
 env
 The gym environment on which the agent will run. More...
 
 new_shape
 The new shape requested in config to be used with reshape_func. More...
 
 reshape_func
 The input reshape function for states. More...
 

Private Member Functions

None _generate_plot (self, Dict[int, List[float]] rewards_collector)
 Generates plot with matplotlib for Episodes vs. More...
 
None _log (self, int ep, float mean_reward, bool distributed_mode, int verbose)
 Helper method to perform logging operations (both on console and cache). More...
 
None _remove_log_file (self)
 Removes the log.txt file if it is present in the set save_path. More...
 
None _write_log_file (self, List[str] log)
 Writes the logging messages from input to and saves it to set save_path as log.txt. More...
 

Static Private Member Functions

Union[None, float] _list_mean (List[Union[float, int]] x)
 This function computes the mean of the input list. More...
 
np.ndarray _reshape_func_default (np.ndarray x, Optional[Tuple[int,...]] shape=None)
 This is the default reshape function. More...
 

Detailed Description

This class is a generic class to train any agent in any environment.

Constructor & Destructor Documentation

◆ __init__()

def rlpack.environments.environments.Environments.__init__ (   self,
Agent  agent,
Dict[str, Any]  config,
Optional[ Callable[[np.ndarray, Tuple[int, ...]], np.ndarray] ]   reshape_func = None 
)
Parameters
agentAgent: The agent to be trained and/or evaluated in the environment specified in config.
configDict[str, Any]: The configuration setting for experiment.
reshape_funcOptional[Callable[[np.ndarray, Tuple[int, ...]], np.ndarray]]: The function to reshape the input states. Default: None. Default behavior is to not do any reshaping.

Member Function Documentation

◆ _generate_plot()

None rlpack.environments.environments.Environments._generate_plot (   self,
Dict[int, List[float]]  rewards_collector 
)
private

Generates plot with matplotlib for Episodes vs.

rewards.

Parameters
rewards_collectorDict[int, List[float]]: Dict of lists of rewards collected in each episode. Each episode is present as a key.

◆ _list_mean()

Union[None, float] rlpack.environments.environments.Environments._list_mean ( List[Union[float, int]]  x)
staticprivate

This function computes the mean of the input list.

Parameters
xList[Union[float, int]]: The list for which mean is to be computed
Returns
Union[None, float]: The mean value.

◆ _log()

None rlpack.environments.environments.Environments._log (   self,
int  ep,
float  mean_reward,
bool  distributed_mode,
int   verbose 
)
private

Helper method to perform logging operations (both on console and cache).

Parameters
epint: The episode which is currently being logged.
mean_rewardfloat: The mean reward acquired between two successive calls of this method.
distributed_modebool: Indicates if the environment is being run in distributed mode.
verbosebool: Indicates the verbose level. Refer notes for more details. This also refers to values logged on screen. If you want to disable the logging on screen, set logging level to WARNING. Default: -1

◆ _remove_log_file()

None rlpack.environments.environments.Environments._remove_log_file (   self)
private

Removes the log.txt file if it is present in the set save_path.

◆ _reshape_func_default()

np.ndarray rlpack.environments.environments.Environments._reshape_func_default ( np.ndarray  x,
Optional[Tuple[int, ...]]   shape = None 
)
staticprivate

This is the default reshape function.

If new_shape has been set in config, input states are reshaped to new shapes, else returns the input as it is. Default behavior is not perform any reshaping.

Parameters
xnp.ndarray: The input numpy array to reshape.
shapeOptional[Tuple[int, ...]]: The new shape to which we want states to be reshaped. Default: None.
Returns
np.ndarray: The reshaped (or unchanged) array.

◆ _write_log_file()

None rlpack.environments.environments.Environments._write_log_file (   self,
List[str]  log 
)
private

Writes the logging messages from input to and saves it to set save_path as log.txt.

This method open files in append mode.

Parameters
logList[str]: The logging messages to write

◆ evaluate_agent()

None rlpack.environments.environments.Environments.evaluate_agent (   self)

Method to evaluate a trained model.

This method renders the environment and loads the model from save_path. config must have set mode='eval' to run evaluation.

◆ is_eval()

bool rlpack.environments.environments.Environments.is_eval (   self)

Check if environment is to be run in evaluation mode or not.

Returns
bool: True if evaluation mode is set.

◆ is_train()

bool rlpack.environments.environments.Environments.is_train (   self)

Check if environment is to be run in training mode or not.

Returns
bool: True if training mode is set.

◆ train_agent()

None rlpack.environments.environments.Environments.train_agent (   self,
bool   render = False,
bool   load = False,
bool   plot = False,
int   verbose = -1,
bool   distributed_mode = False 
)

Method to train the agent in the specified environment.

Parameters
renderbool: Indicates if we wish to render the environment (in animation). Default: False.
loadbool: Indicates weather to load a previously saved model or train a new one. If set true, config must be save_path or set or environment variable SAVE_PATH must be set.
plotbool: Indicates if to plot the training progress. If set True, rewards and episodes are recorded and plot is saved in save_path.
verbosebool: Indicates the verbose level. Refer notes for more details. This also refers to values logged on screen. If you want to disable the logging on screen, set logging level to WARNING. Default: -1
distributed_modeIndicates if the environment is being run in distributed mode. Rewards are logged on console every reward_logging_frequency set in the console.

Notes

Verbose levels:

  • -1: Log everything.
  • 0: Log episode wise rewards.
  • 1: Log model level losses.
  • 2: Log Agent specific values.

Field Documentation

◆ agent

rlpack.environments.environments.Environments.agent

The input RLPack agent to be run.

◆ config

rlpack.environments.environments.Environments.config

The input config for setup.

◆ env

rlpack.environments.environments.Environments.env

The gym environment on which the agent will run.

◆ new_shape

rlpack.environments.environments.Environments.new_shape

The new shape requested in config to be used with reshape_func.

◆ reshape_func

rlpack.environments.environments.Environments.reshape_func

The input reshape function for states.