Callbacks

Callback API

class Callback

An object to call functions while the Playground is running. You can define the custom functions on_{position} where position can be :

>>> run_begin
...     episodes_cycle_begin
...         episode_begin
...             steps_cycle_begin
...                 step_begin
...                 # env.step()
...                 step_end
...             steps_cycle_end
...         # done==True
...         episode_end
...     episodes_cycle_end
... run_end

set_params(params)

Sets run parameters

set_playground(playground)

Sets reference to the used playground

on_step_begin(step, logs=None)

Triggers on each step beginning

Parameters

• step (int) – current step.

• logs (Optional[dict]) – current logs.

on_step_end(step, logs=None)

Triggers on each step end

Parameters

• step (int) – current step.

• logs (Optional[dict]) – current logs.

on_steps_cycle_begin(step, logs=None)

Triggers on each step cycle beginning

Parameters

• step (int) – current step.

• logs (Optional[dict]) – current logs.

on_steps_cycle_end(step, logs=None)

Triggers on each step cycle end

Parameters

• step (int) – current step.

• logs (Optional[dict]) – current logs.

on_episode_begin(episode, logs=None)

Triggers on each episode beginning

Parameters

• episode (int) – current episode.

• logs (Optional[dict]) – current logs.

on_episode_end(episode, logs=None)

Triggers on each episode end

Parameters

• episode (int) – current episode.

• logs (Optional[dict]) – current logs.

on_episodes_cycle_begin(episode, logs=None)

Triggers on each episode cycle beginning

Parameters

• episode (int) – current episode.

• logs (Optional[dict]) – current logs.

on_episodes_cycle_end(episode, logs=None)

Triggers on each episode cycle end

Parameters

• episode (int) – current episode.

• logs (Optional[dict]) – current logs.

on_run_begin(logs=None)

Triggers on each run beginning

Parameters

logs (Optional[dict]) – current logs.

on_run_end(logs=None)

Triggers on run end

Parameters

logs (Optional[dict]) – current logs.

Logger

class Logger(metrics=None, detailed_step_metrics=None, episode_only_metrics=None, titles_on_top=True)

Default logger in every Playground run.

This will print relevant informations in console.

You can regulate the flow of informations with the argument verbose in run() directly :

• 0 is silent (nothing will be printed)

• 1 is cycles of episodes (aggregated metrics over multiple episodes)

• 2 is every episode (aggregated metrics over all steps)

• 3 is cycles of steps (aggregated metrics over some steps)

• 4 is every step

• 5 is every step detailed (all metrics of all steps)

You can also replace it with you own Logger, with the argument logger in run().

To build you own logger, you have to chose what metrics will be displayed and how will metrics be aggregated over steps/episodes/cycles. To do that, see the Metric codes format.

Default logger in every Playground run.

Parameters

• metrics (Optional[List[Union[str, tuple]]]) – Metrics to display and how to aggregate them.

• detailed_step_metrics (Optional[List[str]]) – Metrics to display only on detailed steps.

• episode_only_metrics (Optional[List[str]]) – Metrics to display only on episodes.

• titles_on_top (bool) – If true, titles will be displayed on top and not at every line.

on_step_begin(step, logs=None)

Triggers on each step beginning

Parameters

• step – current step.

• logs – current logs.

on_step_end(step, logs=None)

Triggers on each step end

Parameters

• step – current step.

• logs – current logs.

on_steps_cycle_begin(step, logs=None)

Triggers on each step cycle beginning

Parameters

• step – current step.

• logs – current logs.

on_steps_cycle_end(step, logs=None)

Triggers on each step cycle end

Parameters

• step – current step.

• logs – current logs.

on_episode_begin(episode, logs=None)

Triggers on each episode beginning

Parameters

• episode – current episode.

• logs – current logs.

on_episode_end(episode, logs=None)

Triggers on each episode end

Parameters

• episode – current episode.

• logs – current logs.

on_episodes_cycle_begin(episode, logs=None)

Triggers on each episode cycle beginning

Parameters

• episode – current episode.

• logs – current logs.

on_episodes_cycle_end(episode, logs=None)

Triggers on each episode cycle end

Parameters

• episode – current episode.

• logs – current logs.

on_run_begin(logs=None)

Triggers on each run beginning

Parameters

logs – current logs.

Metric codes

To fully represent a metric and how to aggregate it, we use metric codes as such:

<logs_key>~<display_name>.<aggregator_function>

Where logs_key is the metric key in logs

Display_name is the optional name that will be displayed in console.

If not specified, the logs_key will be displayed.

Finaly aggregator_function is one of { avg, sum, last }:

• avg computes the average of the metric while aggregating. (default)

• sum computes the sum of the metric while aggregating.

• last only shows the last value of the metric.

Examples

• reward~rwd.sum will aggregate the sum of rewards and display Rwd

• loss will show the average loss as Loss (no surname)

• dt_step~ will show the average step_time with no name (surname is ‘’)

• exploration~exp.last will show the last exploration value as Exp