RingWorld

RingWorldERExperiment

The experimental module used to run experiments in RingWorld for the ER Agents.

main_experiment

Run an experiment from config. See RingWorldERExperiment.working_experiment for details on running on the command line and RingWorldERExperiment.default_config for info about the default configuration.

working_experiment

Creates a wrapper experiment where the main experiment is called with progress=true, testing=true and the config is the default_config with the addition of the keyword arguments.

Automatically generated docs for RingWorldERExperiment config.

Experiment details.

• seed::Int: seed of RNG
• steps::Int: Number of steps taken in the experiment
• synopsis::Bool: Report full results or a synopsis.

Logging Extras

By default the experiment will log and save (depending on the synopsis flag) the logging group :EXP. You can add extra logging groups and [group, name] pairs using the below arguments. Everything added to save_extras will be passed to the save operation, and will be logged automatically. The groups and names added to log_extras will be ommited from save_results but still passed back to the user through the data dict.

• <log_extras::Vector{Union{String, Vector{String}}>: which group and <name> to log to the data dict. This will not be passed to save.
• <save_extras::Vector{Union{String, Vector{String}}>: which groups and <names> to log to the data dict. This will be passed to save.

Environment details

This experiment uses the RingWorld environment. The usable args are:

• size::Int: Number of states in the ring.

agent details

RNN

The RNN used for this experiment and its total hidden size, as well as a flag to use (or not use) zhu's deep action network. See

• cell::String: The typeof cell. Many types are possible.
• deepaction::Bool: Whether to use Zhu et. al.'s deep action 4 RNNs idea. -internal_a::Int: the size of the action representation layer when deepaction=true
• numhidden::Int: Size of hidden state in RNNs.

Prediction Problem

Define the prediction problem for the experiment.

• outhorde::String: The horde used to construct the targets.
• outgamma::Float64: The discount (used by specific hordes).

Optimizer details

Flux optimizers are used. See flux documentation and ExpUtils.Flux.get_optimizer for details.

• opt::String: The name of the optimizer used
• Parameters defined by the particular optimizer.

Learning update and replay details including:

• Replay:

  • replay_size::Int: How many transitions are stored in the replay.
  • warm_up::Int: How many steps for warm-up (i.e. before learning begins).

• Update details:

  • batch_size::Int: size of batch
  • truncation::Int: Length of sequences used for training.
  • update_freq::Int: Time between updates (counted in agent interactions)
  • target_update_freq::Int: Time between target network updates (counted in agent interactions)
  • hs_learnable::Bool: Strategy for dealing w/ hidden state in buffer.

Default Performance:

Time: 0:00:56
Dict{String, Matrix{Float32}} with 2 entries:
  \"err\"  => [0.0 0.0; 0.0 0.0; … ; 0.0128746 -0.000129551; 0.00117147 -0.00140008]
  \"pred\" => [0.0 0.0; 0.0 0.0; … ; 0.0128746 -0.000129551; 1.00117 -0.00140008]

get_ann_size

Helper function which constructs the environment and agent using default config and kwargs then returns the number of parameters in the model.

construct_agent

Construct the agent for the ringworld experiments.

construct_env

Construct ringworld environment with size::Int.