RingWorld
Experience Replay Experiment
RingWorldERExperiment
— ModuleRingWorldERExperiment
The experimental module used to run experiments in RingWorld for the ER Agents.
RingWorldERExperiment.main_experiment
— Functionmain_experiment
Run an experiment from config. See RingWorldERExperiment.working_experiment
for details on running on the command line and RingWorldERExperiment.default_config
for info about the default configuration.
RingWorldERExperiment.working_experiment
— Functionworking_experiment
Creates a wrapper experiment where the main experiment is called with progress=true, testing=true and the config is the default_config with the addition of the keyword arguments.
RingWorldERExperiment.default_config
— FunctionAutomatically generated docs for RingWorldERExperiment config.
Experiment details.
seed::Int
: seed of RNGsteps::Int
: Number of steps taken in the experimentsynopsis::Bool
: Report full results or a synopsis.
Logging Extras
By default the experiment will log and save (depending on the synopsis flag) the logging group :EXP
. You can add extra logging groups and [group, name] pairs using the below arguments. Everything added to save_extras
will be passed to the save operation, and will be logged automatically. The groups and names added to log_extras
will be ommited from save_results but still passed back to the user through the data dict.
<log_extras::Vector{Union{String, Vector{String}}>
: which group and <name> to log to the data dict. This will not be passed to save.<save_extras::Vector{Union{String, Vector{String}}>
: which groups and <names> to log to the data dict. This will be passed to save.
Environment details
This experiment uses the RingWorld environment. The usable args are:
size::Int
: Number of states in the ring.
agent details
RNN
The RNN used for this experiment and its total hidden size, as well as a flag to use (or not use) zhu's deep action network. See
cell::String
: The typeof cell. Many types are possible.deepaction::Bool
: Whether to use Zhu et. al.'s deep action 4 RNNs idea. -internal_a::Int
: the size of the action representation layer whendeepaction=true
numhidden::Int
: Size of hidden state in RNNs.
Prediction Problem
Define the prediction problem for the experiment.
outhorde::String
: The horde used to construct the targets.outgamma::Float64
: The discount (used by specific hordes).
Optimizer details
Flux optimizers are used. See flux documentation and ExpUtils.Flux.get_optimizer
for details.
opt::String
: The name of the optimizer used- Parameters defined by the particular optimizer.
Learning update and replay details including:
Replay:
replay_size::Int
: How many transitions are stored in the replay.warm_up::Int
: How many steps for warm-up (i.e. before learning begins).
Update details:
batch_size::Int
: size of batchtruncation::Int
: Length of sequences used for training.update_freq::Int
: Time between updates (counted in agent interactions)target_update_freq::Int
: Time between target network updates (counted in agent interactions)hs_learnable::Bool
: Strategy for dealing w/ hidden state in buffer.
Default Performance:
Time: 0:00:56
Dict{String, Matrix{Float32}} with 2 entries:
\"err\" => [0.0 0.0; 0.0 0.0; … ; 0.0128746 -0.000129551; 0.00117147 -0.00140008]
\"pred\" => [0.0 0.0; 0.0 0.0; … ; 0.0128746 -0.000129551; 1.00117 -0.00140008]
RingWorldERExperiment.get_ann_size
— Functionget_ann_size
Helper function which constructs the environment and agent using default config and kwargs then returns the number of parameters in the model.
RingWorldERExperiment.construct_agent
— Functionconstruct_agent
Construct the agent for the ringworld experiments.
RingWorldERExperiment.construct_env
— Functionconstruct_env
Construct ringworld environment with size::Int
.