Getting started
Contents
Getting started#
Tip
This section is (almost) identical to the quickstart section in the repository README. If you already read that one, you may want to skip to the advanced usage guides.
Installation#
Install docker on your machine (if on Linux, check the additional installation steps)
Install Beobench using:
pip install beobench
OS support
Linux: recommended and tested (Ubuntu 20.04).
Windows: use via Windows Subsystem for Linux (WSL) recommended.
macOS: experimental support for Apple silicon systems — only intended for development purposes (not running experiments). Intel-based macOS support untested.
Running a first experiment#
Experiment configuration#
To get started with our first experiment, we set up an experiment configuration. Experiment configurations can be given as a yaml file or a Python dictionary. The configuration fully defines an experiment, configuring everything from the RL agent to the environment and its wrappers. The figure below illustrates the config structure.
Let’s look at a concrete example. Consider this config.yaml
file:
agent:
# script to run inside experiment container
origin: ./agent.py
# configuration that can be accessed by script above
config:
num_steps: 100
env:
# gym framework from which we want use an environment
gym: sinergym
# gym-specific environment configuration
config:
# sinergym environment name
name: Eplus-5Zone-hot-continuous-v1
wrappers: [] # no wrappers added for this example
general:
# save experiment data to ``./beobench_results`` directory
local_dir: ./beobench_results
Agent script#
The agent.origin
setting in the configuration file above sets the agent script to be ./agent.py
. The agent script is the main code that is run inside the experiment container. Most of the time this script will define an RL agent but it could really be anything. Simply put, we can think of Beobench as a tool to (1) build a special Docker container and then (2) execute an agent script inside that container.
Let’s create an example agent script, agent.py
:
from beobench.experiment.provider import create_env, config
# create environment and get starting observation
env = create_env()
observation = env.reset()
for _ in range(config["agent"]["config"]["num_steps"]):
# sample random action from environment's action space
action = env.action_space.sample()
# take selected action in environment
observation, reward, done, info = env.step(action)
env.close()
The only Beobench-specific part of this script is the first line:
we import the create_env
function and the config
dictionary from beobench.experiment.provider
.
The create_env
function allows us to create the environment
as definded in our configuration.
The config
dictionary gives us access to the full experiment configuration
(as defined before). These two imports are only available inside an experiment container.
Note
We can use these two imports regardless of the gym framework we are using. This invariability allows us to create agent scripts that work across frameworks.
After these Beobench imports, the agent.py
script above just takes a few random actions in the environment. Feel free to customize the agent script to your requirements.
Alternatively, there are also a number of pre-defined agent scripts available, including a script for using RLlib.
Execution#
Given the configuration and agent script above, we can run the experiment using the command:
beobench run --config config.yaml
import beobench
beobench.run(config = "config.yaml")
Either command will:
Build an experiment container with Sinergym installed.
Execute
agent.py
inside that container.
You have just run your first Beobench experiment.
Next steps#
To learn more about using Beobench, look at the advanced usage section in the documentation.