smarts.benchmark.entrypoints.benchmark_runner_v0 module

smarts.benchmark.entrypoints.benchmark_runner_v0.benchmark(benchmark_args, agent_locator) → Tuple[Dict, Dict]

Runs the benchmark using the following: :param benchmark_args: Arguments configuring the benchmark. :type benchmark_args: dict :param agent_locator: Locator string for the registered agent. :type agent_locator: str :param debug_log: Whether the benchmark should log to stdout. :type debug_log: bool

smarts.benchmark.entrypoints.benchmark_runner_v0.benchmark_from_configs(benchmark_config, agent_locator, debug_log=False)

Runs a benchmark given the following.

Parameters:

• benchmark_config (str) – The file path to the benchmark configuration.

• agent_locator (str) – Locator string for the registered agent.

• debug_log (bool) – Deprecated. Whether the benchmark should log to stdout.