smarts.benchmark package

smarts.benchmark.auto_install_requirements(benchmark_spec: Dict[str, Any])[source]

Install dependencies as specified by the configuration given.

smarts.benchmark.list_benchmarks(benchmark_listing)[source]

Lists details of the currently available benchmarks.

smarts.benchmark.run_benchmark(benchmark_name: str, benchmark_version: float | None, agent_locator: str, benchmark_listing: Path, debug_log: bool = False, auto_install: bool = False)[source]

Runs a benchmark with the given configuration. Use scl benchmark list to see the available benchmarks.

Parameters:

  • benchmark_name (str) – The name of the benchmark to run.

  • benchmark_version (float|None) – The version of the benchmark.

  • agent_locator (str) – Locator string for the registered agent.

  • benchmark_listing (pathlib.Path) – A configuration file that lists benchmark metadata and must list the target benchmark.

  • debug_log – Debug to stdout.