Installing Dependencies

Make sure you have Python and Poetry installed (Poetry is a Python package manager).

If you don't have Poetry installed:

sudo apt update -y
sudo apt install -y pipx
pipx install poetry

# Optional
poetry config virtualenvs.in-project true

From the repo root:

poetry install

to install all required dependencies.

Input data

Forecast CSVs live under local/private_input_data/ (gitignored). For Summer 2026 the simulation script expects:

• local/private_input_data/pro_forecasts_2026_summer.csv
• local/private_input_data/bot_forecasts_2026_summer.csv

You can:

• If you have access, download prior analysis outputs (and sometimes inputs) from this Google Drive folder, or
• Request access to analysis data from the instructions shared here which will show you how to use the Metaculus Data Needs Form

Running the analysis

For the current (Summer 2026) tournament:

poetry run python aib_analysis/run_summer_simulations.py

That writes JSON artifacts to local/summer_2026_simulations/.

Check logs/latest_info.log for warnings (annulled questions, weight mismatches, unmatched questions, missing control bots, etc.).

For older seasons (Q1–Q4), use aib_analysis/run_q1_q2_simulations.py and edit the __main__ paths/folders as needed.

Viewing results

poetry run streamlit run aib_analysis/front_end.py

In the UI, set the tournaments folder to the simulation output path (e.g. local/summer_2026_simulations). Restart Streamlit or use Clear cache in the ⋮ menu after changing underlying JSON or loading/scoring code.

To get a markdown results summary (leaderboards, Pro vs bot head-to-head, per-type and team-size breakdowns, charts) from the same JSON outputs:

poetry run python aib_analysis/utils/summarize_season_results.py --simulations-folder local/summer_2026_simulations --season-name "Summer 2026"

That writes results_summary.md, CSV leaderboards, and PNG charts to local/summer_2026_simulations/results_summary/.

Structure

The project is built around SimulatedTournament (see data_models.py / simulated_tournament.py). It is initialized from Forecast objects and derives users, scores, etc. Most analyses are filtered, aggregated, or combined tournaments built from those forecasts.