How I Use Weights & Biases to Track My Experiments

Once your experiment count grows past a handful of runs, tracking becomes the job. You start asking the same questions every time you come back to a project:

Which run actually worked?
What changed between that run and the one before it?
Which config produced that result?
Which checkpoint is worth keeping?

I use Weights & Biases to answer those questions. Here's exactly how I set it up.

What Changes When You Switch

TensorBoard is logs. WandB is memory.

WandB isn't just a prettier version of the same thing. The mental model is different.

TensorBoard reads from files your training job writes to disk and shows you curves. WandB tracks each run as a first-class object — config, metrics, artifacts, gradients, system stats, and source code are all attached to it and queryable later.

The practical difference shows up when you have five runs with different hyperparameters and you want to know which one to build on. With TensorBoard, you're eyeballing folder names and trying to remember what run_47_final_v2 was about. With WandB, you open the project table, sort by test/mean_reward, and the answer is right there.

That said, TensorBoard isn't bad. For a single experiment with a clean hypothesis, it's perfectly fine. The gap opens when your experiment count grows and you need to compare, revisit, and reproduce.

Prerequisites: Set Up Your W&B Project

Before the training code runs, you need a W&B account, an API key, and a project to log into. This takes about five minutes.

1. Create an account

2. Create an API key

W&B needs an API key to authenticate your machine. Follow the quickstart guide or go straight to User Settings → Create new API key. Copy it immediately — W&B only shows the full key once.

Install wandb, set your API key, then log in — toggle uv or pip at the top:

uv pip install wandb
export WANDB_API_KEY=<your_api_key>
wandb login

See environment variables for the full list (WANDB_API_KEY, WANDB_ENTITY, WANDB_PROJECT, and more). For uv, see the uv docs.

3. Create a project

A project is the folder where all your runs live — configs, metrics, artifacts, and videos grouped together.

You do not need to create it manually in the dashboard. W&B creates the project on the first wandb.init() call. Pick a clear name (e.g. rl-exp or ppo-lunar-lander) and reuse it across runs so everything stays in one table.

Your entity is your username (personal account) or team name (shared workspace). Find it in the top-left of the W&B dashboard after login.

4. Set `ENTITY` and `PROJECT` in your environment

The LunarLander script reads these from a .env file via python-dotenv:


ENTITY=your-wandb-username-or-team
PROJECT=your-project-name
WANDB_API_KEY=your-api-key-here

ENTITY maps to wandb.init(entity=...) and PROJECT maps to wandb.init(project=...). If you set WANDB_ENTITY and WANDB_PROJECT as environment variables instead, W&B picks them up automatically — see the env var docs.

Once this is in place, wandb.init() in the training script will create (or attach to) your project and start logging runs.

Useful links

W&B Quickstart — install, login, first run
User Settings & API keys — create and manage keys
Python SDK reference — wandb.init(), wandb.log(), and more
Stable Baselines3 integration — WandbCallback docs used in this post
TensorBoard sync — how sync_tensorboard=True works

My Setup: PPO on LunarLander-v3

Here's a concrete example. I'm training a PPO agent on LunarLander-v3 using stable-baselines3. Even for something this focused, there's a fair amount to track: mean reward across episodes, episode length, gradient behaviour, the best checkpoint, and final test performance after training. The full code is on GitHub.

Here's how the WandB integration is wired in.

Initialising the Run With Config

The first thing I do before any environment or model setup is initialise the WandB run and pass in the full training config:


run = wandb.init(
    entity=ENTITY,
    project=PROJECT,
    name="ppo-lunar-lander-v3_mk2",
    config=dict(
        env_id="LunarLander-v3",
        n_envs=16,
        n_epochs=4,
        batch_size=64,
        total_timesteps=1_000_000,
        algorithm="PPO",
        policy="MlpPolicy",
    ),
    sync_tensorboard=True,
    monitor_gym=False,
    save_code=True,
)

sync_tensorboard=True means anything SB3 writes to TensorBoard automatically syncs to WandB. You get both without managing two systems separately.

save_code=True snapshots your training script at the start of the run. When you come back three weeks later, you can see exactly what code produced a given result — not just the metrics.

The WandB Callback


wandb_callback = WandbCallback(
    gradient_save_freq=1_000,
    model_save_path=os.path.join(BASE_DIR, "model", "wandb"),
    verbose=2,
)

gradient_save_freq=1_000 logs gradient histograms every 1000 steps. This is where silent training problems show up — vanishing gradients, weight saturation — before they kill a run. It's the kind of thing you'd miss entirely if you were only watching reward curves.

This runs alongside SB3's EvalCallback, which handles best-model checkpointing on a separate eval environment:


eval_callback = EvalCallback(
    eval_env,
    best_model_save_path=BEST_MODEL_DIR,
    log_path=LOG_DIR,
    eval_freq=max(10_000 // N_ENVS, 1),
    n_eval_episodes=5,
    deterministic=True,
)

Both callbacks are passed to model.learn() together. WandB gets metrics every step; eval runs every eval_freq steps and saves the best checkpoint.

Logging Final Test Metrics

After training, I run 10 inference episodes on a clean environment and log the results back to the same run:


mean_reward, std_reward = evaluate_policy(
    model,
    test_env,
    n_eval_episodes=10,
    deterministic=True,
)

wandb.log({"test/mean_reward": mean_reward, "test/std_reward": std_reward})
run.finish()

Attaching the test result to the training run means every experiment in the project table has a test/mean_reward to sort by. You're not correlating folder names to results — the connection is already there.

What a Finished Run Contains

By the time training ends, a single WandB run has everything in one place:

What	Where
Training curves (reward, episode length)	Charts — synced from TensorBoard
Gradient histograms	Charts — logged every 1000 steps
Hyperparameter config	Overview — queryable across all runs
Best model checkpoint	Artifacts
Training and test videos	Artifacts
Final test reward	Summary — sortable in the project table
Source code snapshot	Files

Full Training Script

The complete agent.py for this experiment is below. It's also on GitHub if you want to see the rest of the folder including the eval and upload scripts.

Where I'd Go From Here

This setup covers a single experiment cleanly. Where it starts paying off compoundly is when you run sweeps — WandB has a built-in sweep agent that will grid-search or Bayesian-search over your config, spin up multiple runs, and populate your project table automatically. That's the next step I'd add to this workflow.

For now, even without sweeps, the difference is tangible. Every run I've done on this project is searchable, reproducible, and attached to the code that produced it. The checkpoint I shipped was one click to find. The config that worked is sitting in the run overview, not buried in a comment or a filename I'd eventually forget.

The overhead to get there is maybe 20 lines. It's worth it from the first run.