Skip to content

DOC: Migrate cookbooks 1, 2, and 4 to proper doc locations - PR #2#1564

Open
varunj-msft wants to merge 1 commit intomicrosoft:mainfrom
varunj-msft:varunj-msft/7933-Migrate-cookbooks-to-scenarios-PR2
Open

DOC: Migrate cookbooks 1, 2, and 4 to proper doc locations - PR #2#1564
varunj-msft wants to merge 1 commit intomicrosoft:mainfrom
varunj-msft:varunj-msft/7933-Migrate-cookbooks-to-scenarios-PR2

Conversation

@varunj-msft
Copy link
Copy Markdown
Contributor

Description

Migrates three cookbook pages to their proper documentation locations as part of User Story #7933 (Migrate Cookbooks to Scenarios). This is PR 2 of 3.

Cookbook 1 (Sending Prompts) → doc/code/scenarios/9_sending_raw_datasets.py - Shows baseline-only mode using RedTeamAgent with scenario_strategies=[] and custom dataset loading.
Cookbook 2 (Precomputing Turns) → doc/code/datasets/5_simulated_conversation.py - Focuses on the generate_simulated_conversation_async API and replaying precomputed conversation prefixes on different targets.
Cookbook 4 (Testing Bias) → doc/code/executor/benchmark/2_bias_benchmark.py - Covers QuestionAnsweringBenchmark and FairnessBiasBenchmark for bias/fairness testing.
Updated doc/myst.yml with 3 new TOC entries. Cookbooks directory is not deleted yet (PR 3).

Follows up on #1558 (PR 1: Scenario doc pages).

Tests and Documentation

All 3 .ipynb files generated via jupytext --execute --set-kernel pyrit-dev --to ipynb with live API calls; all cells produce expected outputs.
Integration tests auto-discover the new notebooks and pass:
tests/integration/scenarios/test_notebooks_scenarios.py — 9_sending_raw_datasets (26s)
tests/integration/datasets/test_notebooks_datasets.py — 5_simulated_conversation (70s)
tests/integration/executors/test_executor_notebooks.py — 2_bias_benchmark (116s)
ruff check passes on all new .py files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant