Pytest

1 / Python Testing Foundation

Pytest was the testing framework across my Python projects. In Vigil, custom Pytest plugins intercepted system calls and enforced security policy baselines during Agent Evaluation test runs. In Foundry, pytest-django handled integration testing of Django views and API endpoints.

2 / Custom Fixtures and Plugins

Vigil's test harness required custom Pytest fixtures that provisioned ephemeral Docker containers before each test, executed agent tool calls inside the sandbox, and captured container state for post-execution assertions. The custom plugin architecture allowed test cases to define security policy expectations declaratively — which system calls were permitted, which resource limits should be enforced — and the harness verified compliance automatically.

3 / Parameterized Testing

Parameterized test execution was essential for testing agent behavior across multiple input scenarios. Since LLM-driven agents produce non-deterministic outputs, the test assertions focused on side effects (file system state, database state, environment variables) rather than exact text output. This state-based assertion strategy provided reliable verification regardless of slight text variations in agent responses.