Testing Module 28000

blaggacao · October 7, 2024, 12:37pm

A really lucky coincidence has it that the new Testing Module landed under PR 28000.

github.com/frappe/frappe

Testing Module

frappe:develop ← blaggacao:testing

opened 08:32PM - 06 Oct 24 UTC

blaggacao

+1114 -845

###### A not-really-public API within `test_runner.py` has mainly been removed w…ith this PR. If you have been bitten and want it temporarily back, please comment to this PR, I'll add it back to the [dumpster](https://github.com/frappe/frappe/pull/27887). I'll try hard to keep an SLA of 2 days during the first 2 weeks after merge, including weekends, best-effort afterwards. ### Enhance Frappe Testing Module Documentation This PR improves the documentation and structure of the Frappe testing module: - Add explanatory docstrings to key files (new: runner.py, discovery.py, environment.py, result.py, __init__.py) - Create a README.md for the frappe/testing module, providing an overview and directing users to the well-commented code - Improve code organization and readability - Enhance logging configuration in __init__.py - Ends up fixiing some regressions of previous PRs in the series (mainly: https://github.com/frappe/frappe/pull/27995 & bis) These changes aim to make the testing module more accessible and easier to understand for developers working with the Frappe framework.

With that new clarity in place and such an emblematic PR number, please ask me anything you ever wondered about framework testing on that PR (or here, but I only check sporadically here).

I still have a fresh context in mind and we shouldn’t let go this emblematic number unexploited.

The plan is that a curated list of Q&A remains on the PR that we then can pipe throug an LLM to create a resume c/o some documentation or a FAQ file.

PS: please also check out the code, it might now reveal some answers all by itself.

PPS: Keep in mind that the CI test runner, until further eventual refactoring, still uses a slightly different code-path but with a very similar setup.

blaggacao · October 8, 2024, 3:10pm

Your two go-to readmes:

github.com

frappe/frappe/blob/develop/frappe/tests/README.md

# Frappe Test Framework

This README provides an overview of the test case framework available in Frappe. These utilities are designed to facilitate efficient and effective testing of Frappe applications.

This is different from the `frappe.testing` module which houses the discovery and runner infrastructure for CLI and CI.

## Directory Structure

The test framework is organized into the following structure:

```
frappe/tests/
├── classes/
│   ├── context_managers.py
│   ├── unit_test_case.py
│   └── ...
├── utils/
│   ├── generators.py
│   └── ...
├── test_api.py

This file has been truncated. show original

github.com

frappe/frappe/blob/develop/frappe/testing/README.md

# Frappe Testing Module

This module provides a comprehensive framework for running tests in Frappe applications. It includes functionality for test discovery, execution, result reporting, and environment setup.

## Key Components

- `TestConfig`: Configuration class for customizing test execution
- `TestRunner`: Main class for running test suites with additional Frappe-specific functionality
- `TestResult`: Custom test result class for improved output formatting and logging
- `discover_all_tests`: Function to discover all tests in specified Frappe apps
- `discover_doctype_tests`: Function to discover tests for specific DocTypes
- `discover_module_tests`: Function to discover tests in specific modules

## Usage

This module is typically used by Frappe's CLI commands for running tests, but can also be used programmatically for custom test execution scenarios.

For detailed information about each component, please refer to the well-commented code in the following files:

- [`__init__.py`](./__init__.py): Module initialization and logging setup

This file has been truncated. show original