The Story Behind Logging in Frappe -- define your custom logger in your app

Frappe Framework
1

The Story Behind Logging in Frappe – define your custom logger in your app

image
image800×565 76.5 KB

In any serious system, logging is not just a technical utility—it is the storybook of your application’s life. Every request, every failure, every subtle behavior that the system wants to whisper to you emerges through logs.

In Frappe, logging is designed with intention: to let you see what the framework sees, to replay events after they happen, and to give developers and administrators complete visibility across bench, sites, background jobs, API calls, and user actions. When you understand logging, you don’t merely debug issues—you begin to understand the soul of your Frappe application.

Logging in Frappe is more than a feature; it is the framework’s way of observing itself. It records its own pulse, quietly and continuously, helping developers trace the unseen flow of activity within the system.

To truly understand Frappe, you must understand its logging ecosystem—not just what gets logged, but why it is logged, where it is stored, and how the framework elegantly separates responsibilities across layers.

Content:

  1. Why Logging Matters in a Framework Like Frappe
  2. Two Worlds of Logging Inside Frappe
  3. The Magical Part — Bench Logs vs Site Logs
  4. How Frappe Creates a Logger (Behind the Scenes)
  5. The Full Journey of a Log Message
  6. Database Log DocTypes
  7. Creating Your Own Logger

1. Why Logging Matters in a Framework Like Frappe

Every large system eventually hits problems:

  • A background job fails randomly
  • A user complains of slow performance
  • An integration returns unexpected data
  • An email doesn’t get sent
  • A database transaction deadlocks

Without logs, debugging becomes guesswork.
But with logs, the system becomes self-explanatory.

Frappe approaches logging with a clear philosophy:

“Give developers visibility at two levels: the system (bench) and the tenant (site).”

This is crucial in a multi-tenant world like Frappe.
The framework is designed to support multiple sites within a single bench environment, and logs must make sense in both global and per-site contexts.

2. Two Worlds of Logging Inside Frappe

Frappe uses two distinct logging architectures, each designed for a different purpose:

A. File-Based Logging (Python Logger)

  • These are .log files stored on disk.
  • Used for backend engineers, sysadmins, devops.
  • Captures messages like:
    • Warnings
    • Errors
    • Debug prints
    • Scheduler events
    • Request-level diagnostics

This uses Python’s native logging module under the hood.

B. Database-Based Logging (DocType Logs)

  • These logs are stored as documents inside MariaDB/PostgreSQL.
  • Used for functional users and system managers.
  • Captures:
    • Activity Log
    • Access Log
    • Error Log
    • API Request Log
    • Email Queue
    • Scheduler Job Log
    • Integration Request
    • Route History
    • Notifications

Together, they create a complete 360° monitoring system.

You can think of it like this:

For Developers For Users / Admins
File logs Database logs
Debugging internals Auditing business events
Stored on disk Stored in DocTypes (DB)
Technical Functional

This separation is intentional and elegant.

3. The Magical Part — Bench Logs vs Site Logs

Frappe makes one of the smartest architectural decisions in its logging design:

Every log can be written in two places:

  • Bench-level (global)
  • Site-level (tenant-specific)

This solves the biggest problem in multi-tenant systems:
“Which site caused this error?”

(1) Bench Logs → Located at:

{bench}/logs/

These logs collect everything—across all sites.

Perfect for DevOps monitoring, centralized log aggregation, and overall system health.

(2) Site Logs → Located at:

{bench}/sites/{site}/logs/

These logs isolate events for a specific site.

Perfect for debugging an issue raised by one customer without noise from other sites.

(3) The Smart Behavior

Whenever Frappe knows which site is being processed (web request, background job), the logger writes to both places:

bench/logs/my-module.log
sites/{site}/logs/my-module.log

This dual logging makes life much easier for developers and sysadmins.

4. How Frappe Creates a Logger (Behind the Scenes)

Every time you call:

frappe.logger("module_name")

Frappe performs these steps internally:

  1. Builds a unique logger name using:
module + "-" + site
  1. Checks if this logger already exists in its cache

→ If yes, returns the same logger (no duplicates)

  1. Creates two handlers:
  • Bench-level handler
  • Site-level handler (only if site context exists)
  1. Formats the output with a standard pattern:
timestamp LEVEL module message
  1. Configures log rotation
  • Default size: 100 KB
  • 20 backup files

The whole process is lightweight, efficient, and transparent.

5. The Full Journey of a Log Message

Imagine you write:

logger = frappe.logger("my-app")
logger.info("Payment webhook received")

The journey of that message is:

  1. Frappe identifies the module (“my-app”).
  2. Frappe checks site context.
  • If a site is active → log to both (bench + site)
  • If no site → log only to bench
  1. Message is formatted:
2025-10-31 12:34:56 INFO my-app Payment webhook received
  1. Written to rotating files.
  2. Developers inspect logs when required.

This workflow never changes, no matter how complex your setup becomes.

6. Database Log DocTypes

  • Frappe’s database-backed logs act like the system’s living memory, quietly recording everything that happens—not just errors, but the movements of users, the decisions of background jobs, and the conversations the system has with other services.
  • Every type of log plays its own role in this memory:
    • Error Logs capture the moments when the system stumbles.
    • Activity Logs follow the footsteps of users navigating through the system.
    • API Request Logs document incoming calls, while Integration Requests document outgoing ones.
    • Access Logs reveal what information was opened or exported.
    • Scheduled Job Logs tell the story of background tasks working silently behind the scenes.
  • Together, these logs form a rich narrative, allowing administrators and developers to replay events, understand what went right or wrong, and gain clarity into the system’s behavior over time.
  • But memory must be managed—too much becomes clutter, and too little becomes meaningless.
    That’s why Frappe automatically applies retention rules using Log Settings, clearing out older entries at scheduled intervals so the database stays healthy and responsive.
  • This balance is intentional and elegant: Frappe remembers enough to help you debug, audit, and analyze, but never so much that it becomes burdened by its own history.

7. Creating Your Own Logger

Option A — Add a Logger Inside my_app/__init__.py

import frappe, logging

_log = frappe.logger(
    module="my_app",
    allow_site="site_name",
    with_more_info=False,
)
_log.setLevel(logging.INFO)

def info(msg):
    _log.info(msg)

def error(msg):
    _log.error(msg)

def warning(msg):
    _log.warning(msg)

def debug(msg):
    _log.debug(msg)

Usage:

import my_app

my_app.info("Everything ok")
my_app.error("Something went wrong")

Option B — Initialize Logger During App Install

Inside my_app/utils/install.py

import logging, frappe

def after_install():
    setup_my_app_file_logger()

def setup_my_app_file_logger():
    logger = frappe.logger("my_app", allow_site=frappe.local.site, with_more_info=True)
    logger.setLevel(logging.INFO)

Inside my_app/hooks.py

after_install = "my_app.utils.install.after_install"

Inside my_app/__init__.py

Use lazy initialization for the module-level logger, why?
because the module-level logger in my_app/__init__.py is created at import time, which may occur before site initialization, while the logger in after_install() will work correctly because site context is available during installation.

import frappe, logging

_log = None  
  
def get_logger():  
    global _log  
    if _log is None:  
        _log = frappe.logger("my_app", allow_site=True)  
    return _log 

def info(msg):
    get_logger().info(msg)

def error(msg):
    get_logger().error(msg)

def warning(msg):
    get_logger().warning(msg)

def debug(msg):
    get_logger().debug(msg)

The after_install hook is called during the installation process after the site has been initialized. However, module-level imports can happen at various times, so lazy initialization is the safest approach for ensuring site context is available when creating loggers.

1 Like