Anonymize user and customer data on test VM

ERPNext
1

We cloned a live erpnext with real customer data. We would like to anonymize the user and customer data so that we can truly us it for testing.

Does anyone have a script to anonymize the customer and user data?

Or is there some bench option to do anonymize customer data?

How to empty customer doctype activity log?
2

There’s nothing out-of-the-box that does this.

I personally do this whenever I migrate data from Production to a lower environment.
The solution is nothing fancy: just a series of SQL statements that update certain Columns, in certain Tables.

UPDATE "tabCustomer" SET email_id = NULL;
UPDATE "tabAddress" SET city = 'Beverly Hills';

I have dozens, mostly related to Phone, Address, and various configurations and settings (tabSingles values)

3

This project might also help to generate Demo data once you know which fields to overwrite because they contain personally identifiable information:

You could also generate first and last name, city and address data from different sources, in order to be close to real data, but not from real people. I’d be kind of a big mixer exercise, in a way. This can also help to adapt to country specific people, or also to promote an international and universal mix if you want. This would be an exercise of applying realistic probabilities for the desired fake population.

4

So it is done directly at the database level. Got it.

How about the user doctype? I ask because the ID is the email address.
Do you have any script for that? Of course only if you can share.

5

I’ve never done any mass-update of email address on Users. Normally I just mark them as disabled.

To anonymize, you’d have to update not just tabUser, but every other table’s owner and modified_by fields. Plus the User ID is referenced in various other places (permissions, workflows, etc)

6

ok. Got it.

Thank you for replying and for the advice.

1 Like
7

Just updating what I have done so far on erpnext 15.

Login as admininstrator.
Go to customer doctype.
Then select all customers in the list view.
Click on action button and select edit
Select the field “user” and for the data leave it blank.

This deletes the user doctype link to the customer doctype.

then go to user doctype
filter “User Type” with website user
Then select all customers in the list view
Delete all the users

Next go back to customer doctype
select all the customers in the list view
Click on action button and select edit
Select the field “Customer mobile” and for the data leave it blank
next select the field “Customer email” and for the data leave it blank.

This deletes the customer email and mobile number.

Now my next problem. Hahahaha

under the activity log for each customer it shows the email address and mobile number that was removed. This is good but not for me.

1 Like
How to empty customer doctype activity log?
8

https://ridhira.desigoogly.com/ helped in the erpnext telegram chat with the following code. It does some elements of what was needed. Thank you for sharing.


import frappe
import random
import string

def get_random_string(length=10):
    """Generate a random string of fixed length."""
    letters = string.ascii_lowercase
    return ''.join(random.choice(letters) for i in range(length))

def anonymize_user_data():
    """Anonymize email and mobile numbers for all users."""
    frappe.db.commit()
    
    # Anonymize User email and full name
    users = frappe.get_list('User', filters={'email': ['!=', 'Administrator']})
    for user in users:
        new_name = get_random_string()
        new_email = f"{new_name}@example.com"
        
        frappe.db.set_value('User', user.name, 'email', new_email)
        frappe.db.set_value('User', user.name, 'full_name', f"User {new_name.capitalize()}")
        frappe.db.set_value('User', user.name, 'mobile_no', f"99999{random.randint(10000, 99999)}")
        
    # Anonymize other contact information
    contacts = frappe.get_all('Contact')
    for contact in contacts:
        new_email_id = f"{get_random_string()}@anonymized.com"
        new_mobile = f"99999{random.randint(10000, 99999)}"

        frappe.db.set_value('Contact', contact.name, 'email_id', new_email_id)
        frappe.db.set_value('Contact', contact.name, 'mobile_no', new_mobile)

    # You can add more DocTypes here as needed
    # For example, to anonymize data in Lead, Customer, etc.
    # frappe.db.sql("UPDATE tabLead SET email_id = CONCAT(MD5(email_id), '@anonymized.com'), mobile_no = CONCAT('99999', SUBSTR(MD5(mobile_no), 1, 5))")
    
    frappe.db.commit()
    frappe.clear_cache()
    print("User data anonymized successfully.")

Now need to figure out how to delete the activity log in the customer doctype.

1 Like
9

Looking good!