Bulk import data from csv files using rest APIs

ERPNext User Forum
1

I have a 3rd party program generating data remotely i want to append the data generated there to my bench, the data is in csv format, I wanted to know how shall i do the process, I dont want to iterate over every row and call post method every time

I want to know how to import in Bulk from csv files

It would be a greate help!!

Regards,
Sneha

2

@Sneha_Manjunath I did this by just creating a bulk api method for creation. and using this new method you can bulk create the entire csv file with one single api . you still have to iterate your file and arrange the data in one single array .

3

Sir can you elaborate a bit, I did’nt get any bulk api method for creation.
Is there any reference code or documentation which I can refer

I’m specifically doing this on the Attendance doctype of HRMS, I’m neither getting the end points to post the data

@app.route('/', methods=['GET'])
def attendance_assignment():
    doctype_name = 'Attendance'
    attendance_url = url + doctype_name
    df = pd.read_csv("Attendance.csv")
    data_to_import = {}
    for index, row in df.iterrows():
        entry = {
            "naming_series": row["naming_series"],
            "employee": row["employee"],
            "status": row["status"],
            "attendance_date": row["attendance_date"],
            "company": row["company"],
            "working_hours": float(row["working_hours"]),
            "shift": int(row["shift"]),
            "in_time": row["in_time"],
            "out_time": row["out_time"],
            "late_entry": int(row["late_entry"]),
            "early_exit": int(row["early_exit"])
        }

        data_import = json.dumps(entry)
        try:
            employee_response = requests.request("POST", attendance_url, data=data_import, headers=headers)
            if employee_response.status_code == 200:
                employee_data = employee_response.json()
                data_to_import[index] = str(employee_data)
            else:
                data_to_import[index] = str(employee_response.status_code)
        except Exception as e:
            data_to_import[index] = str({"error": f"An error occurred: {str(e)}"})
    return jsonify(data_to_import)

I did something like this to begin with but here for every row I’m calling post method which is not feasible in a long run, so wanted to know how to bulk push the data in one single post call and also how my data structure must be while pushing the data

4

@Sneha_Manjunath I created a new api method . it’s simple

5

Can you please share one such api method which you have created! Please

6

@Sneha_Manjunath it’s a simple method that takes list and then insert them to the database

1 Like
7

OK sir thank you