[discussion] Immutable Ledger

Dear all,

We are proposing a major change in the way the accounting ledgers work in ERPNext.

Please review and comment [Proposal] Immutable Ledger · Issue #11782 · frappe/erpnext · GitHub

Discussion

In most accounting systems, the ledger is immutable. If a cancellation or adjustment has to be made, it has to be as a new entry that reverses or changes the impact of a previous entry.

Here are some problems related to this:

  • Forward posting is computationally expensive.
  • In stock, where valuation is based on First-in-first-out (FIFO), the entire sequence gets regenerated, upsetting valuations and profits for all subsequent transactions.
  • Taxes calculated and paid for a period may also get changed.
  • Making changes back in time can lead to unexpected consequences

In ERPNext we have allowed back dated entries that forward updates balances and resets FIFO stacks and this is a bad practice. We should fix this so that forward dated ledgers are not allowed.

Expected Change

All cancellations must be made in the forward direction only, we can maintain a separate “transaction date” for record purposes if a transaction was posted on current date, but belonged to another date/

  • All Stock and Accounting Ledger transactions to be forward posted only.
  • Add Transaction Date property on Accounting and Stock Transactions for reporting purpose.
  • Reverse entries for cancellations on current date. This will also solve the problem of negative stock at a past date.
  • Stock and General Ledger will never be deleted
16 Likes

I am not so sure that this is a great idea. It is difficult to expect people to post only current dated transactions. Can we not ensure that the transactions are posted, but the recalculation of the impact of such posting on the entire chain of events and on the P&L and the Balance Sheet is done as a batch job between 1am and 5am or something?

Let’s not take away the flexibility that organizations have for the sake of performance. But I do understand that performance is an important thing.

The other option is to consider storing this information (say Stock Ledger) in the database rather than calculating it on the fly. So, let’s say that somebody is posting an invoice for April and let’s say that they have used Moving Average as stock valuation. So the database maintains a Stock Ledger, finding out the Stock Valuation and the Qty for the previous posting is very easy, so posting the Stock Valuation after the current entry is also very easy. Now the table has a field that says, has the impact of this number been calculated for subsequent transactions. Let’s say that this transaction was posted at peak times, so all transactions after the datetime stamp of the posted transactions get flagged as “No” for the impact of this calculation. As the server encounters lull times, the background job kicks off and posts the correct Stock Valuation rates after the last predated transaction and as it does, it changes the value of that Recalculation Complete field to Yes.

I know I am proposing more complicated solutions, but I anticipate some of my clients being not very happy with me if we don’t allow this flexibility for organizations.

Things never work perfectly for organizations and that’s the reality. So, when they discover that they missed out an Invoice, they definitely need the flexibility to post that invoice on that date.

The other option is to consider a Posting Date and a Display Date. Nobody should be able to change the Posting Date, the current datetime is what it automatically selects, but the Display Date shows the date they want the Invoice to show, for regulatory or whatever else purposes. ERPNext can calculate the valuation based on Posting Date.

We need an option even if it is an imperfect one. Not having an option will stress out our customers and prospects.

Thanks

Jay

10 Likes

I agree with @JayRam most clients like to have some flexibility with accounts.

We should understand our target customers are mostly SMEs rather than corporate clients. Mostly these customers are moving their accounting from systems which allow changing of any record at anytime (I mean there is no submit option to permanently fix a record in old system.)

Why can’t we use created_on field of doctype as a fixed date which can not be changed.

3 Likes

Even though this qualifies as a “breaking change” I’m in favor of it. I have always thought that the ability to adjust old entries with a couple extra clicks was bad practice. This may be more of criticism of the “submit” structure, but having immutable ledger entries will improve this.

Let me add some fuel to @rmehta’s reasoning here, in that this is probably the single biggest feature that can be added to reduce fraud prevention and embezzlement in the ERPNext small-business ecosystem. In small businesses, you are most likely to be stolen from by your most trusted employees and employed family members. At larger companies with audit systems in place, it takes complicit parties to achieve fraud. Giving a bookkeeper without a equity stake in business the ability to “just fix things” is probably not good management, at least not without a verification step. I’m sure we all have horror stories to share here, so I won’t ramble.

12 Likes

This is valid scenario in many business to Do not allow back dated entry. ERPNext Should have this feature.

As per my understanding, For those who want back dated entry for some reason, no need to worry, as they can easily generate any report based on Posting Date or Transaction Date.

If we receive X amount from customer in Bank account by mistake and we are returning it after one month, then its necessary to do both entries in ERP to match it as per Bank Statement and Monthly balance. It will add credibility to system.

For cancellation, it will solve issue by reverse entry posting, but then report might be wrong if cancel date and posting date is not equal.

5 Likes

I am still trying to understand the implications of this change. If this means I am no longer allowed to amend/add a back-dated account/stock transaction, it means ERPNext will almost become unusable in our organization.
So let us examine the requirements that asked for this new feature.

Fine. But is there any quantification about this computational cost. As per I understand, this is only used very rarely. People using this don’t really care how much time it takes to do the forward entries. Or are you saying it affects all the transactions irrespective of whether it is current dated or back dated?

Right. However that is exactly the purpose of back-dated entries. We have done a mistake earlier, which is getting corrected by this back-dated entry. Otherwise why will anyone do a back-dated entry?

But at the end every body lives happily ever after…

That could be a compelling reason to make this change if it is true. However I am not sure this change will help or the only way to resolve that. Once a document is submitted, we don’t allow modifying the same document. We have to amend it and the action will be audited. As long as your employees cannot delete the records behind your back, they can’t cheat you without your knowledge. With this change, they can still reverse the actions with a current dated transaction, so effectively they can still cheat someone who doesn’t monitor their transactions, right?

So, what are the compelling reasons to do this change or not do it?
For: We are humans and we do mistakes, so we need a mechanism to fix it. It could be a missing stock entry or voucher entry. We want that to update the valuation of subsequent transactions. Will the new approach provide a way of achieving this? I guess no.

Against: This could lead to accounting frauds, so not recommended by some companies. can this be done without disabling back-dated entries. Perhaps to certain extent, by providing a report of all back-dated entries. We could also make this difficult by asking for a new authorization level, so that normal employees cannot do the same. Another approach would be to disable it by default and provide a global default to turn it on.

5 Likes

Some thoughts

  1. Maybe we could think of a concept of open and closed periods for accounting. This will help handle reporting easily and closed periods would also minimize the data to be read / modified.
  2. Any back dated posting with different transaction date could be posted to the latest open period.
  3. Period closing could also be separate for Manufacturing, AR, AP etc.
11 Likes

If this is only affecting back-dated transactions, I don’t think there is a problem worth solving here from the performance perspective. We could simply state ‘back-dating’ is a bad practice and results in poor performance, which needs no code changes. If you think people won’t read the docs, we could ask for a setting to be turned on for ‘edit posting date’, which is a far simpler change than the one proposed.

Cant really get the implication of this :grimacing: Only one question, i need to load last 4 years sales in erpnext, can i do that after the changes to Immutable Ledger?

1 Like

@JoEz, you may not get the implication because may it is not one requirements for you country.

But some countrys, requires that the books should be immutable, Brazilian accounting laws and requirements define that is an standard.

About your question, when you talk about load 4 years of sales in erpnext, I dont see any kind of problem, because you have “Sales Orders”, that are not accouting related, and if you need to follow an accounting standard, an accountant never will suggest you to load, past dated transactions, him will suggest to close the books, and load one opening transaction.

So, in that way, I dont think that it will block any user, in any situation, given the user go trought the properly document, for the properly scenario.

I think too, that both behaviors are acceptable for ERPNext, in some countries or companies, you may not worry about immutability of the books, but when you reach a single one that it’s a requirement, it will be decisive to pick or not ERPNext as a choice.

7 Likes

That’s what i thought, just needed confirmation. Thx

If an organization does not want to allow back dated entries, its very easy to do that even with the current version and it has always been easy to do that. Just take away users ability to Edit Posting Date/Time.

What I’m whining about is our proposal to take that away from organizations that want to have that flexibility. Sure there are risks, but its possible to mitigate those risks as well. A report that lists all the transactions where the posting date is prior to the Created On date will give the head honcho a good perspective of what mischief her/his people are up to.

Thanks

Jay

5 Likes

Sure, we need to give the organizations that want to use immutable ledger the flexibility to do exactly that, but not at the cost of the flexibility of other organizations that don’t want to be constrained by that feature.

Thanks

Jay

2 Likes

Totally agree…I think ERPNext is already on the track this …offering ‘high-flexibility’ model…which is very useful for most SME… I don’t agree if this flexibility is taken-out

Yes correct…why need change to the core?? It really depends on the organizations whether they allow back-dated or not…

1 Like

Can we allow back-dated entries till the closure of that accounting period? So if my company has closed FY 2016-17 then I should not be able to do back-dated entry on or before 31st March 2017 (Last day of FY 2016-17). But I should be able to do back-dated entries in current financial year till I close current financial year.

Companies which are required by Law or otherwise to follow strict accounting practices, would typically close books of accounts on quarterly or even monthly basis. So they will automatically constrain their flexibility of posting back-dated entries. Others can choose not to close books of accounts till the time they are sure they will not be needed to make back-dated entries.

My two cents worth,

Shrikant

7 Likes

I totally agree with Shrikant, most companies can change back entries until that period is closed. It would be helpful to have this functionality and be able to change accounting entries according to period and type of accounting entries.

1 Like

While I agree with the fact that back dated entry is bad, however it should be the organisations decision to allow or not allow back dated entries.

Many businesses may use back dated entry to manipulate their books, but there are genuine cases where back dated entry is needed.

6 Likes

Its bad practice when the books are closed for a period. If books are open for the period, this is normal and expected.

There are two concepts - entry date (date the transaction is entered) and applicable date (date the transaction is to take place) - both are there in ERPNext right now, and can be used. Or as you suggested, add a Transaction Date. Preventing entry in OPEN (not closed) periods is going to cause more problems than it solves. I agree with @JayRam on this.

In the enterprise systems I’ve worked with, there is a “Temporary” Entry table and a “Final” entry table. Entries in the “Temporary” table can be modified and you can do whatever. When “Submit” happens, the transaction is moved from the Temporary Table to the Final table, is given an immutable ID (no more cancellations or amends allowed), its added to the General Ledger (never to be deleted again) and any adjustments must be done with a new “reversal” or “adjustment” entry. This kind of system in ERPNext (rather than the draft concept currently) would solve a lot of problems - especially with missing voucher numbers when cancellations/deletions happen, and would prevent manipulation while still allowing the flexibility to deal with real scenarios.

4 Likes

Hi,
A lot already said in fact. Would like to share my POV:

Fiscal Year: What if we have Periods / Months setup in Fiscal Year like:
Period 1: Start Date >>> End Date (End Date maybe skipped as Start Date would make it obvious)
Period 2 >>> and so on ending up with Period 13 & 14 for Pre Post Audit Adjustments.
And this Month Pattern can be associated with Fiscal Year which will then be associated with any of the Company as 1 company may have Jan-Dec and another company may have July-June Fiscal Years.

Month Closing: At Month End designated personnel may Open Next Period / Month and similarly with approval may open Prior Periods / Months.

Current Month: Current Month / Period; these fields may also be made available at Company Form for Accounts Receivable, Accounts Payable, General Ledger, Inventory etc. separately considering the organizational structure and segregation of duties.

Dates: Transactions may have 2 different Dates; GL Date & Transaction Date. Once any of the transaction is voided GL Date would be of the date at which ledger needs to be impacted which may differ from Transaction Date (for reporting / referencing purpose).

Adjustments: Dates / Period / Month for Inventory Adjustments, Purchase / Sales Returns etc. are to be Business Decisions, which should be taken from User.

Flags / Integrity: Transactions may have flags like P - Posted; Blank - Unposted. But at the end every reversal or adjustment needs to be there in the ledger to track transactional impacts and later on in future maybe used for Integrity Reporting i.e. GL Balances Vs AR Balances, AP Balances Vs GL Balances etc.

Company Setup / Form: May have below fields:

  1. Fiscal Period Pattern: for the period dates which may further be used for budget patterns, comparisons & allocations in future. Period 12 may end at June 29 and Period 13 / 14 will start and end at June 30.
  2. Number of Periods: 12 or 14 based on utilization of Audit / Adjustments periods 13 & 14 to report As Of Period 12 and to avoid including post-audit adjustments in period 12.
  3. Current Period AR
  4. Current Period AP
  5. Current Period GL
  6. Current Period Inventory