Example: Create an order line in acquisitions with vendor price 18.90 and a discount of 5 % results in a total (tax excl.) of 17.95. The correct value in the database (ecost_tax_excluded) is 17.955000 which should be rounded to 17.96. The problem is that all financial calculations in Perl and JavaScript are done with binary floating-point arithmetic instead of decimal arithmetic. For a short introduction to this topic see https://floating-point-gui.de. The solution is to use Math::BigFloat in Perl and bignumber.js (https://github.com/MikeMcl/bignumber.js) or big.js (https://github.com/MikeMcl/big.js) in JavaScript for every financial calculation.
Created attachment 161918 [details] [review] Bug 36049: Fix price rounding and add a unit test This patch only fixes price rounding by using Math::BigFloat. To test: 1) Create an order line in acquisitions with vendor price 18.90 and a discount of 5 %. 2) Verify that the total (tax excl.) in the order table on the basket page is 17.95. 3) Apply the patch. 4) Reload the basket page. 5) Verify that the total (tax excl.) in the order table on the basket page now is 17.96, which is correct. 6) Run: prove t/Number/Price.t Sponsored-by: Karlsruhe Institute of Technology (KIT)
Hi Raphael. I had a go at testing, however the price doesn't change for me - it stays as 17.95. David
(In reply to David Nind from comment #2) > Hi Raphael. > > I had a go at testing, however the price doesn't change for me - it stays as > 17.95. > > David Replying to myself - it worked when I set OrderPriceRounding to round. Also wondering whether this is the same/similar to bug 35114
Created attachment 161951 [details] [review] Bug 36049: Fix price rounding and add a unit test This patch only fixes price rounding by using Math::BigFloat. To test: 1) Create an order line in acquisitions with vendor price 18.90 and a discount of 5 %. 2) Verify that the total (tax excl.) in the order table on the basket page is 17.95. 3) Apply the patch. 4) Reload the basket page. 5) Verify that the total (tax excl.) in the order table on the basket page now is 17.96, which is correct. 6) Run: prove t/Number/Price.t Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com>
(In reply to David Nind from comment #3) > (In reply to David Nind from comment #2) > > Hi Raphael. > > > > I had a go at testing, however the price doesn't change for me - it stays as > > 17.95. > > > > David > > Replying to myself - it worked when I set OrderPriceRounding to round. > > Also wondering whether this is the same/similar to bug 35114 Hey David, sorry, we forgot to mention that syspref OrderPriceRounding should be enabled to "Round" to nearest cent.
(In reply to David Nind from comment #3) > Also wondering whether this is the same/similar to bug 35114 The similarity to bug 35114 is that both bugs deal with rounding issues. Thats why I added this bug to "See Also". I think that the patch in bug 35114 only solves part of the problem by rearranging rounding operations and using a nearest rounding mode instead of truncation in JavaScript. Yes, it improves the situation as there are fewer cases with incorrect results, but cases with incorrect results still remain as the underlying problem is not using decimal arithmetic. For example "Math.round(18.90 * (100 - 5)) / 100" in JavaScript evaluates to "17.95" instead of "17.96". This can only be fixed by using a decimal arithmetic package like bignumber.js or big.js.
Thanks Raphael! That helps. I'll have another go at testing bug 35114.
Created attachment 162699 [details] [review] Bug 36049: Add unit test Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com>
Created attachment 162700 [details] [review] Bug 36049: Fix price rounding This patch only fixes price rounding by using Math::BigFloat. To test: 1) Create an order line in acquisitions with vendor price 18.90 and a discount of 5 %. 2) Verify that the total (tax excl.) in the order table on the basket page is 17.95. 3) Apply the patch. 4) Reload the basket page. 5) Verify that the total (tax excl.) in the order table on the basket page now is 17.96, which is correct. 6) Run: prove t/Number/Price.t Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com>
Created attachment 162701 [details] [review] Bug 36049: Fix price rounding This patch only fixes price rounding by using Math::BigFloat. To test: 0) Enable syspref OrderPriceRounding 1) Create an order line in acquisitions with vendor price 18.90 and a discount of 5 %. 2) Verify that the total (tax excl.) in the order table on the basket page is 17.95. 3) Apply the patch. 4) Reload the basket page. 5) Verify that the total (tax excl.) in the order table on the basket page now is 17.96, which is correct. 6) Run: prove t/Number/Price.t Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com>
In basket view, shouldn't "Budgeted cost tax exc." also honor OrderPriceRounding ? Same about "Budgeted cost tax inc" and "GST". In the new order form, "Total" (and "Budgeted cost:" depending on previous question) now aren't consistent with the basket view. It's the previous step and the same data should have the same value. So we would have basket's "total tax exc" honoring well OrderPriceRounding but there is still buggy rounding around. Either for the same data on the previous page or on the same page close to "total tax exc". --- Commit was split to confirm that the test fails without implementation. That shows the test does it's work :)
(In reply to Victor Grousset/tuxayo from comment #11) > In basket view, shouldn't "Budgeted cost tax exc." also honor > OrderPriceRounding ? > Same about "Budgeted cost tax inc" and "GST". If it doesn't honor it right now, then this seems a bit out of scope here and would be better suited for a separate bug for discussion/decision. > In the new order form, "Total" (and "Budgeted cost:" depending on previous > question) now aren't consistent with the basket view. It's the previous step > and the same data should have the same value. I believe that this calculation is done with JavaScript only and might require a different kind of fix. It might be also not following the system preference? > So we would have basket's "total tax exc" honoring well OrderPriceRounding > but there is still buggy rounding around. Either for the same data on the > previous page or on the same page close to "total tax exc". Can you explain this one a bit differently?
(In reply to Katrin Fischer from comment #12) > (In reply to Victor Grousset/tuxayo from comment #11) > > In basket view, shouldn't "Budgeted cost tax exc." also honor > > OrderPriceRounding ? > > Same about "Budgeted cost tax inc" and "GST". > > If it doesn't honor it right now, then this seems a bit out of scope here > and would be better suited for a separate bug for discussion/decision. Ok. It introduces an inconsistency (17.95 & 17.96 in the same page) but now that it's known and deemed fine for this bug. Not blocking then. > > In the new order form, "Total" (and "Budgeted cost:" depending on previous > > question) now aren't consistent with the basket view. It's the previous step > > and the same data should have the same value. > > I believe that this calculation is done with JavaScript only and might > require a different kind of fix. It might be also not following the system > preference? Ok, same as above, not a issue for here then :) > > So we would have basket's "total tax exc" honoring well OrderPriceRounding > > but there is still buggy rounding around. Either for the same data on the > > previous page or on the same page close to "total tax exc". > > Can you explain this one a bit differently? That was a sum up of the two previous points. But failed since it actually doesn't help and adds confusion ^^"
Created attachment 171598 [details] [review] Bug 36049: Add unit test Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Created attachment 171599 [details] [review] Bug 36049: Fix price rounding This patch only fixes price rounding by using Math::BigFloat. To test: 0) Enable syspref OrderPriceRounding 1) Create an order line in acquisitions with vendor price 18.90 and a discount of 5 %. 2) Verify that the total (tax excl.) in the order table on the basket page is 17.95. 3) Apply the patch. 4) Reload the basket page. 5) Verify that the total (tax excl.) in the order table on the basket page now is 17.96, which is correct. 6) Run: prove t/Number/Price.t Sponsored-by: Karlsruhe Institute of Technology (KIT) Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Works, makes sense, QA script happy, code looks good, passing QA :)
I'll have another look before pushing this as you are in principle right about adding inconsistencies. Maybe Raphael can weigh in here as he might have a better idea having looked at the code more closely?
(In reply to Katrin Fischer from comment #17) > I'll have another look before pushing this as you are in principle right > about adding inconsistencies. Maybe Raphael can weigh in here as he might > have a better idea having looked at the code more closely? To fix the whole problem, we would have to change every financial calculation in Koha (in Perl and JavaScript, the database does it right) to use a decimal datatype. I think it is not feasible to do it all at once. So, we should identify isolated parts of Koha that could be fixed independently without introducing new inconsistencies and fix those parts separately. The patch in this bug fixes a rounding function that is used in several parts of Koha and would introduce at least one inconsistency in the acquisition module (see bug 37937). I would not push this patch until we think that no new inconsistencies are introduced by it. I wrote this bug report mainly to point out that there is a problem with financial calculations in Koha and to provide an example of how this problem can be solved.
Thanks for all the hard work! Pushed to main for the next 24.11.00 release as RM Assistant
Backported to 24.05.x for upcoming 24.05.06