more useful sort order for multi-commodity MixedAmounts: treat missing commodities as 0 - #1564

With the new Eq instance, all "zero" mixed amounts are equal, no matter how many Amounts or commodities they contain, as long as all Amounts are zero - is that right ?

That sounds sensible to humans but disruptive to code, I'm surprised you haven't seen more breakage. Will a mixedAmountsReallyEqual be needed ? I guess not ?

Yes, that is correct. I have been running this for a while and haven't encountered any issues, but there are lots of opportunities for tiny problems to sneak in. If you want to be conservative we could leave the typeclass instances unchanged, and make sure we call sortBy maCompare in all the appropriate places.

So I think sorting is done:

1. based on quantities in all commodities, with alphabetically first commodities being more significant and missing commodities assumed to be zero

2. if all quantities are equal, then based on the quantity of the total prices (@@), if any, treating no price or a unit price price as a total price of zero, and ignoring prices' commodities (?)

3. if all those are equal, the sort order is undefined

Did I understand it ?

Okay, let's write this out in full detail.

A mixed amount is a list of amounts, and when we're comparing two mixed amounts we step through them sequentially. Each amount has a key which determines its precedence in the comparison. An amount's key is determined as follows:

1. First, look at the amount's commodity. Amounts with alphabetically earlier commodities have higher precedence. If two amounts have the same commodity, then move on to the next step.
2. Second, look at the commodity of the amount's price, if it has any. Amounts with no price have higher precedence than amounts with a price; if they do have a price, amounts with alphabetically earlier price commodities have higher precedence. If two amounts have both the same commodity and the same price commodity (or lack of price commodity), then move on to the next step.
3. Next look at whether it's a total price or a unit price. Amounts with total price have precedence over amounts with unit price. If they have a unit price, then they are considered in order of the quantity of the unit price.

All that information goes in to determining the key of each Amount in the mixed amount. You now compare the two mixed amounts by stepping through all the amount keys in order of precedence. If a key is present in both mixed amounts, you just compare the quantities of those amounts. If a key is missing from one of the amounts, treat it as being present with quantity 0. If the keys have the same quantity, compare the quantity of their total price, if present, and treat that quantity as 0 if there is no price or if there is a unit price.

Phew…

Phew is right... thanks for writing it out.

It would be good if we can present a simpler model to the user. Either different rules, or a simpler way to explain the above rules. So that it's not too difficult for a human to look at a sorted report and know why it is sorted that way.

PS since it touches our core data type I think this is not something for 1.22. So we can think this over.

As I think you said elsewhere, we could omit some of this detail in user docs. Amount "keys" are a combination of commodity symbol and price, but generally balance reports aggregate differently-priced amounts and don't show prices, so it's good enough to just talk about commodity symbols. Except for the case where all quantities are the same and prices become significant. But probably even that can be glossed over, I doubt it's going to arise in a confusing way in practice.

And of course we're on the fringe of a fringe feature that no-one has really asked for. I think our main goal here is some kind of sensible/intuitive/explainable behaviour when people use --sort with multiple commodities. And secondly, cleaner/more principled internal design.

I'm happy to have this wait for after 1.22.

Most reports don't actually display prices, so the sort order only needs to look at the commodity symbol and quantity. I'm not sure if any reports both display amounts with prices and sort by amount.

It's actually worth noting that this is not actually more complicated than the sorting behaviour in master. The only difference is that the behaviour in master is automatically implemented by the compiler, while our custom behaviour has to be implemented manually.

Existing behaviour:

A mixed amount is a list of amounts, and when we're comparing two mixed amounts we step through them sequentially. Each amount has a key which determines its precedence in the comparison. An amount's key is determined as follows:

1. First, look at the amount's commodity. Amounts with alphabetically earlier commodities have higher precedence. If two amounts have the same commodity, then move on to the next step.
2. Second, look at the commodity of the amount's price, if it has any. Amounts with no price have higher precedence than amounts with a price; if they do have a price, amounts with alphabetically earlier price commodities have higher precedence. If two amounts have both the same commodity and the same price commodity (or lack of price commodity), then move on to the next step.
3. Next look at whether it's a total price or a unit price. Amounts with total price have precedence over amounts with unit price. If they have a unit price, then they are considered in order of the quantity of the unit price.

All that information goes in to determining the key of each Amount in the mixed amount. You now compare the two mixed amounts by stepping through the amount keys of each mixed amount. This is where the behaviour starts to differ.

1. Look at the amount with highest precedence key in each mixed amount. If one of those amounts has higher precedence than the other, this determines the comparison of the keys. If the two keys have the same precedence, then compare the quantities of the two amounts. If those are equal, then compare whether the amounts are multipliers. If those are equal, then compare the styles. If those are equal, compare the quantity of the prices. (Plus some more comparisons on the prices which we don't care about).
2. If step 1 was equal, then repeat it with the amounts with second highest precedence in each mixed amount, and so on.

The only place sorting by amount is used is in MultiBalanceReport, where it explicitly strips prices before sorting. So from a user-visible perspective, the price based stuff is unnecessary.