csv import: 4 rule file accepts a record of only 3 fields

If you're not sure this is a bug, or if some discussion would help,
**
Brief chat on hledger.irc Jan 7th 16:08-16:56
**

Have you looked at (the right version of) the user manual?
Yes

A small reproducible example (--anon can help anonymise data)

I created my CSV data file and inadvertently missed a ',' after the payee name in record 2.
CSV data:

2021-12-22,garo_bike ;  ebici,expenses:trasporto:ebici,-50.0
2021-12-23,caffe_siciliaexpenses:cibo:dolce,-10.5
2021-12-27,simone ; pane:simone,expenses:cibo:pane,-4.8

RULES file:

account1 assets:bank:checking
fields date, description, account2, amount

CMD and result:

I then did a --dry-run:
An noticed the strange output in the 2nd posting

$ hledger import --dry-run --rules-file=./inc_exp2018_ods.csv.rules csv_missing_delim.csv
; would import 3 new transactions from csv_missing_delim.csv:

2021-12-22 garo_bike ;  ebici
    assets:bank:checking               -50.0
    expenses:trasporto:ebici            50.0

2021-12-23 caffe_siciliaexpenses:cibo:dolce
    assets:bank:checking              %4
    -10.5                            %-4

2021-12-27 simone ; pane:simone
    assets:bank:checking            -4.8
    expenses:cibo:pane               4.8

The output of hledger --version
1.24, linux-x86_64

What platform you're on
Linux

This is the output from hledger import ... --debug 7

ignored_delim_1.txt

And this is with --debug 6:

$ hledger import --dry-run --rules-file=./inc_exp2018_ods.csv.rules csv_missing_delim.csv --debug=6      trying reader:"journal"
trying reader:"csv"
using conversion rules file:"./inc_exp2018_ods.csv.rules"
csv rules:
 CsvRules
   { rdirectives = []
   , rcsvfieldindexes =
     [ ( "date", 1 ), ( "description", 2 ), ( "account2", 3 ), ( "amount", 4 ) ]
   , rassignments =
     [
       ( "account1", "assets:bank:checking" )
     ,
       ( "account2", "%3" )
     ,
       ( "amount", "%4" )
     ,
       ( "date", "%1" )
     ,
       ( "description", "%2" )
     ]
   , rconditionalblocks = []
   }
using separator:','
first 3 csv records:
 [
   [ "2021-12-22", "garo_bike ;  ebici", "expenses:trasporto:ebici", "-50.0" ]
 ,
   [ "2021-12-23", "caffe_siciliaexpenses:cibo:dolce", "-10.5" ]
 ,
   [ "2021-12-27", "simone ; pane:simone", "expenses:cibo:pane", "-4.8" ]
 ]
newestfirst:False
mdataseemsnewestfirst:Just False
; would import 3 new transactions from csv_missing_delim.csv:

2021-12-22 garo_bike ;  ebici
    assets:bank:checking               -50.0
    expenses:trasporto:ebici            50.0

2021-12-23 caffe_siciliaexpenses:cibo:dolce
    assets:bank:checking              %4
    -10.5                            %-4

2021-12-27 simone ; pane:simone
    assets:bank:checking            -4.8
    expenses:cibo:pane               4.8

@simonmichael

I've looked into this a little bit. The defined behaviour is that if there is a missing field, it will be left as a reference to the field position (in this case %4). This is then being interpreted as an amount, hence the somewhat silly %-4 balance.

The amount, on the other hand, is interpreted as the account name: -10.5. This is not a valid account name (they are required to have each part starting with an alphanumeric charcacter), but the parser accepts it for backward compatibility reasons. @simonmichael: Do we still need to maintain backward compatibility here? Can we tighten up our account name parser?

But even tightening the account name parser would not completely solve this problem, as a positive number is a completely valid account name.

@agander

Thanks for the report @agander.
@Xitian9 yes I see no reason not to tighten up the account parsing there.
And I think it's a bug that %4 is shown there, showing nothing would be more intuitive.

imp: csv: Give an error if unable to substitute csv templates. (hledg…

@agander

@agander, any thoughts on #1814 ?

imp: csv: Give an error if unable to substitute csv templates. (hledg…

imp: csv: Substitute %nosuchfield if csv template fails. (hledgerorg#…

imp: csv: Substitute empty string if csv template fails. (hledgerorg#…

Merge pull request #1814 from Xitian9/csverror

@agander

@agander a fix has landed in master, in case you want to try it.

hledger, hledger-lib: Update to 1.28

Navigation Menu

Repository navigation

csv import: 4 rule file accepts a record of only 3 fields #1803

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

csv import: 4 rule file accepts a record of only 3 fields #1803

Description

Activity

agander commented on Jan 7, 2022

agander commented on Jan 7, 2022

Xitian9 commented on Jan 7, 2022

simonmichael commented on Jan 26, 2022

simonmichael commented on Feb 7, 2022

simonmichael commented on May 22, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions