CSV import only displays 21 rows since version 1.5.1

General discussion on all other topics under the sun.

Moderator: Renato

Post Reply
Chacsam
New MMEX User
Posts: 6
Joined: Sun Sep 07, 2014 5:50 am
Are you a spam bot?: No

CSV import only displays 21 rows since version 1.5.1

Post by Chacsam »

Hi,
I have been working with MMEX since 2013 and it's nice to see it improving regularly.

For years, I have been working with CSV imports I preprocess in Excel to ensure format consistency (and allow me to include "standardized" payees in the process) and it's been working fine.
My CSV has always been 50-row long (formulas) and I just used the "Rows to ignore function" (1 from beginning: the title, x from the end depending on content).
Unfortunately since 1.5.1, that window only displays and allows me to remove 21 rows. The rows beyond row #21 are always imported with empty date & content if the row is blank in the csv.
Anyone esle got a similar issue? (Windows 10, 64-bit)
MartinArmstrong
MVP MMEX User
Posts: 300
Joined: Tue Mar 02, 2021 10:24 am
Are you a spam bot?: No

Re: CSV import only displays 21 rows since version 1.5.1

Post by MartinArmstrong »

@Chacsam

You are correct in saying that in 1.5.1 only 21 rows are displayed in the Preview pane of the Import CSV dialog, however the setting of 'Rows to ignore From start:' and 'From end:' are correctly applied to the beginning and end of the whole CSV file.

I have attached a 50 row CSV file in which the Amount varies from 1 to 50 so you can easily see which rows are imported.

The attached screenshot of the Import dialog shows that I want to ignore 3 rows from the start and 10 rows from the end.

The Preview pane is misleading in that it now only displays only 21 rows shows that the 10 rows to be ignored at the end are rows 12 to 21. However this does not affect the import from the CSV file. The last 10 rows of the file (i.e 41 to 50) will be ignored.

When I do the Import, you can see from the attached Import success dialog that 50 - 3 - 10 = 37 rows are imported.
And from the attached Transactions screenshot you can see that the correct rows have been imported.

Is this different from the behaviour that you are seeing in 1.5.1? If so, could you please post a copy of your CSV file and say how many rows you are ignoring from the start and the end.

You are correct that the behaviour (of the Preview pane at least) has changed between 1.3.6 where it shows all rows from the CSV and 1.5.1 where it shows only 21. I will raise a bug report in github for this. [Edit: Issue #3487 raised]

Fifty row import.csv
(1.11 KiB) Downloaded 157 times
Import dialog.png
Import dialog.png (75.87 KiB) Viewed 2334 times
Import success.png
Import success.png (18.15 KiB) Viewed 2334 times
Transactions.png
Transactions.png (80.15 KiB) Viewed 2334 times
Chacsam
New MMEX User
Posts: 6
Joined: Sun Sep 07, 2014 5:50 am
Are you a spam bot?: No

Re: CSV import only displays 21 rows since version 1.5.1

Post by Chacsam »

Thank You @MartinArmstrong

You are right, on the contrary of what the preview might suggest, the lines are skipped as from the last one (ie 50) and not line 21.
I never noticed before as I only skipped blank line.
So it's only a "preview display" bug.

Nevertheless it has an additional consequence -to me- it's impossible to skip more than 21 rows from the end.
Thanks for raising the issue!
MartinArmstrong
MVP MMEX User
Posts: 300
Joined: Tue Mar 02, 2021 10:24 am
Are you a spam bot?: No

Re: CSV import only displays 21 rows since version 1.5.1

Post by MartinArmstrong »

@Chacsam

In the latest Alpha build, Nikolay has removed the 21 row limit in the Preview.

If there are more than 100 rows in the file, Preview will now show the first 50 rows at the start of its list, and the last 50 rows from the file at the end of the list. Import will also now allow you to ignore up to 100 rows from the end of the file (although when you go over 50, the Preview will obviously look wrong - but the import will ignore the correct lines.

The Ignore rows from start/end is really provided to deal general one is to have a bank CSV file with...

Code: Select all

Preamble line 1 (e.g Account name)
Preamble line 2 (e.g. Date of download)
Preamble line ...
Header line (e.g. "Date, Payee, Deposit, Withdrawal, etc")
Transaction data line 1
Transaction data line 2
...
Transaction data line N
Postamble line 1
Postamble line 2
Postamble line ...
I think that yours is an unusual use case. How many rows do you want to ignore from the end of your files?

If you need to ignore more than the 100 provided in the Alpha build, would it not be better to just delete those rows from your CSV file before importing?
MartinArmstrong
MVP MMEX User
Posts: 300
Joined: Tue Mar 02, 2021 10:24 am
Are you a spam bot?: No

Re: CSV import only displays 21 rows since version 1.5.1

Post by MartinArmstrong »

I just realised that in your original post you said:

"The rows beyond row #21 are always imported with empty date & content if the row is blank in the csv"

Why are you having to set Import up to ignore blank lines? Blanks are automatically skipped:

Blank lines skipped.png
Blank lines skipped.png (20.84 KiB) Viewed 2264 times

The above was from a csv with 20 rows of data and 7 blank lines at the end. I didn't set 'Rows to ignore From end'.

If this doesn't work for you, then please could you post your CSV file here.
Nikolay
MMEX Developer
Posts: 1535
Joined: Sat Dec 06, 2008 2:27 pm
Are you a spam bot?: No
Location: Sankt-Petersburg, Russia

Re: CSV import only displays 21 rows since version 1.5.1

Post by Nikolay »

Chacsam
New MMEX User
Posts: 6
Joined: Sun Sep 07, 2014 5:50 am
Are you a spam bot?: No

Re: CSV import only displays 21 rows since version 1.5.1

Post by Chacsam »

Solved in 1.5.2, thanks!
Post Reply