File Mover Blog

November 12, 2023

How to merge and add filenames to csv datafiles

How to merge and add filenames to csv datafiles

Q: I have a new daily job where I will receive 3 CSV files. I need to merge these together and also add a new field to the data that contains the original filename as I need to split them later on in the process and use the original filename to send back to the client. Also, I may not always get all 3 files so the rule needs to work for receiving 1,2, or all 3 files.

We will run the rule at 18:00 for example and process however many files have been uploaded, then run the rule every 1 hour after to pick up any files that the client has uploaded late. The ideal scenario would be for all 3 files to be uploaded by the client by the scan time, but they want us to be able to pick up any files they may upload late.

The data does not contain headers either. I’m sure I have asked this before but I can’t find the solution if I have, I have found similar and have tried to amend it to suit but failed miserably.

A: We asked and received the following example files:

  • BTW_Invites_20231023_1.csv
  • BTW_Reminders_20231023_1.csv
  • BTW_Results_20231023_1.csv

We used a Windows folder as Source:

limagito file mover windows folder as source

We added the following ‘Include Filename’ Filter because we want to be sure to only process .csv files:

limagito file mover filename filter setup

As Destination we add our ‘Pascal Script’ option:

limagito file mover destination setup

Destination Pascal Script, do not forget to adjust the ctOutputPath const (Path must end with a \ ).

The script will add the filename of the source csv file to the end of each entry of the merged csv file. The Destination merged csv file is determined by stripping parts of the source filename. In our example it will be: ‘BTW_Merged_20231023_1.csv’.

This script is just an example and it can be adjusted so it fits your needs, just let us know.

You can download the script: here

limagito file mover merge and add filenames script

RunTime Log Result:

limagito file mover runtime log result

#csv #filetransfer #filemanagement

If you need any help with this ‘merge and add filenames’ option, please let us know.

Best Regards,

Limagito Team

By Limagito-Team CSV Pascal Script , Share: