File mover software: move, copy, delete your files with our filemover | Blog | ORDER NOW

Contact Us Now

[contact-form-7 id="1125" title="Contact form 1"]

CSV

Home / Archive by Category "CSV" ( - Page 3)
  • 07 May

    Copy only the modified contents of a file

    Q: We are using Limagito for different projects and we have a query to its functionality. We want to see that it can copy only the modified contents of a file  in a new file with same name. Below is an example on what is being appended and how.

    Let us use the following example: Equipment_AE_Alarm_Event0_2023-05-04_00.00.01.csv.

    • The original content of this csv file:

    “Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
    “2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
    “2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
    “2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
    “2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
    “2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″

    • Expected content of the output file would be the same when we scan this file for the first time:

    “Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
    “2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
    “2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
    “2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
    “2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
    “2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″

    • Data is being added to this csv so the content of file during the next scan is showing more entries:

    “Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
    “2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
    “2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
    “2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
    “2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
    “2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″
    “2023-05-03 00:00:06;””sample text6″”;200;KLMN;0″
    “2023-05-03 00:00:06;””sample text7″”;200;KLMN;1″
    “2023-05-03 00:00:08;””sample text8″”;200;KLMN;0″
    “2023-05-03 00:00:10;””sample text9″”;200;KLMN;1″
    “2023-05-03 00:00:12;””sample text10″”;200;KLMN;0″

    • Expected content of the output file after the second scan should be:

    “Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
    “2023-05-03 00:00:06;””sample text6″”;200;KLMN;0″
    “2023-05-03 00:00:06;””sample text7″”;200;KLMN;1″
    “2023-05-03 00:00:08;””sample text8″”;200;KLMN;0″
    “2023-05-03 00:00:10;””sample text9″”;200;KLMN;1″
    “2023-05-03 00:00:12;””sample text10″”;200;KLMN;0″

     

    A: This should be possible using our scripting option. We created a script that will keep the data of the csv files it already handled in a separate folder (= memory folder). These memory files will be used during next scans to only create output files where the content has new data.

    For the test, we added 3 folders:

    • Input
    • Memory
    • Output

    Be sure the ‘Memory’ folder is a local folder on the system where our File Mover is running.

    Modified contents of a file example

    Our Source will be the Input folder:

    Limagito file mover using WIN as source

    We added ‘*.csv’ as Filename Include filter because we only want to scan for csv files:

    limagito file mover filename filter

    Optional: we changed the ‘Source File Sort Order’ option to ‘File last write date ascending (Old to New)’. This will handle the oldest csv files first.

    limagito file mover scan older files first

    The function is set to ‘Copy Files’:

    limagito file mover function setup

    As Destination we’ll use our ‘Pascal Script’ option:

    You’ll need to adjust the following 2 constant values ( always end the constant with a \ ):

    • ctMemPath ( = path were we’ll keep our memory data )
    • ctOutPath ( = output path)

    We added a timestamp to the output filename. The format we used is ‘YYYYMMDDHHNNSS’. The reason we did this is to prevent data loss (in case the next system would not pickup the output files).

     

    Var
  tmpEntry, tmpText, tmpFileIn, tmpFileMem, tmpFileOut, tmpFileExtIn, tmpFileExtOut: String;
  tmpList, tmpListMem: TStringList;
Const
  ctMemPath = 'C:\Test\Astra\Memory\';
  ctOutPath = 'C:\Test\Astra\Output\';
Begin
  psExitCode:= 0;
  tmpFileIn := psFilePath + psFileName;
  tmpFileMem := ctMemPath + psFileName;
  tmpFileExtIn := ExtractFileExt(psFileName);
  tmpFileExtOut := '.' + FormatDateTime('YYYYMMDDHHNNSS', Now) + tmpFileExtIn;
  tmpFileOut := psStringReplace(psFileName, tmpFileExtIn, tmpFileExtOut);
  tmpFileOut := ctOutPath + tmpFileOut; 
  // Create Var
  tmpList := TStringList.Create; tmpListMem := TStringList.Create;
  Try
    Try 
      tmpList.LoadFromFile(tmpFileIn);
      // Check if file is already available in Memory folder
      If tmpList.Count > 1 Then
      Begin
        tmpEntry := tmpList.Strings[0]; tmpText := tmpList.Text; 
        If FileExists(tmpFileMem) Then
        Begin
          psLogWrite(1, '', psFileName + ' available in memory folder: ' + ctMemPath);                 
          Try
            tmpListMem.LoadFromFile(tmpFileMem);                          
            tmpList.Text := psStringReplace(tmpText, tmpListMem.Text, '');
            // tmpList.Insert(0, tmpEntry);
            psExitCode := 1;         
          Except
            psLogWrite(1, '', 'Error loading data from: ' + tmpFileMem);
          End;          
        End Else
        Begin
          psExitCode := 1;
          psLogWrite(1, '', psFileName + ' not available in memory folder: ' + ctMemPath);               
        End;  
        If tmpList.Count > 1 Then
        Begin  
          // Save to output folder
          If psExitCode = 1 Then 
          Begin
            psExitCode := 0;
            Try
              psLogWrite(1, '', 'Saving data to: ' + tmpFileOut);       
              tmpList.SaveToFile(tmpFileOut);
              psExitCode := 1;
            Except
              psLogWrite(1, '', 'Error saving data to: ' + tmpFileOut);
            End;  
          End;
          // Save to Memory folder      
          If psExitCode = 1 Then
          Begin
            psExitCode := 0;
            Try
              psLogWrite(1, '', 'Saving data to: ' + tmpFileMem);
              tmpList.Text := tmpText; tmpList.Delete(0);       
              tmpList.SaveToFile(tmpFileMem);
              psExitCode := 1;
            Except
              psLogWrite(1, '', 'Error saving data to: ' + tmpFileOut);
            End;          
          End;
        End;        
      End Else
        psLogWrite(1, '', 'No data available in: ' + tmpFileIn);   
    Except
      psLogWrite(1, '', 'Error loading data from: ' + tmpFileIn);
    End;
  Finally
    tmpList.free; tmpListMem.Free;
  End;
End.

    limagito file mover pascal script as destination

    This blog article is just a basic example of the customers request and can be tweaked so that it meets your requirements. Just contact us if you need help.

    Don’t forget to add an extra Rule which will cleanup the Memory folder on regular basis.

    #FileTransfer #csv #scada #MFT

    If you need any info  about this ‘Copy only the modified contents of a file’ request, please let us know.

    Best regards,Limagito Team

     By Limagito-Team CSV Pascal Script  , , 
  •  04 Aug
    Combine csv files to one and add a column to the end of each line 
    Q: Combine csv files to one and add a column to the end of each line.
    1. I have some .CSV files that we receive and want to combine them all into 1 file.
    2. We should also add a column to the end of each line, a new piece of data for each record. The extra data to add will depend on the filename of the cvs file.
      • If the csv filename contains FILEID1 then we should add FILEID1-DATA.
      • If the csv filename contains FILEID2 then we should add FILEID2-DATA.
    Is this possible?
    A: Yes, this is possible using some custom Pascal Script we created for you.
    • Please open our Pascal Script option:
    Limagito File Mover Pascal Script
    • Add and enable the following ‘On Rule Begin’ Pascal Script. This script will clear string var ( psVSA and psVSE ) when the rule begins.
    Begin
  psExitCode:= 1;
  // ... add your code here
  psVSA := '';
  psVSE := '';
End.
    Limagito File Mover On Rule Begin Pascal Script
    • Add and enable the following ‘On Rule End’ Pascal Script. Do not forget to adjust the combined output filename.
    Var
  tmpList: TStringList;
Const
  ctOutputFile = 'C:\Test\Out_Csv\MergeCsvFile.csv';
Begin
  // Init
  psExitCode:= 1;
  // ... add your code here
  If psVSE = '' Then
  Begin
    // No Error Occured
    If Trim(psVSA) <> '' Then
    Begin
      tmpList := TStringList.Create;
      Try
        tmpList.CommaText := psVSA;
        Try
          tmpList.SaveToFile(ctOutputFile);
        Except
          Begin
            psExitCode := 0;
            psLogWrite(1, '', 'Save To File Exception: ' + ctOutputFile);
          End;
        End;
      Finally
        tmpList.Free;
      End;
    End
    Else
      psLogWrite(1, '', 'No csv data to save');
  End
  Else
  Begin
    // Error Occured
    psLogWrite(1, '', 'Error handling csv file(s): ' + psVSE);
  End;
End.
    Limagito File Mover On Rule End Pascal Script
    • Add ‘Pascal Script’ as Destination
    Limagito File Mover Destinations
    Var
  iList: Integer;
  tmpCount: Integer;
  tmpList, tmpListMem: TStringList;
  tmpAddData, tmpEntry: String;
Const
  ctAddHeader = ',"D"';
Begin
  If psVSE <> '' Then
  Begin
    psExitCode:= 0;
    psLogWrite(1, '', 'Skip File due to previous error: ' + psVSE);
  End
  Else
    psExitCode:= 1;
  // Set AddData
  If pos('FILEID1', UpperCase(psFileName)) > 0 then
    tmpAddData := 'FILEID1-DATA'
  Else if pos('FILEID2', UpperCase(psFileName)) > 0 then
    tmpAddData := 'FILEID2-DATA'
  Else
    tmpAddData := '';
  // Load From File
  tmpList := TStringList.Create;
  tmpListMem := TStringList.Create;
  Try
    If Trim(psVSA) <> '' Then
      tmpListMem.CommaText := psVSA;
    Try
      psLogWrite(1, '', 'Load From File: ' + psFilePath + psFileName);
      tmpList.LoadFromFile(psFilePath + psFileName);
      tmpCount := tmpList.Count;
      If tmpCount > 0 Then
      Begin
        For iList := 0 to (tmpCount - 1) Do
        Begin
          tmpEntry := tmpList.Strings[iList];
          If iList = 0 Then
          Begin
            // Header
            If Trim(tmpEntry) <> '' Then
              tmpListMem.Add(tmpEntry + ctAddHeader);
          End
          Else
          Begin
            // Data
            If Trim(tmpEntry) <> '' Then
              tmpListMem.Add(tmpEntry + tmpAddData);
          End;
        End;
        // Update psVSA
        If Trim(tmpListMem.CommaText) <> '' Then
          psVSA := tmpListMem.CommaText;
      End;
    Except
      Begin
        psExitCode := 0;
        psVSE := 'LoadFromFile ' + psFilePath + psFileName + ' Exception';
      End;
    End;
  Finally
    tmpList.Free;
    tmpListMem.Free;
  End;
End.
    Limagito File Mover combine csv files Pascal Script
    If you need any help with this ‘Combine csv files’ request, please let us know.
    Best Regards,
    Limagito Team
    #filetransfer #filemanagement
     By Limagito-Team CSV Pascal Script  
 1 2 3
SEARCH
CATEGORIES
ARCHIVES
TAGS
365 api code command csv date delete drive Eldos email error feature filename filter ftp HL7 install License limagito limagitox log monitor multiple oauth OAuth2 parameter parameters Pascal pdf Remote rename Script service SFTP SharePoint sign SMTP SQL SSH unzip vendor week xls Xml zip
Limagito filemover is an automated file mover solution handling everything from moving, deleting to copying files.
Please don’t hesitate to contact us if you have any questions about our product and services.
DISCOVER OUR SITE
 How to import data into a SQL table where a primary key as guid is needed 
April 21, 2024 no responses
 Generate Certificate Signing Request (CSR) 
April 21, 2024 no responses
 How to export all rules to CSV with column name, source and destination 
April 20, 2024 no responses
 © 2007-2024 IT services & software development by CPsoft // Webdesign & webhosting by Siteplan