Contact Us Now

Q: We are using Limagito for different projects and we have a query to its functionality. We want to see that it can copy only the modified contents of a file  in a new file with same name. Below is an example on what is being appended and how.

Let us use the following example: Equipment_AE_Alarm_Event0_2023-05-04_00.00.01.csv.

• The original content of this csv file:

“Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
“2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
“2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
“2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
“2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
“2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″

• Expected content of the output file would be the same when we scan this file for the first time:

“Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
“2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
“2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
“2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
“2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
“2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″

• Data is being added to this csv so the content of file during the next scan is showing more entries:

“Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
“2023-05-03 00:00:02;””sample text1″”;200;KLMN;1″
“2023-05-03 00:00:03;””sample text2″”;300;KLMN;1″
“2023-05-03 00:00:04;””sample text3″”;100;KLMN;0″
“2023-05-03 00:00:05;””sample text4″”;200;KLMN;1″
“2023-05-03 00:00:06;””sample text5″”;200;KLMN;1″
“2023-05-03 00:00:06;””sample text6″”;200;KLMN;0″
“2023-05-03 00:00:06;””sample text7″”;200;KLMN;1″
“2023-05-03 00:00:08;””sample text8″”;200;KLMN;0″
“2023-05-03 00:00:10;””sample text9″”;200;KLMN;1″
“2023-05-03 00:00:12;””sample text10″”;200;KLMN;0″

• Expected content of the output file after the second scan should be:

“Time;””AlarTxt””;””AlmPrio””;””Almusr””;””acksts”””
“2023-05-03 00:00:06;””sample text6″”;200;KLMN;0″
“2023-05-03 00:00:06;””sample text7″”;200;KLMN;1″
“2023-05-03 00:00:08;””sample text8″”;200;KLMN;0″
“2023-05-03 00:00:10;””sample text9″”;200;KLMN;1″
“2023-05-03 00:00:12;””sample text10″”;200;KLMN;0″

A: This should be possible using our scripting option. We created a script that will keep the data of the csv files it already handled in a separate folder (= memory folder). These memory files will be used during next scans to only create output files where the content has new data.

For the test, we added 3 folders:

• Input
• Memory
• Output

Be sure the ‘Memory’ folder is a local folder on the system where our File Mover is running.

Our Source will be the Input folder:

We added ‘*.csv’ as Filename Include filter because we only want to scan for csv files:

Optional: we changed the ‘Source File Sort Order’ option to ‘File last write date ascending (Old to New)’. This will handle the oldest csv files first.

The function is set to ‘Copy Files’:

As Destination we’ll use our ‘Pascal Script’ option:

You’ll need to adjust the following 2 constant values ( always end the constant with a \ ):

• ctMemPath ( = path were we’ll keep our memory data )
• ctOutPath ( = output path)

We added a timestamp to the output filename. The format we used is ‘YYYYMMDDHHNNSS’. The reason we did this is to prevent data loss (in case the next system would not pickup the output files).

Var
  tmpEntry, tmpText, tmpFileIn, tmpFileMem, tmpFileOut, tmpFileExtIn, tmpFileExtOut: String;
  tmpList, tmpListMem: TStringList;
Const
  ctMemPath = 'C:\Test\Astra\Memory\';
  ctOutPath = 'C:\Test\Astra\Output\';
Begin
  psExitCode:= 0;
  tmpFileIn := psFilePath + psFileName;
  tmpFileMem := ctMemPath + psFileName;
  tmpFileExtIn := ExtractFileExt(psFileName);
  tmpFileExtOut := '.' + FormatDateTime('YYYYMMDDHHNNSS', Now) + tmpFileExtIn;
  tmpFileOut := psStringReplace(psFileName, tmpFileExtIn, tmpFileExtOut);
  tmpFileOut := ctOutPath + tmpFileOut; 
  // Create Var
  tmpList := TStringList.Create; tmpListMem := TStringList.Create;
  Try
    Try 
      tmpList.LoadFromFile(tmpFileIn);
      // Check if file is already available in Memory folder
      If tmpList.Count > 1 Then
      Begin
        tmpEntry := tmpList.Strings[0]; tmpText := tmpList.Text; 
        If FileExists(tmpFileMem) Then
        Begin
          psLogWrite(1, '', psFileName + ' available in memory folder: ' + ctMemPath);                 
          Try
            tmpListMem.LoadFromFile(tmpFileMem);                          
            tmpList.Text := psStringReplace(tmpText, tmpListMem.Text, '');
            // tmpList.Insert(0, tmpEntry);
            psExitCode := 1;         
          Except
            psLogWrite(1, '', 'Error loading data from: ' + tmpFileMem);
          End;          
        End Else
        Begin
          psExitCode := 1;
          psLogWrite(1, '', psFileName + ' not available in memory folder: ' + ctMemPath);               
        End;  
        If tmpList.Count > 1 Then
        Begin  
          // Save to output folder
          If psExitCode = 1 Then 
          Begin
            psExitCode := 0;
            Try
              psLogWrite(1, '', 'Saving data to: ' + tmpFileOut);       
              tmpList.SaveToFile(tmpFileOut);
              psExitCode := 1;
            Except
              psLogWrite(1, '', 'Error saving data to: ' + tmpFileOut);
            End;  
          End;
          // Save to Memory folder      
          If psExitCode = 1 Then
          Begin
            psExitCode := 0;
            Try
              psLogWrite(1, '', 'Saving data to: ' + tmpFileMem);
              tmpList.Text := tmpText; tmpList.Delete(0);       
              tmpList.SaveToFile(tmpFileMem);
              psExitCode := 1;
            Except
              psLogWrite(1, '', 'Error saving data to: ' + tmpFileOut);
            End;          
          End;
        End;        
      End Else
        psLogWrite(1, '', 'No data available in: ' + tmpFileIn);   
    Except
      psLogWrite(1, '', 'Error loading data from: ' + tmpFileIn);
    End;
  Finally
    tmpList.free; tmpListMem.Free;
  End;
End.

This blog article is just a basic example of the customers request and can be tweaked so that it meets your requirements. Just contact us if you need help.

Don’t forget to add an extra Rule which will cleanup the Memory folder on regular basis.

#FileTransfer #csv #scada #MFT