April 2023 - File transfer tool from Limagito

27 Apr
Centralized log file in industry standard CSV format
Q: Could a centralized log file in industry standard CSV format be implemented within the file mover software? This would be a log file that *all* jobs could be configured to log to. We have looked at the individual History log files and even though we could probably combine them into a single, daily file to push up to Snowflake – it appears they are not character delimited and only have a space between columns which would be very difficult if not impossible to parse out into individual fields.
A: We added a global logging option in version v2023.2.26.0.
Feedback customer:
We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?
Here is the custom message format I was testing with: %{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{MESSAGE}
It appears the RuleId is always 0, instead of pulling the actual RuleId value
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /
It appears the RuleName is included as a prefix to the message:
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /
Preferably this would be a separate tag (ex. %{RULENAME}) and not a prefix on the message so we could control the output better, for example:
%{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{RULENAME}|%{MESSAGE} would output a message like this:
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2|[SFTP] CheckSFTPConnected, SB Request Path after Login: /
A: We added some new options and parameters in version v2023.4.27.0
We added options to remove certain information like “Rule Name” from the default log message. We enabled “Use Custom Message Format” and added the format from the customer as test.
The following predefined variables can be used to produce a custom message output format:
DATETIME : Date & time log item occurs
DATE : Date log item occurs
TIME : Time log item occurs
LEVEL : Level or Eventype
LEVELINT : Level as numeric
MESSAGE : Message sent to logger
ENVIRONMENT : Customizable variable (normally Production, Test, etc)
PLATFORM : Customizable variable (normally Desktop, Mobile, etc)
APPNAME : Customizable variable (by default set as filename without extension)
APPVERSION : Application file version
APPPATH : Application run path
HOSTNAME : Computer name
USERNAME : Logged user name
OSVERSION : OS version
CPUCORES : Number of CPU cores
THREADID : Thread ID log item set
RULEID : RuleID number
RULENAME : RuleName
LOGSOURCE : Description of log source
Logging result:
TODO:
We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?
#FileTransfer #Logging #MFT
If you need any info about this ‘Centralized log file’, please let us know.
Best regards,
Limagito Team
By Limagito-Team Logging Centralized, log file
24 Apr
File renaming logic using day of week function
Q: We need the dates tv listing dates moved to publications dates. This newspaper publishes 3 days a week, Wednesday, Friday and Saturday. In order to automatically flow the graphic onto the page, the file name needs to match the publication date.
We need the following logic,
Wednesday graphic gets Wednesday date,
Thursday graphic gets Wednesday date (the day before)
Friday graphic gets Friday date
Saturday graphic gets Friday date (the day before)
Sunday / Monday / Tuesday graphics get the previous Saturday date.
For example:
Sunday / Monday / Tuesday dates:
PH_20230423.pdf
PH_20230424.pdf
PH_20230425.pdf
We’d want renamed to
PH_Sun_20230422.pdf
PH__Mon_20230422.pdf
PH_Tues_20230422.pdf
Technically the “Sun”, “Mon”, “Tues” could be anything as long as they’re unique and predictable.
Moving onto the next set:
PH_20230426.pdf
PH_20230427.pdf
Would be renamed to
PH_Wed_20230426.pdf
PH_Thurs_20230426.pdf
And finally:
PH_20230428.pdf
PH_20230429.pdf
Would be renamed to
PH_Fri_20230428.pdf
PH_Sat_20230428.pdf

A: Yes this is possible using our Mover Pascal Script and Destination RegEx renaming options.
Open ‘Pascal Script’ setup
Enable and add the following ‘On Destination’ Pascal Script:
Var tmpLen: Integer; tmpDateStr, tmpFilePart: String; tmpDate1, tmpDate2: TDateTime; Begin psExitCode:= 1; // ... add your code here // PH_20230423.pdf > PH_Sun_20230422.pdf tmpDateStr := psFileName; tmpDateStr := psStringReplace(tmpDateStr, '.pdf', ''); tmpLen := Length(tmpDateStr); If tmpLen >= 8 Then Begin tmpDateStr := Copy(tmpDateStr, tmpLen - 8 + 1, 8); tmpDate1 := psStrToDate(tmpDateStr, 'YYYYMMDD', #0); Case DayOfWeek(tmpDate1) of 1: // Sunday tmpDate2 := psIncDay(tmpDate1, -1); 2: // Monday tmpDate2 := psIncDay(tmpDate1, -2); 3: // Tuesday tmpDate2 := psIncDay(tmpDate1, -3); 4: // Wednesday tmpDate2 := tmpDate1; 5: // Thursday tmpDate2 := psIncDay(tmpDate1, -1); 6: // Friday tmpDate2 := tmpDate1; 7: // Saturday tmpDate2 := psIncDay(tmpDate1, -1); Else psExitCode := 0; // Error End; psVSA := FormatDateTime('DDD', tmpDate1); psVSB := FormatDateTime('YYYYMMDD', tmpDate2); psLogWrite(1, '', 'Date: ' + psVSB + ', Day: ' + psVSA); End Else psLogWrite(1, '', 'Length error of ' + tmpDateStr); End.
Afterwards, adjust the File Rename option in your Destination setup:
RegEx: (.*)_(.*)\.(.*)
Replacement: \1_%VSA_%VSB.\3
– RunTime Log Result:
#FileTransfer
If you need any info about this ‘day of week function’, please let us know.
Best regards,
Limagito Team
By Limagito-Team Pascal Script Rename dayoftheweek, dayofweek
08 Apr
Rule events on error and on success event memory option
During a demo we received a question from Arnold if it was possible to trigger a success event only once. This success event memory, as we call it, will be cleared with the next “On error event”. This option was added in version v2023.4.8.0
We already had a kind of “Error event memory” in the previous versions but this one was filename based so we renamed this option to ‘File Error event memory’. We added a second “Error event memory” where enabled we will only trigger an error event once. This error event memory will be cleared with the next “On success event”.
#FileTransfer
If you need any info about this ‘event memory’ option, please let us know.
Best regards,
Limagito Team
By Limagito-Team News error event memory, event memory, success event memory