Q: Could a centralized log file in industry standard CSV format be implemented within the file mover software? This would be a log file that *all* jobs could be configured to log to. We have looked at the individual History log files and even though we could probably combine them into a single, daily file to push up to Snowflake – it appears they are not character delimited and only have a space between columns which would be very difficult if not impossible to parse out into individual fields.
A: We added a global logging option in version v2023.2.26.0.
Feedback customer:
We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?
Here is the custom message format I was testing with: %{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{MESSAGE}
It appears the RuleId is always 0, instead of pulling the actual RuleId value
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /
It appears the RuleName is included as a prefix to the message:
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /
Preferably this would be a separate tag (ex. %{RULENAME}) and not a prefix on the message so we could control the output better, for example:
%{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{RULENAME}|%{MESSAGE} would output a message like this:
2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2|[SFTP] CheckSFTPConnected, SB Request Path after Login: /
A: We added some new options and parameters in version v2023.4.27.0
We added options to remove certain information like “Rule Name” from the default log message. We enabled “Use Custom Message Format” and added the format from the customer as test.
The following predefined variables can be used to produce a custom message output format:
- DATETIME : Date & time log item occurs
- DATE : Date log item occurs
- TIME : Time log item occurs
- LEVEL : Level or Eventype
- LEVELINT : Level as numeric
- MESSAGE : Message sent to logger
- ENVIRONMENT : Customizable variable (normally Production, Test, etc)
- PLATFORM : Customizable variable (normally Desktop, Mobile, etc)
- APPNAME : Customizable variable (by default set as filename without extension)
- APPVERSION : Application file version
- APPPATH : Application run path
- HOSTNAME : Computer name
- USERNAME : Logged user name
- OSVERSION : OS version
- CPUCORES : Number of CPU cores
- THREADID : Thread ID log item set
- RULEID : RuleID number
- RULENAME : RuleName
- LOGSOURCE : Description of log source
Logging result:
TODO:
We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?
#FileTransfer #Logging #MFT
Best regards,
Limagito Team
