• 27 Apr

    Centralized log file in industry standard CSV format

    Q: Could a centralized log file in industry standard CSV format be implemented within the file mover software? This would be a log file that *all* jobs could be configured to log to. We have looked at the individual History log files and even though we could probably combine them into a single, daily file to push up to Snowflake – it appears they are not character delimited and only have a space between columns which would be very difficult if not impossible to parse out into individual fields.

    A: We added a global logging option in version v2023.2.26.0.

    Feedback customer:

    We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?

    Here is the custom message format I was testing with: %{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{MESSAGE}

    It appears the RuleId is always 0, instead of pulling the actual RuleId value
    2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /

    It appears the RuleName is included as a prefix to the message:

    2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2 [SFTP] CheckSFTPConnected, SB Request Path after Login: /

    Preferably this would be a separate tag (ex. %{RULENAME}) and not a prefix on the message so we could control the output better, for example:

    %{DATETIME}|%{LEVEL}|%{LEVELINT}|%{HOSTNAME}|%{RULEID}|%{RULENAME}|%{MESSAGE} would output a message like this:

    2023-04-24 15:05:58|INFO|1|CHHMOVER01|0|cerner – XR HIM2|[SFTP] CheckSFTPConnected, SB Request Path after Login: /

    A: We added some new options and parameters in version v2023.4.27.0

    Limagito file mover global logging

    We added options to remove certain information like “Rule Name” from the default log message. We enabled “Use Custom Message Format” and added the format from the customer as test.

    limagito file mover Centralized log file

    The following predefined variables can be used to produce a custom message output format:

    • DATETIME : Date & time log item occurs
    • DATE : Date log item occurs
    • TIME : Time log item occurs
    • LEVEL : Level or Eventype
    • LEVELINT : Level as numeric
    • MESSAGE : Message sent to logger
    • ENVIRONMENT : Customizable variable (normally Production, Test, etc)
    • PLATFORM : Customizable variable (normally Desktop, Mobile, etc)
    • APPNAME : Customizable variable (by default set as filename without extension)
    • APPVERSION : Application file version
    • APPPATH : Application run path
    • HOSTNAME : Computer name
    • USERNAME : Logged user name
    • OSVERSION : OS version
    • CPUCORES : Number of CPU cores
    • THREADID : Thread ID log item set
    • RULEID : RuleID number
    • RULENAME : RuleName
    • LOGSOURCE : Description of log source

    Logging result:

    limagito file mover runtime log

    TODO:

    We anticipate wanting to ship our log files up to a remote system every 5 or 10 minutes. Since the log file is locked we can’t force this to happen and of course it would not be wise for us to mess with the “active” log file anyway. Instead of (or in addition to) the “Start new log every day”, could there be a “Start new log every __ minutes” and let us adjust the number of minutes before a log file rolls over to a new one?

    #FileTransfer #Logging #MFT

    If you need any info  about this ‘Centralized log file’, please let us know.

    Best regards,

    Limagito Team

    By Limagito-Team Logging ,
  • 24 Apr

    File renaming logic using day of week function

    Q: We need the dates tv listing dates moved to publications dates. This newspaper publishes 3 days a week, Wednesday, Friday and Saturday. In order to automatically flow the graphic onto the page, the file name needs to match the publication date.

    We need the following logic,

    • Wednesday graphic gets Wednesday date,
    • Thursday graphic gets Wednesday date (the day before)
    • Friday graphic gets Friday date
    • Saturday graphic gets Friday date (the day before)
    • Sunday / Monday / Tuesday graphics get the previous Saturday date.

    For example:

    Sunday / Monday / Tuesday dates:

    • PH_20230423.pdf
    • PH_20230424.pdf
    • PH_20230425.pdf

    We’d want renamed to

    • PH_Sun_20230422.pdf
    • PH__Mon_20230422.pdf
    • PH_Tues_20230422.pdf

    Technically the “Sun”, “Mon”, “Tues” could be anything as long as they’re unique and predictable.

    Moving onto the next set:

    • PH_20230426.pdf
    • PH_20230427.pdf

    Would be renamed to

    • PH_Wed_20230426.pdf
    • PH_Thurs_20230426.pdf

    And finally:

    • PH_20230428.pdf
    • PH_20230429.pdf

    Would be renamed to

    • PH_Fri_20230428.pdf
    • PH_Sat_20230428.pdf

     

    A: Yes this is possible using our Mover Pascal Script and Destination RegEx renaming options.

    • Open ‘Pascal Script’ setup

    Limagito File Mover Pascal Script

    • Enable and add the following ‘On Destination’ Pascal Script:
    Var
      tmpLen: Integer;
      tmpDateStr, tmpFilePart: String;
      tmpDate1, tmpDate2: TDateTime;
    Begin
      psExitCode:= 1;
      // ... add your code here
      // PH_20230423.pdf > PH_Sun_20230422.pdf
      tmpDateStr := psFileName;
      tmpDateStr := psStringReplace(tmpDateStr, '.pdf', '');
      tmpLen := Length(tmpDateStr);
      If tmpLen >= 8 Then
      Begin
        tmpDateStr := Copy(tmpDateStr, tmpLen - 8 + 1, 8);
        tmpDate1 := psStrToDate(tmpDateStr, 'YYYYMMDD', #0);
        Case DayOfWeek(tmpDate1) of
        1: // Sunday
          tmpDate2 := psIncDay(tmpDate1, -1);
        2: // Monday
          tmpDate2 := psIncDay(tmpDate1, -2);
        3: // Tuesday
          tmpDate2 := psIncDay(tmpDate1, -3);
        4: // Wednesday
          tmpDate2 := tmpDate1;
        5: // Thursday
          tmpDate2 := psIncDay(tmpDate1, -1);
        6: // Friday
          tmpDate2 := tmpDate1;
        7: // Saturday
          tmpDate2 := psIncDay(tmpDate1, -1);
        Else
          psExitCode := 0; // Error
        End;
        psVSA := FormatDateTime('DDD', tmpDate1);
        psVSB := FormatDateTime('YYYYMMDD', tmpDate2);
        psLogWrite(1, '', 'Date: ' + psVSB + ', Day: ' + psVSA);
      End
      Else
        psLogWrite(1, '', 'Length error of ' + tmpDateStr);
    End.

    Limagito File Mover Pascal Script Day of Week

    • Afterwards, adjust the File Rename option in your Destination setup:
    RegEx:  (.*)_(.*)\.(.*)
    Replacement:  \1_%VSA_%VSB.\3
    Limagito File Mover File Rename using RegEx
    – RunTime Log Result:
    Limagito File Mover RunTime Log

    #FileTransfer

    If you need any info about this  ‘day of week function’, please let us know.

    Best regards,

    Limagito Team

  • 08 Apr

    Rule events on error and on success event memory option

    During a demo we received a question from Arnold if it was possible to trigger a success event only once. This success event memory, as we call it, will be cleared with the next “On error event”. This option was added in version v2023.4.8.0

    Limagito File Mover success event memory

    We already had a kind of “Error event memory” in the previous versions but this one was filename based so we renamed this option to ‘File Error event memory’.  We added a second “Error event memory” where enabled we will only trigger an error event once. This error event memory will be cleared with the next “On success event”.

    Limagito File Mover error event memory

    #FileTransfer

    If you need any info about this  ‘event memory’ option, please let us know.

    Best regards,

    Limagito Team

1 2
SEARCH