File mover software: move, copy, delete your files with our filemover | Blog | ORDER NOW

Contact Us Now

[contact-form-7 id="1125" title="Contact form 1"]

encoding

Home / Posts Tagged "encoding"
  • 10 Feb

    Move and change content of a txt-file

    Q: Hello Limagito, Now I need your help again…

    I got a TXT-file delivered from my customer with approximately 250 rows.

    In the txt file I like “search and replace” a text string :

    ;19;18;;

    Should be replaced with

    ;36;35;;

    And the string

    ;21;21;;

    Should be replaced with

    ;37;37;;

    I also tried to set the powershell exe file in the field for Application name (C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe)

    Is there any other way?

    A: Yes there is, we can do this easily with our Pascal Script so no need for external scripts if you want.

    In our example we’ll use a Windows folder as Source:

    Limagito File Mover WIN as Source

    We only want to handle .txt file so we added the following Filename Include Filter:

    Limagito File Mover Include Filename Filter

    As Destination we added a Pascal Script. This script will does the work you requested.

    Limagito File Mover Pascal Script as Destination

    • Adjust ctOutputPath > must end with a \
    • Encoding of destination text file can be adjusted

     

    Var
  tmpFileIn, tmpFileOut: String;
  tmpList: TStringList;
Const
  ctOutputPath = 'C:\Test\Out_Txt\';
Begin
  // Init var
  psExitCode:= 0;
  tmpFileIn := psFilePath + psFileName;
  tmpFileOut := ctOutputPath + psFileName; 
  // ... add your code here
  tmpList := TStringList.Create;
  Try
    Try
      tmpList.LoadFromFile(tmpFileIn);
      tmpList.Text := psStringReplace(tmpList.Text, ';19;18;;', ';36;35;;');
      tmpList.Text := psStringReplace(tmpList.Text, ';21;21;;', ';37;37;;');
      // https://docs.microsoft.com/nl-be/windows/win32/intl/code-page-identifiers
      If psSaveTxt2File(tmpList.Text, ctOutputPath + psFileName, 'ISO-8859-1', False) Then
      Begin
        psExitCode := 1;
        psLogWrite(1, '', 'SaveToFile ' + tmpFileOut + ' Successful');    
      End
      Else
        psLogWrite(1, '', 'SaveToFile ' + tmpFileOut + ' Error');
    Except
      psLogWrite(1, '', 'LoadFromFile ' + tmpFileIn + ' Exception');
    End;
  Finally
    tmpList.Free;
  End;  
End.

    Limagito File Mover Pascal Script as Destination

    Feedback Customer:

    As always you save the day! The Pascal script did my work easily. Some day I have to learn more about pascal!

    I cheat some in powershell and some visual basic, but this stuff is powerful!

    By the way, it only worked in the newer version of Limagito (I still have some workflows in the old version)

    Thank you for help, quick and accurate, as always!

    #FileTransfer #Encoding

    If you need any info about this ‘change content of a txt-file’ request, please let us know.

    Best regards,

    Limagito Team

    By Limagito-Team Pascal Script WIN XLS/CSV ,
  • 17 Jul

    Convert character set of text file

    Q: Convert character set of text file. I think I need some help here. We connect to a SFTP-server and download 3 files. One of these are a UTF-8 file named Vyymmdd.Thhmmss (i.e. V220627.TT110759). We need to download all 3 files, and convert the UTF-8 file to ISO-8859-1. I have found this article: https://limagito.com/text-character-encoding-conversion/

    And edited like this : psChangeTxtEncodingExt(psFilePath + psFileName, ctOutputpath + psFileName, 65001, 28591, True)

    Runtime Log says the file was successfully converted. But if I open the file in Notepad++ and click the “Encoding” option on the menu, it highlights UTF-8.

    A: In version v2022.7.17.0 we added some extra functions to help you convert the character set of  text files:

    Function psChangeTxtEncodingCk(Source, Destination: String; SrcCharset, DstCharset: String; WriteBOM: Boolean; ErrorAction: Integer): Boolean;

    Do not use this function when Source and Destination file are the same, use psChangeTxtEncodingCkExt instead.

    • ErrorAction: Controls how errors are handled. When a character in the input data cannot be converted to the target charset, the action taken is controlled by this property.
      The possible settings are:
      (0) drop the error characters,
      (1) substitute the data set by a ?
      (2) convert to a hex-escaped string (&#xXXXX)
      (3) RESERVED
      (4) RESERVED
      (5) RESERVED
      (6) RESERVED
      (7) Pass non-convertible characters to the output unchanged.
    • The Source and Destination Charset can be any of the supported encodings listed at Chilkat Supported Character Encodings.

    Function psChangeTxtEncodingCkExt(Source, Destination: String; SrcCharset, DstCharset: String; WriteBOM, WriteFileIfModified: Boolean): String;

      • WriteFileIfModified: writes the contents to a file, but only if it is a new file or if the contents are different than the existing file.
      • WriteBOM: the BOM (also known as a preamble), is emitted for charsets that define a BOM (such as utf-8, utf-16, utf-32, etc.)
      • The Source and Destination Charset can be any of the supported encodings listed at Chilkat Supported Character Encodings.

    #Filetransfer #Filemanagement #Encoding

    If you need any info about this ‘Convert character set’ question, please let us know.

    Best regards,

    Limagito Team

     By Limagito-Team Pascal Script  , , , 
  •  19 May
    Text Character Encoding Conversion 
    Dear Users,
    In v2019.05.19.0 we’ve added an option to convert the character encoding of text files.
    To achieve this we’ve added the following Pascal Script: psChangeTxtEncodingExt
    Function psChangeTxtEncodingExt(Source, Destination: String; SrcEncoding, DstEncoding: Integer; WriteBOM: Boolean): Boolean;
    • Source: Source File
    • Destination: Destination File
    • SrcEncoding: Encoding ID Source (please check list below)
    • DstEncoding: Encoding ID Destination (please check list below)
    • WriteBOM: Write BOM Destination File (Normally True)
    The function is also able to read the encoding tag of xml source files and automatically set the source encoding ID.
     
    In the following example we’re going to convert EBCDIC Codepage 273 (German) text files to UTF8.
    > Be sure to add and set a file filter so it will only handle text files (txt, xml, …)
    >Add a Pascal Script as Destination
    Const
  ctOutputPath = 'C:\Test\In_Patrick\Out\';
Begin
  psExitCode:= 0;
  // Source File: EBCDIC in EBCDIC Codepage 273 (German) or Codepage 1141 (German incl. EUR-Sign)
  // Destination File: UTF8
  // https://limagito.com/text-character-encoding-conversion/
 
  If psChangeTxtEncodingExt(psFilePath + psFileName, ctOutputpath + psFileName, 20273, 65001, True) Then
  Begin
    psLogWrite(1, '', 'Conversion to ' + ctOutputpath + psFileName + ' Successful');
    psExitCode:= 1;
  End;
 
End.
    > RunTime Log Result
    The function is able to convert quite some encodings:
    We have our own Encoding ID’s:
    EncodingID Additional information 
0          UTF8
1          UTF7
2          Unicode
3          Default
4          Big Endian Unicode
5          ASCII
6          ANSI
7          Reserved
8          Reserved
9          XML Auto Encoding detection (Source Only)
    But also all Windows Encoding ID’s can be used:
    Code Page Identifiers
    EncodingID  Additional information
37          IBM037, IBM EBCDIC US-Canada
437         IBM437, OEM United States
500         IBM500, IBM EBCDIC International
708         ASMO-708, Arabic (ASMO 708)
709         ASMO-709, Arabic ASMO-449+, BCON V4
710         ASMO-710, Arabic Transparent Arabic
720         DOS-720, Arabic (Transparent ASMO); Arabic (DOS)
737         IBM737, OEM Greek (formerly 437G); Greek (DOS)
775         IBM775, OEM Baltic; Baltic (DOS)
850         IBM850, OEM Multilingual Latin 1; Western European (DOS)
852         IBM852, OEM Latin 2; Central European (DOS)
855         IBM855, OEM Cyrillic (primarily Russian)
857         IBM857, OEM Turkish; Turkish (DOS)
858         IBM858, OEM Multilingual Latin 1 + Euro symbol
860         IBM860, OEM Portuguese; Portuguese (DOS)
861         IBM861, OEM Icelandic; Icelandic (DOS)
862         DOS-862, OEM Hebrew; Hebrew (DOS)
863         IBM863, OEM French Canadian; French Canadian (DOS)
864         IBM864, OEM Arabic; Arabic (864)
865         IBM865, OEM Nordic; Nordic (DOS)
866         cp866, OEM Russian; Cyrillic (DOS)
869         IBM869, OEM Modern Greek; Greek, Modern (DOS)
870         IBM870, IBM EBCDIC Multilingual/ROECE (Latin 2); IBM EBCDIC Multilingual Latin 2
874         windows-874, ANSI/OEM Thai (ISO 8859-11); Thai (Windows)
875         cp875, IBM EBCDIC Greek Modern
932         shift_jis, ANSI/OEM Japanese; Japanese (Shift-JIS)
936         gb2312, ANSI/OEM Simplified Chinese (PRC, Singapore); Chinese Simplified (GB2312)
949         ks_c_5601-1987, ANSI/OEM Korean (Unified Hangul Code)
950         big5, ANSI/OEM Traditional Chinese (Taiwan; Hong Kong SAR, PRC); Chinese Traditional (Big5)
1026        IBM1026, IBM EBCDIC Turkish (Latin 5)
1047        IBM01047, IBM EBCDIC Latin 1/Open System
1140        IBM01140, IBM EBCDIC US-Canada (037 + Euro symbol); IBM EBCDIC (US-Canada-Euro)
1141        IBM01141, IBM EBCDIC Germany (20273 + Euro symbol); IBM EBCDIC (Germany-Euro)
1142        IBM01142, IBM EBCDIC Denmark-Norway (20277 + Euro symbol); IBM EBCDIC (Denmark-Norway-Euro)
1143        IBM01143, IBM EBCDIC Finland-Sweden (20278 + Euro symbol); IBM EBCDIC (Finland-Sweden-Euro)
1144        IBM01144, IBM EBCDIC Italy (20280 + Euro symbol); IBM EBCDIC (Italy-Euro)
1145        IBM01145, IBM EBCDIC Latin America-Spain (20284 + Euro symbol); IBM EBCDIC (Spain-Euro)
1146        IBM01146, IBM EBCDIC United Kingdom (20285 + Euro symbol); IBM EBCDIC (UK-Euro)
1147        IBM01147, IBM EBCDIC France (20297 + Euro symbol); IBM EBCDIC (France-Euro)
1148        IBM01148, IBM EBCDIC International (500 + Euro symbol); IBM EBCDIC (International-Euro)
1149        IBM01149, IBM EBCDIC Icelandic (20871 + Euro symbol); IBM EBCDIC (Icelandic-Euro)
1200        utf-16, Unicode UTF-16, little endian byte order (BMP of ISO 10646); available only to managed applications
1201        unicodeFFFE, Unicode UTF-16, big endian byte order; available only to managed applications
1250        windows-1250, ANSI Central European; Central European (Windows)
1251        windows-1251, ANSI Cyrillic; Cyrillic (Windows)
1252        windows-1252, ANSI Latin 1; Western European (Windows)
1253        windows-1253, ANSI Greek; Greek (Windows)
1254        windows-1254, ANSI Turkish; Turkish (Windows)
1255        windows-1255, ANSI Hebrew; Hebrew (Windows)
1256        windows-1256, ANSI Arabic; Arabic (Windows)
1257        windows-1257, ANSI Baltic; Baltic (Windows)
1258        windows-1258, ANSI/OEM Vietnamese; Vietnamese (Windows)
1361        Johab, Korean (Johab)
10000       macintosh, MAC Roman; Western European (Mac)
10001       x-mac-japanese, Japanese (Mac)
10002       x-mac-chinesetrad, MAC Traditional Chinese (Big5); Chinese Traditional (Mac)
10003       x-mac-korean, Korean (Mac)
10004       x-mac-arabic, Arabic (Mac)
10005       x-mac-hebrew, Hebrew (Mac)
10006       x-mac-greek, Greek (Mac)
10007       x-mac-cyrillic, Cyrillic (Mac)
10008       x-mac-chinesesimp, MAC Simplified Chinese (GB 2312); Chinese Simplified (Mac)
10010       x-mac-romanian, Romanian (Mac)
10017       x-mac-ukrainian, Ukrainian (Mac)
10021       x-mac-thai, Thai (Mac)
10029       x-mac-ce, MAC Latin 2; Central European (Mac)
10079       x-mac-icelandic, Icelandic (Mac)
10081       x-mac-turkish, Turkish (Mac)
10082       x-mac-croatian, Croatian (Mac)
12000       utf-32, Unicode UTF-32, little endian byte order; available only to managed applications
12001       utf-32BE, Unicode UTF-32, big endian byte order; available only to managed applications
20000       x-Chinese_CNS, CNS Taiwan; Chinese Traditional (CNS)
20001       x-cp20001, TCA Taiwan
20002       x_Chinese-Eten, Eten Taiwan; Chinese Traditional (Eten)
20003       x-cp20003, IBM5550 Taiwan
20004       x-cp20004, TeleText Taiwan
20005       x-cp20005, Wang Taiwan
20105       x-IA5, IA5 (IRV International Alphabet No. 5, 7-bit); Western European (IA5)
20106       x-IA5-German, IA5 German (7-bit)
20107       x-IA5-Swedish, IA5 Swedish (7-bit)
20108       x-IA5-Norwegian, IA5 Norwegian (7-bit)
20127       us-ascii, US-ASCII (7-bit)
20261       x-cp20261, T.61
20269       x-cp20269, ISO 6937 Non-Spacing Accent
20273       IBM273, IBM EBCDIC Germany
20277       IBM277, IBM EBCDIC Denmark-Norway
20278       IBM278, IBM EBCDIC Finland-Sweden
20280       IBM280, IBM EBCDIC Italy
20284       IBM284, IBM EBCDIC Latin America-Spain
20285       IBM285, IBM EBCDIC United Kingdom
20290       IBM290, IBM EBCDIC Japanese Katakana Extended
20297       IBM297, IBM EBCDIC France
20420       IBM420, IBM EBCDIC Arabic
20423       IBM423, IBM EBCDIC Greek
20424       IBM424, IBM EBCDIC Hebrew
20833       x-EBCDIC-KoreanExtended, IBM EBCDIC Korean Extended
20838       IBM-Thai, IBM EBCDIC Thai
20866       koi8-r, Russian (KOI8-R); Cyrillic (KOI8-R)
20871       IBM871, IBM EBCDIC Icelandic
20880       IBM880, IBM EBCDIC Cyrillic Russian
20905       IBM905, IBM EBCDIC Turkish
20924       IBM00924, IBM EBCDIC Latin 1/Open System (1047 + Euro symbol)
20932       EUC-JP, Japanese (JIS 0208-1990 and 0212-1990)
20936       x-cp20936, Simplified Chinese (GB2312); Chinese Simplified (GB2312-80)
20949       x-cp20949, Korean Wansung
21025       cp1025, IBM EBCDIC Cyrillic Serbian-Bulgarian
21027       (deprecated)
21866       koi8-u, Ukrainian (KOI8-U); Cyrillic (KOI8-U)
28591       iso-8859-1, ISO 8859-1 Latin 1; Western European (ISO)
28592       iso-8859-2, ISO 8859-2 Central European; Central European (ISO)
28593       iso-8859-3, ISO 8859-3 Latin 3
28594       iso-8859-4, ISO 8859-4 Baltic
28595       iso-8859-5, ISO 8859-5 Cyrillic
28596       iso-8859-6, ISO 8859-6 Arabic
28597       iso-8859-7, ISO 8859-7 Greek
28598       iso-8859-8, ISO 8859-8 Hebrew; Hebrew (ISO-Visual)
28599       iso-8859-9, ISO 8859-9 Turkish
28603       iso-8859-13, ISO 8859-13 Estonian
28605       iso-8859-15, ISO 8859-15 Latin 9
29001       x-Europa, Europa 3
38598       iso-8859-8-i, ISO 8859-8 Hebrew; Hebrew (ISO-Logical)
50220       iso-2022-jp, ISO 2022 Japanese with no halfwidth Katakana; Japanese (JIS)
50221       csISO2022JP, ISO 2022 Japanese with halfwidth Katakana; Japanese (JIS-Allow 1 byte Kana)
50222       iso-2022-jp, ISO 2022 Japanese JIS X 0201-1989; Japanese (JIS-Allow 1 byte Kana - SO/SI)
50225       iso-2022-kr, ISO 2022 Korean
50227       x-cp50227, ISO 2022 Simplified Chinese; Chinese Simplified (ISO 2022)
50229       ISO 2022 Traditional Chinese
50930       EBCDIC Japanese (Katakana) Extended
50931       EBCDIC US-Canada and Japanese
50933       EBCDIC Korean Extended and Korean
50935       EBCDIC Simplified Chinese Extended and Simplified Chinese
50936       EBCDIC Simplified Chinese
50937       EBCDIC US-Canada and Traditional Chinese
50939       EBCDIC Japanese (Latin) Extended and Japanese
51932       euc-jp, EUC Japanese
51936       EUC-CN, EUC Simplified Chinese; Chinese Simplified (EUC)
51949       euc-kr, EUC Korean
51950       EUC, Traditional Chinese
52936       hz-gb-2312, HZ-GB2312 Simplified Chinese; Chinese Simplified (HZ)
54936       GB18030, Windows XP and later: GB18030 Simplified Chinese (4 byte); Chinese Simplified (GB18030)
57002       x-iscii-de, ISCII Devanagari
57003       x-iscii-be, ISCII Bangla
57004       x-iscii-ta, ISCII Tamil
57005       x-iscii-te, ISCII Telugu
57006       x-iscii-as, ISCII Assamese
57007       x-iscii-or, ISCII Odia
57008       x-iscii-ka, ISCII Kannada
57009       x-iscii-ma, ISCII Malayalam
57010       x-iscii-gu, ISCII Gujarati
57011       x-iscii-pa, ISCII Punjabi
65000       utf-7, Unicode (UTF-7)
65001       utf-8, Unicode (UTF-8)
     
    Please let us know if you need any help.
    Regards,
    Limagito Team
     By Limagito-Team Pascal Script  , , , , , , , , , , , 
SEARCH
CATEGORIES
ARCHIVES
TAGS
365 api code command csv date delete drive Eldos email error filename filter ftp HL7 install License limagito limagitox log monitor multiple oauth OAuth2 parameter parameters Pascal pdf Remote rename roadmap Script service SFTP SharePoint sign SMTP SQL SSH unzip vendor week xls Xml zip
limagito logo
Limagito filemover is an automated file mover solution handling everything from moving, deleting to copying files.
Please don’t hesitate to contact us if you have any questions about our product and services.
DISCOVER OUR SITE
 Problem copying a file due to a folder tree that is too long 
October 20, 2024 no responses
 How to verify the pgp signature during decryption 
October 20, 2024 no responses
 How to send files to Azure using limagito file mover 
October 18, 2024 no responses
 © 2007-2024 IT services & software development by CPsoft // Webdesign & webhosting by Siteplan