1. Why is the XSM program named hxsm ?
2. Common software activation troubles
3. Inside a script, how to pass file names as variables ?
4. Inside a script, how to pass sort parameters as variables ?
5. Can I combine command line options and parmfile ?
6. Can I combine FILE=n and static output file names in a parmfile ?
7. How many maximum files can I process with FILE=n, SORTOFn ?
8. When should I use KEEP_ORDER option ?
9. In a parmfile, my FIELDS, INCLUDE, EXCLUDE are long lines, can I write them on separate lines ?
10. When input file is empty, XSM stops on XSM083E error. Is there a workaround ?
11. When deduplicating a file, can I specify which line to suppress, let's say for instance the second one ?
12. What is Y2K option used for ? and how does it run ?
13. Field separator COMMA 'FIELDSEP=,' confuses XSM syntax ; How to workaround ?
14. How can I suppress informations displayed on stderr ?
15. How to use multi-threading (XSM version 6) ?
16. Can I run XSM over a Local Area Network ?
17. How to uninstall XSM ?

ANSWERS:

1. Why is the XSM program named hxsm ?

   In order to avoid confusion with UNIX X11 Session Manager "xsm"
   

   
   
2. Common software activation troubles

   XSM software is activated using an activation key obtained on XSM Extranet from a Serial Number.
   
   When you download/copy XSM on a new platform, XSM displays its Serial Number using following commands:
       ./hhnsinst[VER] ./hxsm[VER]  ( VER=version  example: hhnsinst656 hxsm656)
       ./hxsm[VER] 
   
   Example:
   
       [root@srv1 tmp]# ./hhnsinst656  hxsm656
       Program 'hxsm656' successfully installed
       [root@srv1 tmp]# ./hxsm656
       Thanks for testing the 'HHNS Sort/Merge Program MultiPRO - V.6' Program
       If you wish to register for the 'HHNS Sort/Merge Program MultiPRO - V.6' Product on THIS machine,
       just contact HHNS with the following registration (serial number) key:
   
                       0013-3432-1505-M
       For further informations : http://www.hhns.fr, email:support@hhns.fr
   
       Please press Enter to continue...
   
   You must then generate an activation key for this platform on the Extranet.
   (a platform = hostname + Operating System + Serial Number).
   Then activate XSM with the activation key you get, using following command:
       ./hhnsinst[VER] ./hxsm[VER] cle_d_activation  ( VER=version  example: hhnsinst656 hxsm656)
   
   Example:
   
       [root@srv1 tmp]# ./hhnsinst656  hxsm656 1234-5678-9012-A
       Program 'hxsm656' successfully installed
   
   XSM is now ready. When you start it with no parameter, it displays its usage documentation.
   
   Common troubles: 
   
     
   When you run hhnsinst command, you don't give the proper pathname of XSM program ;
   for instance, your current working directory is not the one containing XSM, or
   you have mistyped XSM program name
     
   You have more than one XSM installed in your PATH ; when your run XSM, it is a non-activated
   version that runs because it has priority in the PATH
     
   [UNIX] You don't have R/W rights on hxsm program. In this case, install program
   hhnsinst stops an error.
     
   On some UNIX/Linux, you must be superuser "root" to activate the software.
     
   Extranet refuses to generate a new key because the Serial Number is already in database:
   Check a platform with same Serial number is not already declared on your account:
    if yes, just proceed to "Reconduct key" to renew the activation key (30-days).
    if not, please contact us.
     
   Extranet refuses to generate a new key because you have already generated 3 actiation keys:
   Contact us.
   
   
   

   
   
3. Inside a script, how to pass file names as variables ?

   a) using a parmfile (parameter file), using SORTIN and SORTOUT environment variables
   associated to INPFIL and OUTFIL orders: 
   
     myparm.xsm contains: 
       SORT     FIELDS=(14,7,B,A)
       RECORD   RECFM=V,LRECL=200
       INPFIL   DD:SORTIN
       OUTFIL   DD:SORTOUT
   
     Script sets SORTIN and SORTOUT values before running XSM:
       export SORTIN=/a/b/c/myinputfile
       export SORTOUT=/d/e/f/myoutputfile
       # Windows:  set SORTIN=/a/b/c/myinputfile
       # Windows:  set SORTOUT=/d/e/f/myoutputfile
       hxsm myparm.xsm
   
   b) using command line mode,
    
       A=/a/b/c/myinputfile
       B=/d/e/f/myoutputfile
       # Windows:  set A=/a/b/c/myinputfile
       # Windows:  set B=/d/e/f/myoutputfile
   
       # old syntax: 
       hxsm -rF -l180 -k14,10,A,C  -o$B $A
       # Windows: hxsm -rF -l180 -k14,10,A,C  -o%B% %A%
   
       # new syntax: 
       hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$A --outfile=$B
       # Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%A% --outfile=%B%
   
       # You can use as well UNIX stdin stdout redirections:
       hxsm --recfm=F --lrecl=180 --key=14,10,A,C < $A > $B
       # Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C < %A% > %B%
       
      Nota Bene: XSM gives better performances if you specify input file name, as it can dynamically
      work out the optimal STORAGE parameter from input file size.
   
      Using a parmfile, you can use DD:SORTIN only if there is a single input file.
      Otherwise, you must give static file names:
       SORT     FIELDS=(14,7,B,A)
       RECORD   RECFM=V,LRECL=200
       INPFIL   /a/b/c/myinputfile1
       INPFIL   /a/b/c/myinputfile2
       INPFIL   /a/b/c/myinputfile3
   

   
   
4. Inside a script, how to pass sort parameters as variables ?

   a) using command mode, we assume the script has already built the necessary variables 
   
       RECFM=...
       LRECL=...
       KEYS=...
       hxsm --recfm=$RECFM --lrecl=$LRECL --key=$KEYS  --infile=$A --outfile=$B
   
   b) dynamically building a temporary parmfile before running XSM
   
       # create temp parmfile
       echo "
       **** temp parmfile /tmp/myparm.$$ ****
       SORT     FIELDS=($start1,$len1,C,A,$start2,$len2,C,A)
       RECORD   RECFM=$recfm,LRECL=$lrecl
       INPFIL   $A
       OUTFIL   $B
       "  > /tmp/myparm.$$
   
       # run XSM
       hxsm /tmp/myparm.$$
   
       # suppress temp file if RC=0
       if [ $? -eq 0 ] ; then rm /tmp/myparm.$$ ; fi
   
   Nota: method b) is recommanded to ease tests and maintenance.
   

   
   
5. Can I combine command line options and parmfile ?

   You can use following options in command line, together with a parmfile:
     -q, --quiet    : quiet
     -v, --verbose  : verbose
     -c, --check    : don't sort, just checks that a previous result is correctly sorted
   
   Other options that define input/output file names, sort keys, sort options, temporary files (sortworks)
   can be used in command line as long as they are not in parmfile.
   
   Nota bene: parmfile must be the last command line parameter
   
   Examples:
   
    parmfile contains only sort keys and file format definition ;
   filenames are defined on command line:
   
     parmfile: 
       SORT     FIELDS=(10,8,,C,A)
       RECORD   RECFM=F,LRECL=100
   
     command line:
       hxsm -v myparm.xsm < /a/b/c/myinputfile > /d/e/f/myoutputfile 
   
    parmfile contains sort keys, file format, input file name ;
   output file name is defined on command line:
   
     parmfile: 
       SORT     FIELDS=(10,8,,C,A)
       RECORD   RECFM=F,LRECL=100
       INPFIL   /a/b/c/myinputfile
   
     command line:
       hxsm -v -o /d/e/f/myoutputfile myparm.xsm
   
   
    This example does not work: output filename is specified both in parmfile and command line!
   
     parmfile: 
       SORT     FIELDS=(10,8,,C,A)
       RECORD   RECFM=F,LRECL=100
       INPFIL   /a/b/c/myinputfile
       OUTFIL   /d/e/f/myoutputfile
   
     command line:
       hxsm -v -o /d/e/f/myoutputfile myparm.xsm
   

   
   
6. Can I combine FILE=n and static output file names in a parmfile ?

   No: in a parmfile, output file names are specified using
       OUTFIL  file_name
    or OUTFIL  FILE=n 
   orders, but you cannot combine both forms in a same parmfile.
   
   OUTFIL FILE=n orders specify file names that have been set by SORTOFn environment variables.
   
   Example: 
     parmfile: 
       SORT FIELDS=(3,3,CH,A,51,11,CH,A)
       RECORD RECFM=F,LRECL=402
       INPFIL /tmp/worldclients.inp
       OUTFIL /tmp/clients.fr,INCLUDE=(19,3,CH,EQ,C'.fr')
       OUTFIL /tmp/clients.uk,INCLUDE=(19,3,CH,EQ,C'.uk')
       OUTFIL /tmp/clients.de,INCLUDE=(19,3,CH,EQ,C'.de')
   
     command line:
       hxsm -v myparm.xsm
   
     equivalent to
   
     parmfile: 
       SORT FIELDS=(3,3,CH,A,51,11,CH,A)
       RECORD RECFM=F,LRECL=402
       INPFIL /tmp/worldclients.inp
       OUTFIL FILE=1,INCLUDE=(19,3,CH,EQ,C'.fr')  # don't forget to set SORTOF1 environment variable
       OUTFIL FILE=2,INCLUDE=(19,3,CH,EQ,C'.uk')  # don't forget to set SORTOF2 environment variable
       OUTFIL FILE=3,INCLUDE=(19,3,CH,EQ,C'.de')  # don't forget to set SORTOF3 environment variable
   
     command line:
       SORTOF1=/tmp/clients.fr
       SORTOF2=/tmp/clients.uk
       SORTOF3=/tmp/clients.de
       hxsm -v myparm.xsm
   

   
   
7. How many maximum files can I process with FILE=n, SORTOFn ?

   99 files. 
   
   Nota #1: first SORTOFn variable must be SORTOF1
   
   Nota #2: variables order must be consecutive:
   
    Correct:
       SORTOF1=/tmp/clients.fr
       SORTOF2=/tmp/clients.uk
       SORTOF3=/tmp/clients.de
       hxsm -v myparm.xsm
   
    Incorrect:
       SORTOF1=/tmp/clients.fr
       SORTOF2=/tmp/clients.uk
       SORTOF4=/tmp/clients.de
       hxsm -v myparm.xsm
   
   Nota #3: If this is not enought, please report it and we will make it higher.
   

   
   
8. When should I use KEEP_ORDER option ?

   Please check Desctuctifs and non-destructive sorts
   

   
   
9. In a parmfile, my FIELDS, INCLUDE, EXCLUDE are long lines, can I write them on separate lines ?

   Yes: You can continue an instruction on next lines as long as it ends with a comma.
   
   Example:
   
       SORT  FIELDS=(011,003,CH,A,
                     001,010,CH,A,
                     015,114,CH,A,
                     014,001,CH,A)
       RECORD  RECFM=V,LRECL=263 
       EXCLUDE COND=(030,002,CH,EQ,C'L1',AND,
                     032,001,CH,EQ,C'1',
                     OR,
                     030,002,CH,EQ,C'L2',AND,
                     032,001,CH,EQ,C'2')
       INPFIL  DD:SORTIN
       OUTFIL  DD:SORTOUT
   
   Nota: In the COND= expression, boolean operator 'AND' has priority on 'OR' operator.
         You can add parenthesis for better readability: 
   
       EXCLUDE COND=((030,002,CH,EQ,C'L1',AND,
                      032,001,CH,EQ,C'1'),
                     OR,
                     (030,002,CH,EQ,C'L2',AND,
                      032,001,CH,EQ,C'2'))
   
   

   
   
10. When input file is empty, XSM stops on XSM083E error. Is there a workaround ?

    May the input data come from stdin (standard input) or a named file, XSM expects data to process !
    If sometimes input data file is empty in a regular batch process, then you must test the
    file before running XSM. This is trivial with UNIX Shell:
    
        #
        # ... some step of a batch process .. 
        #
        INPUT=/a/b/c/myinputfile
        OUTPUT=/d/e/f/myoutputfile
        # input file is size 0 or does not exists:
        if [ -s $INPUT ] ; then
             echo "input file is empty ; nothing to do !"
             # if you need the output files anyway, then pretend they exist:
             > $OUTPUT
        # file is okay for processing:
        else
             hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$INPUT --outfile=$OUTPUT
        fi
        if [ $? -ne 0 ] ; then
            echo "error in SORT step"
        fi
    
    
    When using Windows batch command processor "cmd.exe", it needs a few "UNIX-like" tools ; in our 
    example, we use test.exe, a Windows version of UNIX test command.
    Nota: Perl is a pretty good solution for Windows scripting (and easily portable UNIX/Windows).
    
        Rem 
        Rem ... some step of a batch process .. 
        Rem
        set INPUT=C:\a\b\c\myinputfile
        set OUTPUT=C:\d\e\f\myoutputfile
        Rem input file is size 0 or does not exists:
        test -s %INPUT%
        if %ERRORLEVEL% 0 goto OK
        goto KO
    
        :KO
             echo input file is empty ; nothing to do !
             Rem if you need the output files anyway, then pretend they exist:
             ver 1>nul 2>$OUTPUT
             goto NEXT
        :OK
        Rem file is okay for processing:
             hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%INPUT% --outfile=%OUTPUT%
             goto NEXT
    
    

    
    
11. When deduplicating a file, can I specify which line to suppress, let's say for instance the second one ?

    No, this is not possible.
    

    
    
12. What is Y2K option used for ? and how does it run ?

    Y or Y2K type specifies that a sort key of type DATE contains dates both before and after year 2000.
    XSM then uses year 70 (1970) as a pivot to work out if dates belong to XXth or XXIrst century.
    
    Example:  (French format ddmmyy -> dd/mm/yyyy)
        120186 : 86 > pivot -> 12/01/1986
        120271 : 71 > pivot -> 12/02/1986
        120370 : 70 ≤ pivot -> 12/03/2070
        120468 : 68 < pivot -> 12/04/2086
        120507 : 68 < pivot -> 12/05/2007
        120601 : 68 < pivot -> 12/06/2001
    
    The Y2KSTART option allows to choose a pivot value other thant 1970.
    
    Example: 
        OPTION Y2KSTART=1930
        SORT FIELDS=(11,2,Y2C,D,9,2,CH,D,7,2,CH,D,1,6,CH,A)
        RECORD RECFM=V,LRECL=80
        INPFIL clients.inp
        OUTFIL clients.out
    

    
    
13. Field separator COMMA 'FIELDSEP=,' confuses XSM syntax ; How to workaround ?

    Indeed, in following example, XSM interprets the comma as an instruction line separator and
    interprets the field separator is empty FIELDSEP= 
    
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=,
        RECORD RECFM=V,LRECL=100
        INPFIL clients.csv
        OUTFIL clients.out
    
    Use keywords to avoid this syntax confusion:
    
        BAR         =  the '|'  char
        TAB         =  the Tabulation (X'09') char
        COMMA       =  the ',' char
        COLUMN      =  the ':' char
        DIARESIS    =  the '#' char
        SLASH       =  the '/' char
        BACKSLASH   =  the '\' char
        SEMICOLUMN
     or SEMI-COLUMN   = the ';' char
        SINGLEQUOTE
     or SINGLE-QUOTE  = the "'" char
        DOUBLEQUOTE
     or DOUBLE-QUOTE  = the '"' char
    
    or specify an hexadecimal value: X'hh'   
    
    Examples:
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=COMMA
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=X'2C'
    
    When FIELDSEP is not specified in a SORT VFIELDS expression, its default value is TAB (X'09').
    

    
    
14. How can I suppress informations displayed on stderr ?

    Using -q or --quiet option.
    The advantage of "Quiet" mode over 2>/dev/null redirection is that only XSM errors will appear ;
    In the other case, all error messages are suppressed.
    
    Windows users: redirection is 2>NUL
    
    Verbose mode:
        ~/dev/ $ hxsm658 -v littlef.xsm
        XSM056I HXSM V6.58 - (c) HHNS 1996-2006
        XSM027I Initializing
        XSM028I Reading
        XSM073I 000000080 RECORDS PROCESSED, 000000080 so far.            :    0 SEC.
        XSM031I END OF JOB littlef.xsm                             80 rec.:    1 SEC.
        ~/dev/ $
    
    Normal mode: 
        ~/dev/ $ hxsm658  littlef.xsm
        XSM031I END OF JOB littlef.xsm                             80 rec.:    0 SEC.
        ~/dev/ $
    
    Quiet mode: 
        ~/dev/ $ hxsm658 -q littlef.xsm
        ~/dev/ $
    

    
    
15. How to use multi-threading (XSM version 6) ?

    On default mode, XSM runs in mono-threading and uses only one CPU at a given time.
    If you specify several sortworks (that is on several distinct physical disks), XSM will run
    in multi-threading and will give best performances. It the operating system that will bind the
    threads to available CPU ("processor binding").
    
    
    To force multi-threading, either use
        OPTIONS PROCS=n
    
    either specify several sortworks (even if you have only one, specify it several times)
        SORTWORKS dir1,dir2[,dir3 ...]
    
    Lets say you run a quadri-processor machine, the best thing is to find 4 physical drives available for
    sortworks. Disk I/O are very slow compared to CPU processing; thus it is not interesting to run XSM in
    multi-threading mode if you have only one physical disk for the sortworks :  indeed, XSM threads will be
    waiting because of I/O contentions and result will be similar to a mono-threading process.
    

    
    
16. Can I run XSM over a Local Area Network ?

    No: XSM runs on the operating system it has been activated on. Sharing a disk over a LAN to another
     machine does not allow this other machine to run XSM.
    Furthermore, it is not recommanded for performances to sort remote data from shared disks.
    In some situation, it can be faster to compress / copy / uncompress these data.
    It depends on:
    - LAN performances
    - disks performances
    - CPU performances.
    

    
    
17. How to uninstall XSM ?

     UNIX:    rm  hxsm[version] hhnsinst[version]
    
     Windows: del hxsm[version] hhnsinst[version]