XSM FAQ

  1. Why is the XSM program named hxsm ?
  2. Common software activation troubles
  3. Software Activation with LPAR AIX, POWER6 Live Partition Mobility
  4. How to display software expiration date for an activated XSM binary ?
  5. Inside a script, how to pass file names as variables ?
  6. Inside a script, how to pass sort parameters as variables ?
  7. Can I combine command line options and parmfile ?
  8. Can I combine FILE=n and static output file names in a parmfile ?
  9. How many maximum files can I process with FILE=n, SORTOFn ?
  10. When should I use KEEP_ORDER option ?
  11. In a parmfile, my FIELDS, INCLUDE, EXCLUDE are long lines, can I write them on separate lines ?
  12. When input file is empty, XSM stops on XSM083E error. Is there a workaround ?
  13. When deduplicating a file, can I specify which line to suppress, let's say for instance the second one ?
  14. What is Y2K option used for ? and how does it run ?
  15. Field separator COMMA 'FIELDSEP=,' confuses XSM syntax ; How to workaround ?
  16. How can I suppress informations displayed on stderr ?
  17. How to use multi-threading (XSM version 6) ?
  18. Can I run XSM over a Local Area Network ?
  19. How to uninstall XSM ?

ANSWERS:

  1. Why is the XSM program named hxsm ?
    In order to avoid confusion with UNIX X11 Session Manager "xsm"
    

    Top of page
  2. Common software activation troubles
    XSM software is activated using an activation key obtained on XSM Extranet from a Serial Number.
    
    When you download/copy XSM on a new platform, XSM displays its Serial Number using following commands:
        ./hhnsinst[VER] ./hxsm[VER]  ( VER=version  example: hhnsinst656 hxsm656)
        ./hxsm[VER] 
    
    Example:
    
        [root@srv1 tmp]# ./hhnsinst656  hxsm656
        Program 'hxsm656' successfully installed
        [root@srv1 tmp]# ./hxsm656
        Thanks for testing the 'HHNS Sort/Merge Program MultiPRO - V.6' Program
        If you wish to register for the 'HHNS Sort/Merge Program MultiPRO - V.6' Product on THIS machine,
        just contact HHNS with the following registration (serial number) key:
    
                        0013-3432-1505-M
        For further informations : http://www.hhns.fr, email:support@hhns.fr
    
        Please press Enter to continue...
    
    You must then generate an activation key for this platform on the Extranet.
    (a platform = hostname + Operating System + Serial Number).
    Then activate XSM with the activation key you get, using following command:
        ./hhnsinst[VER] ./hxsm[VER] cle_d_activation  ( VER=version  example: hhnsinst656 hxsm656)
    
    Example:
    
        [root@srv1 tmp]# ./hhnsinst656  hxsm656 1234-5678-9012-A
        Program 'hxsm656' successfully installed
    
    XSM is now ready. When you start it with no parameter, it displays its usage documentation.
    
    Common troubles: 
    • When you run hhnsinst command, you don't give the proper pathname of XSM program ; for instance, your current working directory is not the one containing XSM, or you have mistyped XSM program name
    • You have more than one XSM installed in your PATH ; when your run XSM, it is a non-activated version that runs because it has priority in the PATH
    • [UNIX] You don't have R/W rights on hxsm program. In this case, install program hhnsinst stops an error.
    • On some UNIX/Linux, you must be superuser "root" to activate the software.
    • Extranet refuses to generate a new key because the Serial Number is already in database: Check a platform with same Serial number is not already declared on your account: if yes, just proceed to "Reconduct key" to renew the activation key (30-days). if not, please contact us.
    • Extranet refuses to generate a new key because you have already generated 3 actiation keys: Contact us.

    Top of page
  3. Software Activation with LPAR AIX, POWER6 Live Partition Mobility
    Up to version 6.79, XSM Serial Number is based on software and hardware details.
    Thus when an XSM binary is activated on a LPAR system, it is NOT possible to move LPAR to another
     hardware p/System without re-activating XSM (it will display a new Serial Number).
    
    From version 6.80, XSM Serial Number is no longer based on hardware.
    Once activated on a LPAR system, XSM will continue running when LPAR is moved to another hardware.
    
    

    Top of page
  4. How to display software expiration date for an activated XSM binary ?
    
    $ hhnsinstxxx  -q  [pathname/]hxsmxxx
    
    example:
    
    $ hhnsint680 -q /usr/local/bin/hxsm680
    0024-1095-6502-U D=18092012
     !                 !
     !                 +- expiration : September 18, 2012
     +-Serial Number
    
    Beware:
     - do not mix a 64-bits hhnsinst with a 32-bits XSM and vice-versa.
     - specify the correct pathname to XSM binary if it is not on current directory.
     
    

    Top of page
  5. Inside a script, how to pass file names as variables ?
    a) using a parmfile (parameter file), using SORTIN and SORTOUT environment variables
    associated to INPFIL and OUTFIL orders: 
    
      myparm.xsm contains: 
        SORT     FIELDS=(14,7,B,A)
        RECORD   RECFM=V,LRECL=200
        INPFIL   DD:SORTIN
        OUTFIL   DD:SORTOUT
    
      Script sets SORTIN and SORTOUT values before running XSM:
        export SORTIN=/a/b/c/myinputfile
        export SORTOUT=/d/e/f/myoutputfile
        # Windows:  set SORTIN=/a/b/c/myinputfile
        # Windows:  set SORTOUT=/d/e/f/myoutputfile
        hxsm myparm.xsm
    
    b) using command line mode,
     
        A=/a/b/c/myinputfile
        B=/d/e/f/myoutputfile
        # Windows:  set A=/a/b/c/myinputfile
        # Windows:  set B=/d/e/f/myoutputfile
    
        # old syntax: 
        hxsm -rF -l180 -k14,10,A,C  -o$B $A
        # Windows: hxsm -rF -l180 -k14,10,A,C  -o%B% %A%
    
        # new syntax: 
        hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$A --outfile=$B
        # Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%A% --outfile=%B%
    
        # You can use as well UNIX stdin stdout redirections:
        hxsm --recfm=F --lrecl=180 --key=14,10,A,C < $A > $B
        # Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C < %A% > %B%
        
       Nota Bene: XSM gives better performances if you specify input file name, as it can dynamically
       work out the optimal STORAGE parameter from input file size.
    
       Using a parmfile, you can use DD:SORTIN only if there is a single input file.
       Otherwise, you must give static file names:
        SORT     FIELDS=(14,7,B,A)
        RECORD   RECFM=V,LRECL=200
        INPFIL   /a/b/c/myinputfile1
        INPFIL   /a/b/c/myinputfile2
        INPFIL   /a/b/c/myinputfile3
    

    Top of page
  6. Inside a script, how to pass sort parameters as variables ?
    a) using command mode, we assume the script has already built the necessary variables 
    
        RECFM=...
        LRECL=...
        KEYS=...
        hxsm --recfm=$RECFM --lrecl=$LRECL --key=$KEYS  --infile=$A --outfile=$B
    
    b) dynamically building a temporary parmfile before running XSM
    
        # create temp parmfile
        echo "
        **** temp parmfile /tmp/myparm.$$ ****
        SORT     FIELDS=($start1,$len1,C,A,$start2,$len2,C,A)
        RECORD   RECFM=$recfm,LRECL=$lrecl
        INPFIL   $A
        OUTFIL   $B
        "  > /tmp/myparm.$$
    
        # run XSM
        hxsm /tmp/myparm.$$
    
        # suppress temp file if RC=0
        if [ $? -eq 0 ] ; then rm /tmp/myparm.$$ ; fi
    
    Nota: method b) is recommanded to ease tests and maintenance.
    

    Top of page
  7. Can I combine command line options and parmfile ?
    You can use following options in command line, together with a parmfile:
      -q, --quiet    : quiet
      -v, --verbose  : verbose
      -c, --check    : don't sort, just checks that a previous result is correctly sorted
    
    Other options that define input/output file names, sort keys, sort options, temporary files (sortworks)
    can be used in command line as long as they are not in parmfile.
    
    Nota bene: parmfile must be the last command line parameter
    
    Examples:
    
     parmfile contains only sort keys and file format definition ;
    filenames are defined on command line:
    
      parmfile: 
        SORT     FIELDS=(10,8,,C,A)
        RECORD   RECFM=F,LRECL=100
    
      command line:
        hxsm -v myparm.xsm < /a/b/c/myinputfile > /d/e/f/myoutputfile 
    
     parmfile contains sort keys, file format, input file name ;
    output file name is defined on command line:
    
      parmfile: 
        SORT     FIELDS=(10,8,,C,A)
        RECORD   RECFM=F,LRECL=100
        INPFIL   /a/b/c/myinputfile
    
      command line:
        hxsm -v -o /d/e/f/myoutputfile myparm.xsm
    
    
     This example does not work: output filename is specified both in parmfile and command line!
    
      parmfile: 
        SORT     FIELDS=(10,8,,C,A)
        RECORD   RECFM=F,LRECL=100
        INPFIL   /a/b/c/myinputfile
        OUTFIL   /d/e/f/myoutputfile
    
      command line:
        hxsm -v -o /d/e/f/myoutputfile myparm.xsm
    

    Top of page
  8. Can I combine FILE=n and static output file names in a parmfile ?
    No: in a parmfile, output file names are specified using
        OUTFIL  file_name
     or OUTFIL  FILE=n 
    orders, but you cannot combine both forms in a same parmfile.
    
    OUTFIL FILE=n orders specify file names that have been set by SORTOFn environment variables.
    
    Example: 
      parmfile: 
        SORT FIELDS=(3,3,CH,A,51,11,CH,A)
        RECORD RECFM=F,LRECL=402
        INPFIL /tmp/worldclients.inp
        OUTFIL /tmp/clients.fr,INCLUDE=(19,3,CH,EQ,C'.fr')
        OUTFIL /tmp/clients.uk,INCLUDE=(19,3,CH,EQ,C'.uk')
        OUTFIL /tmp/clients.de,INCLUDE=(19,3,CH,EQ,C'.de')
    
      command line:
        hxsm -v myparm.xsm
    
      equivalent to
    
      parmfile: 
        SORT FIELDS=(3,3,CH,A,51,11,CH,A)
        RECORD RECFM=F,LRECL=402
        INPFIL /tmp/worldclients.inp
        OUTFIL FILE=1,INCLUDE=(19,3,CH,EQ,C'.fr')  # don't forget to set SORTOF1 environment variable
        OUTFIL FILE=2,INCLUDE=(19,3,CH,EQ,C'.uk')  # don't forget to set SORTOF2 environment variable
        OUTFIL FILE=3,INCLUDE=(19,3,CH,EQ,C'.de')  # don't forget to set SORTOF3 environment variable
    
      command line:
        SORTOF1=/tmp/clients.fr
        SORTOF2=/tmp/clients.uk
        SORTOF3=/tmp/clients.de
        hxsm -v myparm.xsm
    

    Top of page
  9. How many maximum files can I process with FILE=n, SORTOFn ?
    99 files. 
    
    Nota #1: first SORTOFn variable must be SORTOF1
    
    Nota #2: variables order must be consecutive:
    
     Correct:
        SORTOF1=/tmp/clients.fr
        SORTOF2=/tmp/clients.uk
        SORTOF3=/tmp/clients.de
        hxsm -v myparm.xsm
    
     Incorrect:
        SORTOF1=/tmp/clients.fr
        SORTOF2=/tmp/clients.uk
        SORTOF4=/tmp/clients.de
        hxsm -v myparm.xsm
    
    Nota #3: If this is not enought, please report it and we will make it higher.
    

    Top of page
  10. When should I use KEEP_ORDER option ?
    Please check Desctructive vs non-destructive sorts
    

    Top of page
  11. In a parmfile, my FIELDS, INCLUDE, EXCLUDE are long lines, can I write them on separate lines ?
    Yes: You can continue an instruction on next lines as long as it ends with a comma.
    
    Example:
    
        SORT  FIELDS=(011,003,CH,A,
                      001,010,CH,A,
                      015,114,CH,A,
                      014,001,CH,A)
        RECORD  RECFM=V,LRECL=263 
        EXCLUDE COND=(030,002,CH,EQ,C'L1',AND,
                      032,001,CH,EQ,C'1',
                      OR,
                      030,002,CH,EQ,C'L2',AND,
                      032,001,CH,EQ,C'2')
        INPFIL  DD:SORTIN
        OUTFIL  DD:SORTOUT
    
    Nota: In the COND= expression, boolean operator 'AND' has priority on 'OR' operator.
          You can add parenthesis for better readability: 
    
        EXCLUDE COND=((030,002,CH,EQ,C'L1',AND,
                       032,001,CH,EQ,C'1'),
                      OR,
                      (030,002,CH,EQ,C'L2',AND,
                       032,001,CH,EQ,C'2'))
    
    

    Top of page
  12. When input file is empty, XSM stops on XSM083E error. Is there a workaround ?
May the input data come from stdin (standard input) or a named file, XSM expects data to process !
If sometimes input data file is empty in a regular batch process, then you must test the
file before running XSM. This is trivial with UNIX Shell:

    #
    # ... some step of a batch process .. 
    #
    INPUT=/a/b/c/myinputfile
    OUTPUT=/d/e/f/myoutputfile
    # input file is size 0 or does not exists:
    if [ -s $INPUT ] ; then
         echo "input file is empty ; nothing to do !"
         # if you need the output files anyway, then pretend they exist:
         > $OUTPUT
    # file is okay for processing:
    else
         hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$INPUT --outfile=$OUTPUT
    fi
    if [ $? -ne 0 ] ; then
        echo "error in SORT step"
    fi


When using Windows batch command processor "cmd.exe", it needs a few "UNIX-like" tools ; in our 
example, we use test.exe, a Windows version of UNIX test command.
Nota: Perl is a pretty good solution for Windows scripting (and easily portable UNIX/Windows).

    Rem 
    Rem ... some step of a batch process .. 
    Rem
    set INPUT=C:\a\b\c\myinputfile
    set OUTPUT=C:\d\e\f\myoutputfile
    Rem input file is size 0 or does not exists:
    test -s %INPUT%
    if %ERRORLEVEL% 0 goto OK
    goto KO

    :KO
         echo input file is empty ; nothing to do !
         Rem if you need the output files anyway, then pretend they exist:
         ver 1>nul 2>$OUTPUT
         goto NEXT
    :OK
    Rem file is okay for processing:
         hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%INPUT% --outfile=%OUTPUT%
         goto NEXT


Top of page
  • When deduplicating a file, can I specify which line to suppress, let's say for instance the second one ?
    No, this is not possible.
    

    Top of page
  • What is Y2K option used for ? and how does it run ?
    Y or Y2K type specifies that a sort key of type DATE contains dates both before and after year 2000.
    XSM then uses year 70 (1970) as a pivot to work out if dates belong to XXth or XXIrst century.
    
    Example:  (French format ddmmyy -> dd/mm/yyyy)
        120186 : 86 > pivot -> 12/01/1986
        120271 : 71 > pivot -> 12/02/1986
        120370 : 70 ≤ pivot -> 12/03/2070
        120468 : 68 < pivot -> 12/04/2086
        120507 : 68 < pivot -> 12/05/2007
        120601 : 68 < pivot -> 12/06/2001
    
    The Y2KSTART option allows to choose a pivot value other thant 1970.
    
    Example: 
        OPTION Y2KSTART=1930
        SORT FIELDS=(11,2,Y2C,D,9,2,CH,D,7,2,CH,D,1,6,CH,A)
        RECORD RECFM=V,LRECL=80
        INPFIL clients.inp
        OUTFIL clients.out
    

    Top of page
  • Field separator COMMA 'FIELDSEP=,' confuses XSM syntax ; How to workaround ?
    Indeed, in following example, XSM interprets the comma as an instruction line separator and
    interprets the field separator is empty FIELDSEP= 
    
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=,
        RECORD RECFM=V,LRECL=100
        INPFIL clients.csv
        OUTFIL clients.out
    
    Use keywords to avoid this syntax confusion:
    
        BAR         =  the '|'  char
        TAB         =  the Tabulation (X'09') char
        COMMA       =  the ',' char
        COLUMN      =  the ':' char
        DIARESIS    =  the '#' char
        SLASH       =  the '/' char
        BACKSLASH   =  the '\' char
        SEMICOLUMN
     or SEMI-COLUMN   = the ';' char
        SINGLEQUOTE
     or SINGLE-QUOTE  = the "'" char
        DOUBLEQUOTE
     or DOUBLE-QUOTE  = the '"' char
    
    or specify an hexadecimal value: X'hh'   
    
    Examples:
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=COMMA
        SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=X'2C'
    
    When FIELDSEP is not specified in a SORT VFIELDS expression, its default value is TAB (X'09').
    

    Top of page
  • How can I suppress informations displayed on stderr ?
    Using -q or --quiet option.
    The advantage of "Quiet" mode over 2>/dev/null redirection is that only XSM errors will appear ;
    In the other case, all error messages are suppressed.
    
    Windows users: redirection is 2>NUL
    
    Verbose mode:
        ~/dev/ $ hxsm658 -v littlef.xsm
        XSM056I HXSM V6.58 - (c) HHNS 1996-2006
        XSM027I Initializing
        XSM028I Reading
        XSM073I 000000080 RECORDS PROCESSED, 000000080 so far.            :    0 SEC.
        XSM031I END OF JOB littlef.xsm                             80 rec.:    1 SEC.
        ~/dev/ $
    
    Normal mode: 
        ~/dev/ $ hxsm658  littlef.xsm
        XSM031I END OF JOB littlef.xsm                             80 rec.:    0 SEC.
        ~/dev/ $
    
    Quiet mode: 
        ~/dev/ $ hxsm658 -q littlef.xsm
        ~/dev/ $
    

    Top of page
  • How to use multi-threading (XSM version 6) ?
    On default mode, XSM runs in mono-threading and uses only one CPU at a given time.
    If you specify several sortworks (that is on several distinct physical disks), XSM will run
    in multi-threading and will give best performances. It the operating system that will bind the
    threads to available CPU ("processor binding").
    
    
    To force multi-threading, either use
        OPTIONS PROCS=n
    
    either specify several sortworks (even if you have only one, specify it several times)
        SORTWORKS dir1,dir2[,dir3 ...]
    
    Lets say you run a quadri-processor machine, the best thing is to find 4 physical drives available for
    sortworks. Disk I/O are very slow compared to CPU processing; thus it is not interesting to run XSM in
    multi-threading mode if you have only one physical disk for the sortworks :  indeed, XSM threads will be
    waiting because of I/O contentions and result will be similar to a mono-threading process.
    

    Top of page
  • Can I run XSM over a Local Area Network ?
    No: XSM runs on the operating system it has been activated on. Sharing a disk over a LAN to another
     machine does not allow this other machine to run XSM.
    Furthermore, it is not recommanded for performances to sort remote data from shared disks.
    In some situation, it can be faster to compress / copy / uncompress these data.
    It depends on:
    - LAN performances
    - disks performances
    - CPU performances.
    

    Top of page
  • How to uninstall XSM ?
    Just delete the 2 XSM program files:
    
     UNIX:    rm  hxsm[version] hhnsinst[version]
    
     Windows: del hxsm[version] hhnsinst[version]
     
     Et voila!
    

    Top of page
  • For any question, contact us.