-
Why is the XSM program named hxsm ?
In order to avoid confusion with UNIX X11 Session Manager "xsm"
-
Common software activation troubles
Current XSM 7 activation procedure
XSM 7.00 is delivered as a ready-to-use binary. After extracting the package, make the
programs executable on Linux or AIX, then run the activation program:
chmod +x hxsm700 hhnsinst
./hhnsinst ./hxsm700
On Windows, run activate_xsm.bat, or use:
hhnsinst.exe hxsm700.exe
The initial activation command does not require an Extranet account. For batch use,
register the platform on the XSM Extranet and generate its activation key.
Earlier XSM 6 activation procedures
The following examples are retained for historical XSM 6 installations. XSM software was
activated using an activation key obtained from the XSM Extranet using a Serial Number.
After downloading or copying XSM to a new platform, its Serial Number was displayed using:
./hhnsinst[VER] ./hxsm[VER] (VER=version; example: hhnsinst656 hxsm656)
./hxsm[VER]
Historical example:
[root@srv1 tmp]# ./hhnsinst656 hxsm656
Program 'hxsm656' successfully installed
[root@srv1 tmp]# ./hxsm656
Thanks for testing the 'HHNS Sort/Merge Program MultiPRO - V.6' Program
If you wish to register for the 'HHNS Sort/Merge Program MultiPRO - V.6' Product on THIS machine,
just contact HHNS with the following registration (serial number) key:
0013-3432-1505-M
For further informations : http://www.hhns.fr, email:support@hhns.fr
Please press Enter to continue...
An activation key was then generated for the platform on the Extranet
(a platform = hostname + operating system + Serial Number).
XSM was activated with that key using:
./hhnsinst[VER] ./hxsm[VER] activation_key (VER=version; example: hhnsinst656 hxsm656)
Example:
[root@srv1 tmp]# ./hhnsinst656 hxsm656 1234-5678-9012-A
Program 'hxsm656' successfully installed
XSM was then ready. When started without parameters, it displayed its usage documentation.
Common troubles:
- When running
hhnsinst, you did not provide the correct XSM pathname; for example, the current working directory does not contain XSM, or the program name was mistyped.
- You have more than one XSM installation in your PATH; a non-activated version may run because it appears earlier in the PATH.
- [UNIX] You do not have read/write access to the XSM program. In this case, the
hhnsinst activation program stops with an error.
- On some UNIX/Linux, you must be superuser "root" to activate the software.
- Extranet refuses to generate a new key because the Serial Number is already in database: Check a platform with same Serial number is not already declared on your account: if yes, just proceed to "Reconduct key" to renew the activation key (30-days). if not, please contact us.
- Extranet refuses to generate a new key because you have already generated three activation keys: Contact us.
-
Software Activation with LPAR AIX, POWER6 Live Partition Mobility
Up to version 6.79, XSM Serial Number is based on software and hardware details.
Thus when an XSM binary is activated on a LPAR system, it is NOT possible to move LPAR to another
hardware p/System without re-activating XSM (it will display a new Serial Number).
From version 6.80, XSM Serial Number is no longer based on hardware.
Once activated on a LPAR system, XSM will continue running when LPAR is moved to another hardware.
-
How can I display the expiration date of an activated XSM binary?
$ hhnsinstxxx -q [pathname/]hxsmxxx
example:
$ hhnsinst680 -q /usr/local/bin/hxsm680
0024-1095-6502-U D=18092012
! !
! +- expiration : September 18, 2012
+-Serial Number
Beware:
- do not mix a 64-bits hhnsinst with a 32-bits XSM and vice-versa.
- specify the correct pathname to XSM binary if it is not on current directory.
-
How can I pass file names as variables in a script?
a) using a parmfile (parameter file), using SORTIN and SORTOUT environment variables
associated to INPFIL and OUTFIL orders:
myparm.xsm contains:
SORT FIELDS=(14,7,B,A)
RECORD RECFM=V,LRECL=200
INPFIL DD:SORTIN
OUTFIL DD:SORTOUT
Script sets SORTIN and SORTOUT values before running XSM:
export SORTIN=/a/b/c/myinputfile
export SORTOUT=/d/e/f/myoutputfile
# Windows: set SORTIN=/a/b/c/myinputfile
# Windows: set SORTOUT=/d/e/f/myoutputfile
hxsm myparm.xsm
b) using command line mode,
A=/a/b/c/myinputfile
B=/d/e/f/myoutputfile
# Windows: set A=/a/b/c/myinputfile
# Windows: set B=/d/e/f/myoutputfile
# old syntax:
hxsm -rF -l180 -k14,10,A,C -o$B $A
# Windows: hxsm -rF -l180 -k14,10,A,C -o%B% %A%
# new syntax:
hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$A --outfile=$B
# Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%A% --outfile=%B%
# You can use as well UNIX stdin stdout redirections:
hxsm --recfm=F --lrecl=180 --key=14,10,A,C < $A > $B
# Windows: hxsm --recfm=F --lrecl=180 --key=14,10,A,C < %A% > %B%
Nota Bene: XSM gives better performances if you specify input file name, as it can dynamically
work out the optimal STORAGE parameter from input file size.
Using a parmfile, you can use DD:SORTIN only if there is a single input file.
Otherwise, you must give static file names:
SORT FIELDS=(14,7,B,A)
RECORD RECFM=V,LRECL=200
INPFIL /a/b/c/myinputfile1
INPFIL /a/b/c/myinputfile2
INPFIL /a/b/c/myinputfile3
-
How can I pass sort parameters as variables in a script?
a) using command mode, we assume the script has already built the necessary variables
RECFM=...
LRECL=...
KEYS=...
hxsm --recfm=$RECFM --lrecl=$LRECL --key=$KEYS --infile=$A --outfile=$B
b) dynamically building a temporary parmfile before running XSM
# create temp parmfile
echo "
**** temp parmfile /tmp/myparm.$$ ****
SORT FIELDS=($start1,$len1,C,A,$start2,$len2,C,A)
RECORD RECFM=$recfm,LRECL=$lrecl
INPFIL $A
OUTFIL $B
" > /tmp/myparm.$$
# run XSM
hxsm /tmp/myparm.$$
# suppress temp file if RC=0
if [ $? -eq 0 ] ; then rm /tmp/myparm.$$ ; fi
Nota: method b) is recommanded to ease tests and maintenance.
-
Can I combine command-line options and a parmfile?
You can use following options in command line, together with a parmfile:
-q, --quiet : quiet
-v, --verbose : verbose
-c, --check : don't sort, just checks that a previous result is correctly sorted
Other options that define input/output file names, sort keys, sort options, temporary files (sortworks)
can be used in command line as long as they are not in parmfile.
Nota bene: parmfile must be the last command line parameter
Examples:
parmfile contains only sort keys and file format definition ;
filenames are defined on command line:
parmfile:
SORT FIELDS=(10,8,,C,A)
RECORD RECFM=F,LRECL=100
command line:
hxsm -v myparm.xsm < /a/b/c/myinputfile > /d/e/f/myoutputfile
parmfile contains sort keys, file format, input file name ;
output file name is defined on command line:
parmfile:
SORT FIELDS=(10,8,,C,A)
RECORD RECFM=F,LRECL=100
INPFIL /a/b/c/myinputfile
command line:
hxsm -v -o /d/e/f/myoutputfile myparm.xsm
This example does not work: output filename is specified both in parmfile and command line!
parmfile:
SORT FIELDS=(10,8,,C,A)
RECORD RECFM=F,LRECL=100
INPFIL /a/b/c/myinputfile
OUTFIL /d/e/f/myoutputfile
command line:
hxsm -v -o /d/e/f/myoutputfile myparm.xsm
-
Can I combine FILE=n and static output file names in a parmfile ?
No: in a parmfile, output file names are specified using
OUTFIL file_name
or OUTFIL FILE=n
orders, but you cannot combine both forms in a same parmfile.
OUTFIL FILE=n orders specify file names that have been set by SORTOFn environment variables.
Example:
parmfile:
SORT FIELDS=(3,3,CH,A,51,11,CH,A)
RECORD RECFM=F,LRECL=402
INPFIL /tmp/worldclients.inp
OUTFIL /tmp/clients.fr,INCLUDE=(19,3,CH,EQ,C'.fr')
OUTFIL /tmp/clients.uk,INCLUDE=(19,3,CH,EQ,C'.uk')
OUTFIL /tmp/clients.de,INCLUDE=(19,3,CH,EQ,C'.de')
command line:
hxsm -v myparm.xsm
equivalent to
parmfile:
SORT FIELDS=(3,3,CH,A,51,11,CH,A)
RECORD RECFM=F,LRECL=402
INPFIL /tmp/worldclients.inp
OUTFIL FILE=1,INCLUDE=(19,3,CH,EQ,C'.fr') # don't forget to set SORTOF1 environment variable
OUTFIL FILE=2,INCLUDE=(19,3,CH,EQ,C'.uk') # don't forget to set SORTOF2 environment variable
OUTFIL FILE=3,INCLUDE=(19,3,CH,EQ,C'.de') # don't forget to set SORTOF3 environment variable
command line:
SORTOF1=/tmp/clients.fr
SORTOF2=/tmp/clients.uk
SORTOF3=/tmp/clients.de
hxsm -v myparm.xsm
-
What is the maximum number of files I can process with FILE=n and SORTOFn?
99 files.
Nota #1: first SORTOFn variable must be SORTOF1
Nota #2: variables order must be consecutive:
Correct:
SORTOF1=/tmp/clients.fr
SORTOF2=/tmp/clients.uk
SORTOF3=/tmp/clients.de
hxsm -v myparm.xsm
Incorrect:
SORTOF1=/tmp/clients.fr
SORTOF2=/tmp/clients.uk
SORTOF4=/tmp/clients.de
hxsm -v myparm.xsm
Nota #3: If this is not enought, please report it and we will make it higher.
-
When should I use the KEEP_ORDER option?
Please check Desctructive vs non-destructive sorts
-
In a parmfile, my FIELDS, INCLUDE, EXCLUDE are long lines, can I write them on separate lines ?
Yes: You can continue an instruction on next lines as long as it ends with a comma.
Example:
SORT FIELDS=(011,003,CH,A,
001,010,CH,A,
015,114,CH,A,
014,001,CH,A)
RECORD RECFM=V,LRECL=263
EXCLUDE COND=(030,002,CH,EQ,C'L1',AND,
032,001,CH,EQ,C'1',
OR,
030,002,CH,EQ,C'L2',AND,
032,001,CH,EQ,C'2')
INPFIL DD:SORTIN
OUTFIL DD:SORTOUT
Nota: In the COND= expression, boolean operator 'AND' has priority on 'OR' operator.
You can add parenthesis for better readability:
EXCLUDE COND=((030,002,CH,EQ,C'L1',AND,
032,001,CH,EQ,C'1'),
OR,
(030,002,CH,EQ,C'L2',AND,
032,001,CH,EQ,C'2'))
-
When input file is empty, XSM stops on XSM083E error. Is there a workaround ?
May the input data come from stdin (standard input) or a named file, XSM expects data to process !
If sometimes input data file is empty in a regular batch process, then you must test the
file before running XSM. This is trivial with UNIX Shell:
#
# ... some step of a batch process ..
#
INPUT=/a/b/c/myinputfile
OUTPUT=/d/e/f/myoutputfile
# input file is size 0 or does not exists:
if [ -s $INPUT ] ; then
echo "input file is empty ; nothing to do !"
# if you need the output files anyway, then pretend they exist:
> $OUTPUT
# file is okay for processing:
else
hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=$INPUT --outfile=$OUTPUT
fi
if [ $? -ne 0 ] ; then
echo "error in SORT step"
fi
When using Windows batch command processor "cmd.exe", it needs a few "UNIX-like" tools ; in our
example, we use test.exe, a Windows version of UNIX test command.
Nota: Perl is a pretty good solution for Windows scripting (and easily portable UNIX/Windows).
Rem
Rem ... some step of a batch process ..
Rem
set INPUT=C:\a\b\c\myinputfile
set OUTPUT=C:\d\e\f\myoutputfile
Rem input file is size 0 or does not exists:
test -s %INPUT%
if %ERRORLEVEL% 0 goto OK
goto KO
:KO
echo input file is empty ; nothing to do !
Rem if you need the output files anyway, then pretend they exist:
ver 1>nul 2>$OUTPUT
goto NEXT
:OK
Rem file is okay for processing:
hxsm --recfm=F --lrecl=180 --key=14,10,A,C --infile=%INPUT% --outfile=%OUTPUT%
goto NEXT
-
When deduplicating a file, can I specify which line to suppress, let's say for instance the second one ?
No, this is not possible.
-
What is Y2K option used for ? and how does it run ?
Y or Y2K type specifies that a sort key of type DATE contains dates both before and after year 2000.
XSM then uses year 70 (1970) as a pivot to work out if dates belong to XXth or XXIrst century.
Example: (French format ddmmyy -> dd/mm/yyyy)
120186 : 86 > pivot -> 12/01/1986
120271 : 71 > pivot -> 12/02/1986
120370 : 70 ≤ pivot -> 12/03/2070
120468 : 68 < pivot -> 12/04/2086
120507 : 68 < pivot -> 12/05/2007
120601 : 68 < pivot -> 12/06/2001
The Y2KSTART option allows to choose a pivot value other thant 1970.
Example:
OPTION Y2KSTART=1930
SORT FIELDS=(11,2,Y2C,D,9,2,CH,D,7,2,CH,D,1,6,CH,A)
RECORD RECFM=V,LRECL=80
INPFIL clients.inp
OUTFIL clients.out
-
Field separator COMMA '
FIELDSEP=,' confuses XSM syntax ; How to workaround ?
Indeed, in following example, XSM interprets the comma as an instruction line separator and
interprets the field separator is empty FIELDSEP=
SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=,
RECORD RECFM=V,LRECL=100
INPFIL clients.csv
OUTFIL clients.out
Use keywords to avoid this syntax confusion:
BAR = the '|' char
TAB = the Tabulation (X'09') char
COMMA = the ',' char
COLUMN = the ':' char
DIARESIS = the '#' char
SLASH = the '/' char
BACKSLASH = the '\' char
SEMICOLUMN
or SEMI-COLUMN = the ';' char
SINGLEQUOTE
or SINGLE-QUOTE = the "'" char
DOUBLEQUOTE
or DOUBLE-QUOTE = the '"' char
or specify an hexadecimal value: X'hh'
Examples:
SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=COMMA
SORT VFIELDS=(4,5,N,A,1,20,C,D),FIELDSEP=X'2C'
When FIELDSEP is not specified in a SORT VFIELDS expression, its default value is TAB (X'09').
-
How can I suppress information displayed on stderr?
Using -q or --quiet option.
The advantage of "Quiet" mode over 2>/dev/null redirection is that only XSM errors will appear ;
In the other case, all error messages are suppressed.
Windows users: redirection is 2>NUL
Verbose mode:
~/dev/ $ hxsm658 -v littlef.xsm
XSM056I HXSM V6.58 - (c) HHNS 1996-2006
XSM027I Initializing
XSM028I Reading
XSM073I 000000080 RECORDS PROCESSED, 000000080 so far. : 0 SEC.
XSM031I END OF JOB littlef.xsm 80 rec.: 1 SEC.
~/dev/ $
Normal mode:
~/dev/ $ hxsm658 littlef.xsm
XSM031I END OF JOB littlef.xsm 80 rec.: 0 SEC.
~/dev/ $
Quiet mode:
~/dev/ $ hxsm658 -q littlef.xsm
~/dev/ $
-
How to use multi-threading (XSM version 6) ?
XSM has used multithreaded processing since version 5. XSM 7.00 introduces adaptive resource-management logic that automatically selects parallelism and resource allocation according to the workload and available resources, reducing the risk of out-of-memory situations. Parameters from earlier releases remain compatible, but PROCS and MEM should normally be removed so that XSM 7.00 adaptive tuning can select the appropriate settings. The procedure below applies to XSM 6 and is retained for historical reference.
By default, XSM 6 runs in single-threaded mode and uses only one CPU at a given time.
If you specify several SORTWORKS directories on distinct physical disks, XSM 6 runs
in multithreaded mode. The operating system binds the threads to the available processors
("processor binding").
To force multithreading, use:
OPTIONS PROCS=n
Alternatively, specify several SORTWORKS directories:
SORTWORKS dir1,dir2[,dir3 ...]
On a four-processor system, the preferred configuration uses four separate physical drives for
SORTWORKS. Disk I/O is much slower than CPU processing, so using multithreading with only one physical
SORTWORKS disk provides little benefit: the threads wait for contended disk I/O and performance is
similar to single-threaded processing.
-
Can I run XSM over a Local Area Network ?
No: XSM runs on the operating system it has been activated on. Sharing a disk over a LAN to another
machine does not allow this other machine to run XSM.
Furthermore, it is not recommanded for performances to sort remote data from shared disks.
In some situation, it can be faster to compress / copy / uncompress these data.
It depends on:
- LAN performances
- disks performances
- CPU performances.
-
How do I uninstall XSM?
Just delete the 2 XSM program files:
UNIX: rm hxsm[version] hhnsinst[version]
Windows: del hxsm[version] hhnsinst[version]
Et voila!
