English
XSM Dernière version : 6.94 Beta release : n/a
- Choisissez votre plateforme et téléchargez
- Un coup d'oeil rapide au Mini-Guide d'installation
- Enregistrez-votre plateforme sur notre Extranet et générez immédiatement vos clés d'activations
NOTA :
Ces versions ne sont pas des versions de démo limitées mais des versions de production.
Sans clé d'activation, XSM fonctionnera mais demandera d'appuyez sur la touche ENTREE à chaque lancement.
Vous pouvez télécharger et tester XSM sans clé d'activation, en mode interactif (pas en batch) ; dans ce cas, vous n'avez pas besoin de créer de compte Extranet.
Consultez le ChangeLog XSM pour les dernières fonctionnalités et les corrections de bugs.
| Plateform | ||||
|---|---|---|---|---|
| Operating System | CPU Architecture | Product Version | Evolution & Support |
|
 Windows |
||||
| Windows 2012, 2008, 2003, 10, 8, Seven, Vista, XP | x86 | XSM v6.94 |  |
|
 Linux
:: Linux users, please choose the appropriate glibc level ::
|
||||
| RHEL8, Centos8, FC28 (kernel ≥ 4.18.0 glibc ≥ 2.18) | x86 | XSM v6.94 | |
|
| RHEL7, Centos7, FC20 (kernel ≥ 3.10.0 glibc ≥ 2.17) | x86 | XSM v6.94 | |
|
| RHEL6, Centos6, FC16 (kernel ≥ 2.6.32 glibc ≥ 2.12) | x86 | XSM v6.94 | |
|
| RHEL5, Centos5, FC9 (kernel ≥ 2.6.18 glibc ≥ 2.5) | x86 | XSM v6.94 | |
|
| RHEL4, Centos4, FC3 (kernel ≥ 2.6.9 glibc ≥ 2.3) | x86 | XSM v6.77 | ||
| SuSE SLES 12 (kernel ≥ 3.12.28 glibc ≥ 2.19) | x86 | XSM v6.94 | |
|
| SuSE 11 z/Linux (kernel 3.0.76 glibc 2.11.3) | IBM z/S390x | XSM v6.93 | |
|
| SuSE9 (kernel ≥ 2.6.18 glibc ≥ 2.3) | IBM PPC | XSM v6.68 | ||
 AIX
| ||||
| AIX 6.1 (compatible AIX 7) | P/systems | XSM v6.94 | |
|
| AIX 5.3 | P/Series | XSM v6.94 | |
|
 Sun Solaris |
||||
| SunOS Solaris 10 | SPARC | XSM v6.94 | |
|
| SunOS Solaris 8 (9 compatible) | SPARC | XSM v6.94 | |
|
| SunOS OpenSolaris 11 | x86 | XSM v6.94 | |
|
Misc.,
 HP-UX
 OS/400 z/OS |
||||
| HP HP-UX 11i v3 | Itanium ia64 | XSM v6.68 | ||
| OS/400 AS400 IFS | i/Series | please contact us for trial | ||
| z/OS 1.9 Unix System Services (USS) | z/Series | please contact us for trial | ||
| Development API for Windows (DLL) and UNIX/Linux (lib.so) | please contact us for trial | |||
|
Can't find your favorite platform on this page ?
Just ask and We'll compile ! |
||||
CHANGE LOG
----------------------------------------------------------------------------
Release Notes for Version 6.94 RC2 :
02-May-2023
Bugs fixed: (Linux x86)
When excuted by crontab or scheduler, XSM was complaining about wrong activation key. Fixed.
----------------------------------------------------------------------------
Release Notes for Version 6.94 :
02-Jan-2022
Compiled on RHEL8/CENTOS8
03-Jul-2020
New features:
--outfiledup= Writes duplicate records to a file, according to OMIT DUPLICATE KEYS/RECORDS option.
or
OUTFILDUP outfile
or
OUTFILDUP DD:variable
--outfiledup-record1 To use with --outfiledup= : Also writes first record that will compare to duplicates.
or In option is not specified, then only next duplicate records are written to duplicate file.
--dup-rec1
or
OPTION WRITEFIRSTDUPLICATE
Misc:
Improved control on sort statements and options.
----------------------------------------------------------------------------
Release Notes for Version 6.93 RC3 :
18-Feb-2020
Bugs fixed:
Work buffer too small for fullpathnames (path + filename) greater than 110 chars. Increased to 260 char.
----------------------------------------------------------------------------
Release Notes for Version 6.93 :
06-June-2017
Misc:
Improved control on sort statements and options.
Hardening control on FIELDS positions and length.
Improved warning/error messages
Bugs fixed:
OMIT COND=(a,OR,b,AND,c,...) buffer was too short for big (+20 lines) conditions. Increased to 2000 char.
Bug on parenthesis in conditions COND=(a,AND,(b,OR,c)). Fixed
Bug on multiple INCLUDE/EXCLUDE statements. Fixed
Bug on --sort command line option. fixed.
Improper message "XSM004W Warning: No SORT/COPY/MERGE statement specified, defaulting to SORT FIELDS=ALL"
when using no parameter file. Fixed.
New features:
OUTREC FIELDS=(...) fixed and improved. Now works with VFIELDS.
--veryverbose gives more and better details.
Now tolerates SORT FIELDS=COPY for IBM SORT compatibility, equivalent to OPTION COPY.
Now tolerates default numeric format COND=(1,3,GT,123)
as equivalent to COND=(1,3,GT,N'123'). for IBM SORT compat.
Warning if multiple OUTFIL DD use same variable or filename.
----------------------------------------------------------------------------
Release Notes for Version 6.92 :
13-June-2015
New features:
INCLUDE/EXCLUDE now work now with VFIELDS (CSV files)
--veryverbose show more details and counters
----------------------------------------------------------------------------
Release Notes for Version 6.91 :
25-May-2015
Misc:
Performance improvement again and again ...
hxsm690_64 sz=100GB Elapse: 6664 sec.
hxsm691_64 sz=100GB Elapse: 3721 sec ... nearly 2 times faster!
(1 x Intel Xeon E5-1620, 64 GB ram, RH7)
Bugs fixed:
Bad buffer size over 500GB input files. All systems
New features:
--skip-headers=n : skip n records from all input files
(when --skip-header=n skip n records only from first input file.
----------------------------------------------------------------------------
Release Notes for Version 6.90 RC5 :
13-May-2014
Bugs fixed:
Segmentation fault fix : Thread stack size tuned. AIX 6.1
Wrong count on OMIT DUPLICATE RECORD with VFIELDS. Fixed
New features:
Ported to Windows 64-bits. Here it is!
----------------------------------------------------------------------------
Release Notes for Version 6.90 RC4 :
26-April-2014
Bugs fixed:
Segmentation fault fix : Thread stack size tuned. AIX 6.1
New features:
Optimization on OMIT DUPLICATE RECORDS + FIELDS=ALL operation
----------------------------------------------------------------------------
Release Notes for Version 6.90 RC3 :
15-March-2014
Bugs fixed:
Segmentation fault fix : Thread stack size increased. AIX 6.1
----------------------------------------------------------------------------
Release Notes for Version 6.90 RC2 :
13-March-2014
Bugs fixed:
Memory allocation improvement. All systems
New features:
Display Error message then stop when RECFM=V and record length is smaller
than sort keys, to prevent segmentation fault.
----------------------------------------------------------------------------
Release Notes for Version 6.82 RC4 :
26-Oct-2013
Bugs fixed:
Limit threads and memory allocation on big machines with many CPUs. All systems
----------------------------------------------------------------------------
Release Notes for Version 6.82 RC2 :
08-Oct-2013
Bugs fixed:
Wrong Serial/Number displayed on AIX6.1. Fixed
----------------------------------------------------------------------------
Release Notes for Version 6.82 :
28-Sep-2013
Bugs fixed:
1) Memory allocation improvement. All systems
2) Include/Exclude filtering bug. Fixed
----------------------------------------------------------------------------
Release Notes for Version 6.81 :
09-Jun-2013
Bugs fixed:
1) AIX, Solaris, Linux : hang/segmentation fault could occur on low memory condition. Fixed.
2) input/output file mismatch with COPY operation. Fixed.
----------------------------------------------------------------------------
Release Notes for Version 6.80 :
03-Jun-2012
Bugs fixed:
1) Error in parsing AND/OR conditions in INCLUDE/EXCLUDE statements. Fixed.
2) Could not use COMP-3 fields in the include or omit. Fixed.
New features:
Up to version 6.79, XSM Serial Number used for licence activation was calculated on IBM Hardware Serial
Number and LPAR Id. Thus it was not possible to move a LPAR running XSM from a system to another.
Upon clients requests and specialy for clients using IBM POWER Live Partition Mobility (LPM), XSM from
v6.80 is now only bound to AIX Operating System and no longer to Hardware, making it possible to move
a LPAR during XSM jobs.
----------------------------------------------------------------------------
Release Notes for Version 6.79 :
18-Apr-2012
Bugs fixed:
1) SUN SPARC Solaris 5.8/5.10, IBM Power AIX 5.3, 6.1 : Wrong Memory allocation. Fixed.
2) SORTOFn for selective output (all systems). Fixed.
Misc:
Performance improvement again and again ...
/usr/bin/sort sz=1GB Elapse: 314 sec.
hxsm671_64 sz=1GB Elapse: 113 sec.
hxsm679_64 sz=1GB Elapse: 77 sec.
(IBM p510 lcpu=4 mem=7168MB ent=0.30 AIX6.1)
----------------------------------------------------------------------------
Release Notes for Version 6.78 :
05-Dec-2011
Bugs fixed:
1) SUN SPARC Solaris 5.8/5.10 : XSM007E_0174 Cannot open Input file. Fixed.
2) IBM Power AIX 5.3, 6.1 : Wrong Memory allocation when low available free memory. Fixed.
----------------------------------------------------------------------------
Release Notes for Version 6.77 :
01-Apr-2011
Bugs fixed:
1) OUTREC FIELDS format incorrectly parsed.
2) inputfile read error could occur on +30GB (SPARC Solaris)
Misc:
Performance improvement on Multitasking and I/O operations
Ex: sort 10 GB (100 millions records x 100 bytes, no sortwork), Intel Core2 E8500, 2GB Ram
Linux-64:
V671 : total 370 sec.
V677 : total 300 sec.
W32:
V671 : total 850 sec.
V677 : total 550 s.
New features:
API released. Included as part of XSM Software licence.
Delivered as static and dynamic libs + C/C++ header.
Compatible with all programing languages which are standard Link Edit compliant
(C, C++, VB, .Net, Pascal, ...)
----------------------------------------------------------------------------
Release Notes for Version 6.71 :
18-Feb-2010
Bugs fixed:
1) Text records too long and IOERROR IGNORE could cause a crash in sort operation.
2) Too many long file names in INPFIL statements caused a crash.
Buffer for input filenames was 4K, too short for +100 long pathnames.
Now use dynamic allocation : no more limit on number of input files.
3) VFIELDS (all systems) : on large files, following error could randomly occur:
"XSM063E The file xxxxx/srtw3254.03a is not sorted at record # yyy"
4) minor correction in INCLUDE/EXCLUDE operation precedence
New features added:
--skip-head=n : will drop the first n records of the first input file
Usefull to throw away a header/title
Minor updates :
1) Linux : no more need to be root to install hxsm
2) Any unknown parameter is flagged as an error [instead of ignored],
and the job is cancelled
3) if RECFM=V, the record length is automatically increased by 2,
thus permitting the user to give a LRECL without dealing with
'carriage-return' and 'line-feed' chars.
4) Memory, sortworks, threads optimization for huge files (+100GB)
----------------------------------------------------------------------------
Release Notes for Version 6.68 :
01-Apr-2009
Port to HP-UX 11i v3 IA64 Itanium (uname says: HP-UX B.11.31 U ia64 HP-UX)
----------------------------------------------------------------------------
Release Notes for Version 6.68 :
15-Mar-2009
Bug fixed:
- VFIELDS (all systems)
Major performance improvement. We now sort 1 Gigabyte in less than a minute on an inexpensive PC!
v6.62 : XSM031I END OF JOB 1gt.xsm 10000000 rec.: 80 SEC.
v6.68 : XSM031I END OF JOB 1gt.xsm 10000000 rec.: 42 SEC.
(Intel Core2 2.2Ghz, 2GB RAM, 1 x IDE 7500rpm, Solaris 10 x86)
----------------------------------------------------------------------------
Release Notes for Version 6.64 :
19-Feb-2009
Bug fixed:
- Free storage detection (AIX, Solaris)
- Thread stack size allocation (AIX)
- dynamic STORAGE allocation (all systems), affecting sortworks size + quantity
New features added:
Verbose mode (-vv) now shows real memory + working dirs + threads being used.
----------------------------------------------------------------------------
Release Notes for Version 6.62 :
07-Feb-2009
Bug fixed:
- OUTREC option
With this option, one can reformat the final output records with padding and constant fields,
and with selected fields from the sorted records.
Option may be speciped in 3 ways :
format #1 : command line option : --outrec=(specs[,specs...])
format #2 : Parameter file : OPTION OUTREC=(specs[,specs...])
format #3 : Parameter file : OUTREC FIELDS=(specs[,specs...])[,RECFM=...][,LRECL=...]
where specs ::= start,length[,type,offset]
start = field start in input record (mandatory, from 1)
length = field length (mandatory, may be < 0 for last field if RECFM T)
type = field type (optionnal)
offset = field offset in output record (optionnal, from 1)
If is omitted in 1st spec, they must be omitted in all specs,
and fields are concatened in output record
Example
hxsm ... '--outrec=(14,20,1,,..6)'
OPTION OUTREC=(14,20,1,6,...)
OUTREC FIELDS=(14,20,1,6,...)
are equivalent to :
hxsm ... '--outrec=(14,20,C,1,1,6,C,21,...)'
OPTION OUTREC=(14,20,C,1,1,6,C,21,...)
OUTREC FIELDS=(14,20,C,1,1,6,C,21,...)
---------------------------------------------------------------------------
Padding and constant fields:
The output record may include fields that are not in the corresponding input record.
These fields may be :
alpha-numeric fields, padded with a letter, a digit, a space, or any other byte
zoned-numeric fields, padded with a digit (most of time 0)
packed-numeric fields, padded with a digit (most of time 0)
In this case, 'specs' syntax are as follows:
1) any_char,length,S,offset : alpha field padded with spaces
example:
OPTION OUTREC=(...,X,10,S,42,...) # cols 42 to 51 padded with spaces
'--outrec=(...,X,10,S,42,...)' # same
2) letter,length,C,offset : alpha field padded with 'letter'
example:
OPTION OUTREC=(...,Z,10,C,42,...) # cols 42 to 51 padded with letter 'Z'
3) digit,length,N,offset : numeric field padded with 'digit'
example:
OPTION OUTREC=(...,0,10,N,42,...) # cols 42 to 51 padded with '0'
4) value,length,B,offset : alphanumeric field padded with byte decimal 'value' (0..255)
example:
OPTION OUTREC=(...,0,8,B,42,...) # cols 42 to 49 padded with binary zero
OPTION OUTREC=(...,20,8,B,42,...) # cols 42 to 49 padded with (ascii) space
OPTION OUTREC=(...,255,8,B,42,...) # cols 42 to 49 padded with HIGH-VALUE (255)
5) digit,length,P,offset : packed numeric field value 'digit'
example:
OPTION OUTREC=(...,0,3,P,42,...) # cols 42 to 44 = x'00000C' (packed 0)
OPTION OUTREC=(...,-1,3,P,42,...) # cols 42 to 44 = x'00001D' (packed -1)
6) digit,length,Z,offset : zoned numeric field value 'digit'
example:
OPTION OUTREC=(...,0,3,Z,42,...) # cols 42 to 44 = x'F0F0C0' (zoned ebcdic +0)
x'30307B' (zoned ascii +0)
OPTION OUTREC=(...,9,3,Z,42,...) # cols 42 to 44 = x'F0F0C9' (zoned ebcdic +9)
x'303043' (zoned ascii +9)
OPTION OUTREC=(...,-1,3,Z,42,...) # cols 42 to 44 = x'F0F0D1' (zoned ebcdic -1)
= x'303071' (zoned ascii -1)
----------------------------------------------------------------------------
hxsm V658 - Multitasking - Nov 2007 -
Supported platforms:
IBM pSeries RS6000 Power3-Power5 - AIX 4.3 AIX 5.2 : 32 bits
IBM pSeries RS6000 Power3-Power5 - AIX 5.3 : 32/64 bits
IBM Z-series - zLinux-390 (Suse 9) : 32 bits kernel=2.6.5
IBM Z-series - zLinux-390 (Suse 9) : 64 bits kernel=2.6.5
Intel Pentium - Linux 2.6 (>FC4) : 32 bits kernel>=2.6.11 glibc>=2.3.5
Intel Pentium - Win32 x86 (NT4, W2K, XP) : 32 bits
Sun SPARC Solaris 8, solaris 9, solaris 10 : 32/64 bits
Release Notes for Version 6.58 :
26-Oct-2007
Several bugs fixed:
- sortwork allocation (Solaris 8)
- dynamic STORAGE allocation (all systems)
- KEEP_ORDER (all Systems)
- improved traces for debug mode (All Systems)
----------------------------------------------------------------------------
Release Notes for Version 6.56 :
Nov-2006
This version is a major enhancement of XSM
'Basic' version 4 limited to 4GB input file size is abandoned.
From version 6, XSM is now released in 'Pro' version without software limitation.
A) New features added :
========================
1 - Almost all options can be used in a command line, using GNU style long options
(i.e. --lrecl=250 instead of -l 250 )
2 - New field types :
numeric ('N') : fixed length numeric fields with a leading floating sign
(-ie +1 -124 ....)
zoned decimal ('Z' or 'ZD') : fixed length numeric fields with a trailing sign
on the last digit ( "PIC S99999 USAGE IS DISPLAY" ) ...
Both methods (Microfocus-like and ACU COBOL-like) are supported.
Note: This is different from 'packed decimal' ('P' or 'PD', COBOL "COMP-3")
3 - Support of MicroFocus COBOL special variable format (known as "MFCOBOL" or "MFVariable") : RECFM=M
4 - Support of multiple format for all input files and the output file :
each input file may have its own record format (F,T,M) and its own record length
the output file may have its own record format (F,T,M) and its own record length
5 - Aditionnal parameter for OUTFILE :
DISP=O if output file exists, then overwrite (default)
DISP=A if output file exists, then append, else create
DISP=N if output file exists, then abort job (same as JCL's DD DISP=NEW)
7 - New 'OUTREC' parameter : records written to the output file may be rebuilt
from the input record, all or some fields, with a different order
8 - Bunch of new options :
--activation-key=your-activ-key # if you don't want to 'install' hxsm
--keep-order # if you want a Non-Destructive sort
--throw-empty-records # as you can expect ...
B) Bugs fixed:
===================
1) Option '-q' (new fashion : '--quiet') will cause XSM to exit silently
with a return code, whatever the cause of the error.
2) INCLUDE/EXCLUDE expressions containing constant C' ' (a blank) :
This caused stopping expression analysis.
This is now fixed
3) Fixed files : in some cases, XSM was writing the output file with blank records.
This is now fixed
C) New syntax for options and command line switches :
=====================================================
+--------------------------------------+---------------------------+------------------------------------------+
| Inside a Parmfile | Old switch | New switch |
| | | |
| SORT FIELDS=(s1,l1,typ1,A/D, ...) | -ks1,l1[,A/D[,typ1] -k... | [--sort] --key=s1,l1[A/D[,typ1] --key=...|
| | | |
| MERGE FIELDS=(...) | -m -k... -k... | --merge --key=.... --key=... |
| | | |
| RECORD RECFM=x,LRECL=nnnn | -rx -lnnnn | --recfm=x --lrecl=nnnn |
| | | |
| OUTFILE name[,RECFM=x[,LRECL=nnnn | -ooutfile | --outfile=name[,RECFM=x[,LRECL=nnnn |
| [,DISP=O/A/N]]] | | [,DISP=O/A/N]]] |
| | | |
| INFILE name[,RECFM=x[,LRECL=nnnn] | name name ...(last parms) | --infile=name[,RECFM=x[,LRECL=nnnn] ... |
| | | |
| SORTWORK dir1[,dir2...] | -tdir1[,dir2..] | --sortwork=dir1 --sortwork=dir2 ... |
| | | |
| STORAGE nnnK/M/G | -ynnnK/M/G | --storage=nnnK/M/G |
| | | |
| COLLATE=ASCII/EBCDIC | (none) | --collating-sequencxe=Ascii/Ebcdic |
| | | |
| OMIT DUPLICATE KEYS | -uk | --unique-key |
| OMIT DUPLICATE RECORDS | -ur | --unique-record |
| | | |
| INCLUDE COND=(s,l,op,const...) | (none) | --include=s,l,op,const[,AND/OR,...] |
| EXCLUDE COND=(s,l,op,const...) | (none) | --include=s,l,op,const[,AND/OR,...] |
| | | |
| IOERROR IGNORE | (none) | --throw-empty-records |
| | | |
| OPTION COPY | (none) | --copy |
| | | |
| OPTION NORUN | (none) | --norun |
| | | |
| OPTION KEEP_ORDER (*) | (none) | --keep-order |
| | | |
| OUTREC FIELDS=(fr_src,len,to_dst, | (none) | --outrec=fr_src,len,to_tgt,type,... |
| (*) typ[,fr_src,len,...)) | | |
+--------------------------------------+---------------------------+------------------------------------------+
(*) : new in version 6.5x
D) Sort/Merge Keys :
====================
FIELDS=(start,len,type,A/D,...) or --key=start,len,A/D,type --key=...
where type may be :
C or CH : Bytes, value 0..255 per byte
I : alpha-numeric field, mixed, where case (upper, lower) must be ignored
B or BI : same as CH in this version, but to avoid :
in future versions, BI may be used for binary values in half-words, words, double-words.
Z or ZD : 'Zoned' decimal, from COBOL 'PIC S99999... USAGE DISPLAY'
set of ascii digits '0'..'9', the last digit holds the sign, and is an ascii char.
P or PD : 'Packed' decimal, from COBOL 'PIC S99999... USAGE COMP-3'
set of BCD pairs of digits, the last pair holds the last digit and the sign.
N : 'Numeric' field, fixed length, with leading blanks and a floating (optional) sign '+/-'
E) Microfocus 'variable' files :
================================
An internal Microfocus 128 bytes header, plus a set af records.
Each record is prefixed by a 2-bytes length (Big Endian, 3xxx),
and is padded to an even number of bytes
Note : fields are defined from col. 1 = first significant bytes of the record (2-bytes length is dropped)
Those files are processed with RECFM=M (max length=4095)
F) Support of mixed files formats and logical records lengths :
===============================================================
For each input file or for the output file, you can specify its own record format (RECFM),
and its own logical record length :
INPFIL pathname,RECFM=F/T/M,LRECL=nnnn # parmfile
--infile=pathname,RECFM=F/T/M,LRECL=nnnn # command line
OUTFIL pathname,RECFM=F/T/M,LRECL=nnnn # parmfile
--outfile=pathname,RECFM=F/T/M,LRECL=nnnn # command line
You can also define a default record format and a default logical record length :
RECORD RECFM=F/T/M,LRECL=nnnn # parmfile
--recfm=F/T/M --lrecl=nnnn # command line
Rules :
if RECFM and/or LRECL is specified for a file, then they are taken in account as expected,
else
if there is a default RECFM and /or a default LRECL, then they are taken in account as expected,
else
default will be RECFM=T,LRECL=250
G) 'OUTREC' parameter :
=======================
This parameter allows building output records with selected fields in the input record.
OUTREC FIELDS=(start_in,length,start_out,type[,start_in...]) # parmfile
--outrec=start_in,length,start_out,type,... # command line
where :
start_in : offset (from 1) in the input record
length : field length, in byte
start_out: offset (from 1) in the output record
type : ignored, compatibility purpose
examples :
Building a 30 bytes output record with contiguous fields from cols. 12-14, 1-10, 14-30
from input record :
OUTREC FIELDS=(12,3,1,C,1,10,4,C,14,17,14,C) # parmfile
--outrec=12,3,1,C,1,10,4,C,14,17,14,C # command line
Building a 80 bytes output record with non-contiguous fields from cols. 12-14, 1-10, 14-30
from input record :
OUTREC FIELDS=(12,3,1,C,1,30,4,C,14,17,64,C) # parmfile
--outrec=12,3,1,C,1,10,4,C,14,17,14,C # command line
Note : 'holes' will be padded with blanks.
H) Non-destructive sort :
=========================
For each set of output records sharing the same sort keys, the original input order will be preserved.
This is done internally by adding the input record number as the last ('minor') sort key.
OPTION KEEP_ORDER # parmfile
--keep-order # command line
Note : this option is NOT compatible with '--unique-record', or '--unique-key'
K) Activation key :
====================
If you don't want to 'install' the hxsm program once for all using the supplied 'hhnsinst',
you may specify your activation at each run time :
OPTION KEY=XXXX-XXXX-XXXX-XXXX-K # parmfile
--activation-key=XXXX-XXXX-XXXX-XXXX-K # command line
This feature will be usefull when the XSM API will be released (soon ...), to run home-brewed
sort/merge applications on different workstations.
L) Multitasking ('Multi-threading') :
=====================================
This feature is usefull when sorting (not merging nor copying) large files.
Its purpose is to 'parallelize' the process of reading the input files by chunks,
then sorting and writing each chunk onto a sortwork file.
It gives pretty good performances when several sortwork physical drives are available.
XSM launch one subtask ('thread') per sortwork directory available, when specified.
If you have only one sortwork directory, or did not specify any (default = current directory),
XSM will NOT start multitasking.
You can force multitasking (for your own curiosity satisfaction) by specifying :
OPTION PROCS=n # parmfile
--procs=n # command line
where "n" is the the number of subtasks that XSM will launch (n=2..4 are good numbers)
When physical drives are managed by 'SAN-like', 'RAID-2', or 'VG/LV' subsystems, it may work well.
M) Following soon :
API (Application Programming interface)
Support of C-Isam Files
Feedback is welcome (support@hhns.fr)
HHNS XSM : Extended Sort/Merge Program,
PROfessionnal version multi-threading (V6)
PROfessionnal version (V5)
Basic version (V4)
Current versions: V5.30 (Pro) / V4.70 (Basic) - Fev 2006
Supported platforms :
IBM PowerPC - AIX 5.3 : 32/64 bits
Intel Pentium - Linux 2.6 (FC4) : 32 bits kernel=2.6.11 glibc=2.3.5
Intel Pentium - Linux 2.6 (FC2) : 32 bits kernel=2.6.5 glibc=2.3.3
Intel Pentium - Linux 2.4 (RH9) : 32 bits kernel=2.4.11 glibc=2.3.2
Intel Pentium - WXP/2K : 32 bits
IBM zNNNN - zLinux 2.6 (Suse9) : 32/64 bits kernel=2.6 glibc=2.3.3
Bug fixed:
===============
EXCLUDE/INCLUDE : last block of data was dropped when recfm=F
----------------------------------------------------------------------------
Release Notes for Version 4.66 / 5.26 :
30-Sep-2005
Supported platforms :
Linux-x86 - kernel 2.2, glibc 2.2.4 (RedHat 7.2)
(==== should run on ALL linux-i386 versions ===)
Linux-x86 - kernel 2.4, glibc 2.3.2 (RedHat 9)
Linux-x86 - kernel 2.6, glibc 2.3.3 (Fedora Core 2)
Win32 - NTFS (NT, 2K, XP) and FAT32 (NT, 2K, XP, 95, 98)
AIX-powerpc-4.3 32-bits
AIX-powerpc-5.3 32-bits
AIX-powerpc-5.3 64-bits
zLinux-S390 32-bits - kernel 2.6, glibc 2.3.3 (Suse 9)
zLinux-S390 64-bits - kernel 2.6, glibc 2.3.3 (Suse 9)
Sunos-sparc-5.8 32 bits
improved OUTFIL parameter (for compatibility with some SORT feature on Main Frames) :
1) Usage : you want to get multiple output files, depending on a selection
in the final output stream;
2) Syntax :
OUTFIL FILE=nn,INCLUDE=(startcol,length,type,operator,value)
or OUTFIL FILE=nn,EXCLUDE=(startcol,length,type,operator,value)
or OUTFIL FILE=nn,INCLUDE=ALL
where :
nn : 01 .. 99 = output file number, to be appended to the
ddname 'SORTOF' ( = system variable 'SORTOF')
startcol,length : field definition
type : always 'CH'
operator : one of 'EQ' 'NE' 'GT' GE' 'LT' 'LE'
value : C'...' = constant to be matched against the field
(You indeed may have several occurences of the same output record in different output files)
3) Example :
UNIX :
SORTIN="/indir/the_big_file" ; export SORTIN
SORTOF1="/outdir/code_01" ; export SORTOF1
SORTOF2="/outdir/code_02" ; export SORTOF2
SORTOF99="/outdir/code_03_99" ; export SORTOF99
Windows :
set SORTIN=D:\indir\the_big_file.dat
set SORTOF1=E:\outdir\code_01.dat
set SORTOF2=E:\outdir\code_02.dat
set SORTOF99=E:\outdir\code_03_99.dat
Parm file (all Oper.Systems) :
SORT FILEDS=(1,10,CH,A) ; --- sort input by costumer number
RECORD RECFM=V,LRECL=250
INPFIL DD:SORTIN ; --- define input file as a 'ddname'
; --- define 3 output files, depending on zip code col.37-38
OUTFIL FILE=1,INCLUDE=(37,2,CH,A,EQ,C'01') ; = ddname SORTOF1
OUTFIL FILE=2,INCLUDE=(37,2,CH,A,EQ,C'02') ; = ddname SORTOF2
OUTFIL FILE=99,INCLUDE=(37,2,CH,A,GE,C'03') ; = ddname SORTOF3
Run XSM (all Oper.Systems) :
hxsm526 -v myparms.xsm
The result will be ordered by col. 1-10 (for example, a costumer number),
and divided into 3 files, depending on col. 37-38 (for example, a zip code)
- 1st file ('code_1') will hold all records with code '01'
- 2nd file ('code_2') will hold all records with code '02'
- 3rd file ('code_03_99') will hold all other records.
WARNING :
IN THIS VERSION, THE OPTION "FILE=01 [02 .. 09]" WILL BE HELD AS "FILE=1 [2 .. 9]"
SO, LET YOU DEFINE THE VARIABLES AS 'SORTOF1', 'SORTOF2' ... 'SORTOF9'
AND NOT AS 'SORTOF01', 'SORTOF02' ... 'SORTOF09'
----------------------------------------------------------------------------
Release Notes for Version 4.65 / 5.25 :
01-Jul-2005
Bug fixed:
internal loop during merge operation on RECFM=F files
New OPTIONS subparameter:
OPTIONS NORUN
This will prevent XSM to really do the job.
Use it to check your parmfile.
Change in the format of the VFIELDS Parameter :
VFIELDS=.....,FIELDSEP=....
The FIELDSEP Subparameter value can be a symbolic name, or an hexadecimal value
Symbolic names accepted :
BAR = the '|' char
TAB = the Tabulation (X'09') char
COMMA = the ',' char
COLUMN = the ':' char
DIARESIS = the '#' char
SLASH = the '/' char
BACKSLASH = the '\' char
SEMICOLUMN or SEMI-COLUMN = the ';' char
SINGLEQUOTE or SINGLE-QUOTE = the "'" char
DOUBLEQUOTE or DOUBLE-QUOTE = the '"' char
Hexadecimal value : X'hh' (UPPERCASE X, SINGLEQUOTE, 2 Hex digits, SINGLEQUOTE)
Examples :
VFIELDS=(.....),FIELDSEP=SEMI-COLUMN
VFIELDS=(.....),FIELDSEP=X'7C'
By default, the FIELDSEP value is TAB (X'09')
----------------------------------------------------------------------------
Release Notes for Version 4.59 / 5.19 :
18-Nov-2004
Performances enhanced for large files (> 10GB)
Bugs fixed:
- INCLUDE/EXCLUDE/OMIT : new format
ex :
INCLUDE COND=(1,5,CH,EQ,C'AB CD',AND,21,2,LE,X'F2B3')
EXCLUDE COND=(11,6,CH,EQ,C'XYZ AB')
(note : previous format is obsolete)
- when binary files (RECFM F) are piped to hxsm's standard input,
a premature EOF was set on UNIX platforms
----------------------------------------------------------------------------
Release Notes for Version 4.57 / 5.17 :
18-Jul-2004
Bugs fixed:
- outfile not written when input file size = storage - epsilon
----------------------------------------------------------------------------
Release Notes for Version 4.56 / 5.16 :
26-Aug-2004
Several bugs fixed:
- 'out of memory' when sorting 'worst case' files (All systems).
- Seek error on some very large files (W32 only)
- improved performances on very large files (All Systems)
----------------------------------------------------------------------------
Release Notes for Version 4.55 / 5.15 :
14-May-2004
1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php
www.hhns.fr/products/xsm/en/xsmqref.php)
2) New OPTION statements
OPTION COPY
INCLUDE/EXCLUDE COND=(cond)
(see below for '(cond)' )
3) New OUTFIL statement
OUTFIL FILE=1,INCLUDE/EXCLUDE=(cond)
OUTFIL FILE=2,INCLUDE/EXCLUDE=(cond)
...
with :
FILE=n : the program will search an environment variable ('DDNAME')
of the form : SORTOFn=outfile_name (i=1,2,...)
and (cond) looking like :
(start,len,CH,OP,C'aaa...aaa',AND/OR,cond.... )
with 'OP' = one of EQ/NE/LT,LE,GT,GE
4) bugs fixed:
options OMIT DUPLICATE RECORDS/KEYS did not drop the first duplicate record/key
----------------------------------------------------------------------------
Release Notes for Version 4.51 / 5.11 :
25-Nov-2003
1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php
www.hhns.fr/products/xsm/en/xsmqref.php)
2) New OPTION statement
OPTION COLLATING=EBCDIC : will respect EBCDIC Collating sequence for :
- lower-case
- upper-case
- digits
3) Bug fixed:
Bad sort order for packed decimal fields when sign is xF instead xC or xD
----------------------------------------------------------------------------
Release Notes for Version 4.50 / 5.10 :
24-Aug-2003
1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php
www.hhns.fr/products/xsm/en/xsmqref.php)
2) New parameters for the SORT/MERGE statement
a) SORT FIELDS=all : will sort the entire records (no key definition)
(you must provide max len. record in LRECL=...)
b) SORT FIELDS=(start1,len1,A/D,start2,len2,A/D,...),FORMAT=BI
(if all fields are of the same type : BI, CH, ...)
3) DDNAME for INPFIL/OUTFIL :
the filenames in INPFIL/OUTFIL statements can be given as 'DD:somename'.
XSM will get the actual filename via the corresponding environment variable.
Ex :
INPFIL DD:SORTIN1,DD:SORTIN2
OUTFIL DD:SORTOUT
those two statements will cause XSM to search the actual filenames via a 'getenv'
for the environment variables "SORTIN1", "SORTIN2", "SORTOUT"
----------------------------------------------------------------------------
Release Notes for Version 4.48 / 5.08 :
19-May-2003
The version 5.08 can sort files larger than 2/4 GB.
The STORAGE parameter is no more needed : it is computed dynamically,
at best.
But when sorting very huge files ( > 64 GB ), it is safe to give a
STORAGE parameter less or equal to (total real storage - (64MB or 128MB)),
in order to avoid heavy paging.
----------------------------------------------------------------------------
Release Notes for Version 4.45 / 5.05 :
04-Feb-2003
Correct a bug giving some 'INTERNAL ERRORS' when sorting huge files
of small records, with 'IOERROR IGNORE' and keys larger than the
smallest record.
----------------------------------------------------------------------------
Release Notes for Version 4.44 / 5.04 :
18-Jan-2003
Correct a bug involving VFIELD's processing
----------------------------------------------------------------------------
Release Notes for Version 4.43 / 5.03 :
03-Jan-2003
Correct a bug in memory allocation when sorting more than one file
----------------------------------------------------------------------------
Release Notes for Version 4.42 / 5.02 :
11-Dec-2002
the 'STORAGE' parameter is computed dynamically. For most jobs, the
parameter is not needed.
You may override the result with your own 'STORAGE' parameter,
in case of very large files to sort, when you think you don't
have enough available storage, in order to avoid paging.
----------------------------------------------------------------------------
Release Notes for Version 4. / 5. :
Oct-2002
==> SUPPORT OF FILES LARGER THAN 2GB / 4GB :
tested with a 5 GB file on
- WinNT/4, sp6
- Win2k, sp1
- Linux 386, Kernel 2.4.13, glibc-6
- Linux 390, Kernel 2.4.17, glibc-6
Note : should run on W9x with a NTFS Driver (www.Winternals.com)
V4.33c (apr 2002)
- some minor bug corrected for small binary files
V4.33b:
- some minor bug corrected, that could provoque a crash at initialization
V4.33 :
- major improvement in handling I/O's
- twice faster
V4.13 :
correct a bug in V4.12 that inserted blank lines in text files.
V4.12
some improvement in handling I/O's
================================================================================
Please send bugs reports, or feelings to :
support@hhns.fr or mail@hhns.fr
Enjoy fast sort.
Henri Henault
