New features with AN-2019-12-05: This is the first localization step for the schily source consolidation. Many programs now (hopefully) call gettext() for all strings that need localization. - The next step will include dgettext() calls for the libraries and the missing programs - The following step will include the extracted strings - The last step will include German translations and install support for the resulting binary message object files. ----------> Please test and report compilation problems! <--------- ***** NOTE: As mentioned since 2004, frontends to the tools should ***** ***** call all programs in the "C" locale ***** ***** by e.g. calling: LC_ALL=C cdrecord .... ***** ***** unless these frontends support localized strings ***** ***** used by the cdrtools with NLS support. ***** *** WARNING *** *** Need new smake *** *** Due to the fact that schily-tools 2014-04-03 introduced to use new macro *** expansions and a related bug fix in smake, you need a newer smake *** to compile this source. If your smake is too old and aborts, ensure to *** use the recent smake by calling: cd ./psmake ./MAKE-all cd .. psmake/smake psmake/smake install The new smake version mentioned above is smake-1.2.4 The recent smake version is smake-1.3 *** Due to the fact that schily-tools 2018-01-26 introduced *** optimizations for the Schily version of SunPro Make, you *** need at least the dmake version from 2018/01/11 with support *** for the "export" directive to compile with this makefile system. WARNING: the new version of the isoinfo program makes use of the *at() series of functions that have been introduced by Sun in August 2001 and added to POSIX.1-2008. For older platforms, libschily now includes emulations for these functions but these emulations have not yet been tested thoroughly. Please report problems! BUG WARNING: Please never report bugs only to Linux distributions as they usually do not forward these bug reports upstream and as the Linux distributions typically do not let skilled people check the bugs. We did not hear about a FIFO problem in star for a long time. Then a problem on Linux occurred once every 6000-10000 tries but it did not happen on Solaris after even 10 million tries, so it was not known besides Linux. - Schily.Copyright: various Copyright dates have been updated - Makefile System: The file TARGETS/55cal had been forgotten to include into the schilytools tarball and as a result, the program "cal" (the UNIX original cal(1) with addet support for the different switch dates from the Julian to the Gregorian calender) was not compiled or installed from the publised schilytools source tree. - autoconf: xconfig.h.in now has " P " markers in the comment for the *at() functions that are mentioned in POSIX. This helps to distinct transient vendor specific interfaces from the standard POSIX *at() interfaces. - Packaging: The package OCSWschilyutils now adopts the new dotfiles installation paths and includes to install the files share/doc/bosh/dotfiles.tar.bz2 share/doc/bsh/dotfiles.tar.bz2 share/doc/ved/dotfiles.tar.bz2 - Packaging: The package OCSWschilybase now contains a lot more man pages for libschily, new include files and libstreamar. - Packaging: The package CSWschilyutils added "printf", "strar" and some man pages - Packaging: The package OCSWstar now includes several new test scripts - Packaging: The package CSWschilyutils added the new fifo(1) program, see note below on the new fifo program. - include/schily/stdio.h: We no longer define getc_unlocked() on Solaris in case that it is defined already. This was needed since Oracle Solaris 11.4 compiles 64 bit by default and made stdio less opaque than before - this result in the same bahavior as for 32 bit programs. This means, we no longer need to use tricks to get fast access to stdio. Thanks to Rolf Dietze for reporting. - libschily: the function getargs() and similar (from the option parser group) fixed a bug that prevented the usability of the official method to include the char '+' inside option names by quoting the character in the option name used in the format string. This bug has been fixed in order to be able to support "star -dump+ ..." - libschily: the function getargs() and similar (from the option parser group) now support an extended format specifier: Previously, %0 and %1 have been supported, now all single digit numbers %0 .. %9 are supported. - libshedit: because of the changes in Oracle Solaris 11.4, we need to #undef FAST_GETC_PUTC in order to avoid to use getc_unlocked(). Thanks to Rolf Dietze for reporting. - libgetopt: The linker mapfile did not contain an entry for "optflags" and as a result, dynamic linking of the Bourne Shell did not work against libgetopt. This is however only possible if a user manually compiles libgetopt as a shared library. By default, this lib is compiled statically only. - libfind: A new function walksname() has been added to set up the internals in "struct WALK" for external path names. This feature is needed to support "star -c list=names -find ...". - libfind: The functions fetchdir()/dfetchdir(), sortdir(), cmpdir() now use size_t instead of int for the entry number count and the allocated size. This breaks binary compatibility in 64 bit mode for those users only, that use the interfaces fetchdir()/dfetchdir(), sortdir(), cmpdir(). The only application that currently uses these interfaces is "star". Simple users of libfind and programs compiled as 32 bit programs are not affected by the change. Warning: packetizers need to take care of the changed name of the shared library file for libfind. - libfind: Because of the binary incompatible change, we incremented the major library version number from 3 to 4. WARNING: This affects only distros that ship 64 bit versions of star using a shared version of libfind. Since Solaris defaults to 32 bit programs, this is expected to only affect Linux distros. - libstreamar: unicode.c has been changed to make sure that is works even in case that the "len" parameter is an unsigned. - libstreamar: to_utf8() from unicode.c has been rewritten to use size_t instead of int. Several changes have been introduced in the other code as well. - libstreamar: from_utf8() from unicode.c has been rewritten to use size_t instead of int. Several changes have been introduced in the other code as well. - Bourne Shell: The Makefile did not contain a call to Makefile.dot and as a result, it did not install dotfiles.tar.gz - Bourne Shell/bsh: */hashcmd.c did not handle a malloc()ed string correctly and could cause a memory leak with hash commands that do not add new aliases. We now create a copy using make()/makestr() before calling ab_push() or ab_insert() and always free "name" at the end of the function hashcmd() - Bourne Shell: The man page now documents that the command "set -o hashcmds" inside the file $HOME/.shrc makes a # on the fist column of that file and followed by a non-space character, no longer a comment. It is thus recommended to have "set -o hashcmds" close to the bottom of the file $HOME/.shrc - SunPro Make: When in parallel mode, a conditional macro assignment like: all := BLA += 1 2 3 has been evaluated twice and "BLA" got "1 2 3 1 2 3" appended when processing the "all" target in parallel mode. This was caused by a misplaced #ifdef from April 2017, when we enabled the parallel mode in the OSS version of SunPro Make. So this bug was present in all previous OpenSource version of "dmake" from schilytools and inside the SchilliX-ON tree. - SunPro Make: The package definitions for CSWmake have been added to packages/OCSW - SCCS: The change from the last schilytools version that included a new delta type "U", was not "complete" with regards to expected overall behavior: - SCCS: the get program now includes 'U' typed deltas in the l-file output. - SCCS: the prs program now includes 'U' typed deltas in the output. - SCCS: the prt program now includes 'U' typed deltas in the output. - SCCS: the val program now includes 'U' typed deltas in special checks. - SCCS: the prs program now uses the same rules as other utilities when checking the -r option argument for compliance. - SCCS: the val program now uses the same rules as other utilities when checking the -r option argument for compliance. - star: The new-volume-script= script is now called at the end of _every tape_, even at the last one. This now allows to have a script that automatically renames a standard archive name into numbered archive names when a tree should be splitted into chunks, e.g. with this command line: star -c f=/tmp/v.tar tsize=1G new-volume-script='sh -c "mv /tmp/v.tar /tmp/v$(($1-1)).tar" nv' . you get partial archives named /tmp/v1.tar ... /tmp/vn.tar Note that this would not work in case that "sh" is the historical Bourne Shell, e.g. as seen on Solaris 10. This is because "shell arithmetics" is a feature that was not present in the historical Bourne Shell. - star: The man page now mentions that "star cli=tar ..." works as an alias to "star cli=suntar ...". - star: The man page now mentions that "star cli=gnutar ..." does not impement the bugs from the GNU option parser. - star: Is "star cli=xxx ..." is used, then star now calls set_progname(xxx) and thus prints error messages labelled with the command line interface name. - star: In "pax" command line mode, star now uses -no-fsync as default in order to behave the same way as the closed source Solaris pax does. - star: Star now allows to combine the option list=names with -find This is useful, whenever a list of filenames is longer than ARG_MAX and thus cannot be used in the star command line after -find. Here is an example on how to use the new feature in a mercurial based development environment: (hg status -a; hg status -m; hg status -c) | sed -e 's/^. //' > .hgfiles star -c list=.hgfiles -find -mtime -100 > /tmp/new.tar The first command puts all files under control of mercurial into thw file ".hgfiles" , while omitting the compile results from the file ".hgfiles". The second command takes that list and puts only those files into the archive that have been modified during the past 100 days. - star: A new option "-dump+" allows to have the "SCHILY.volhdr.hostname" meta data field filled out even when not making dumps with an associated dump level. This helps to remember the origin of tar archives. - star: unicode.c has been changed to use the same comment as unicode.c in libstreamar. - star: unicode.c has been changed to make sure that is works even in case that the "len" parameter is an unsigned. - star: to_utf8() from unicode.c has been rewritten to use size_t instead of int. Several changes have been introduced in the other code as well. - star: from_utf8() from unicode.c has been rewritten to use size_t instead of int. Several changes have been introduced in the other code as well. - star: due to the incompatible interface change in libfind (see above), several changes have been introduced in order to use size_t instead of int for fetchdir()/dfetchdir(), sortdir(), cmpdir(). This finally permits star to be able to make use of the current POSIX maximum meta data size for so called "pax" archives (TAR archives with POSIX.1-2001 meta data enhancements derived from the Solaris 7 concept for tar from 1997). Previous versions of star have been limited to data no longer than 2 GB (even though the star main code supports 8 GB file size), this version now supports 8 GB for the sum of all meta data for a single file in case a 64 bit binary from star is used. - star: the fifo now has a new variant of te function runfifo() that is used by the new fifo(1) standalone program. See below. - star: The README now mentions the new "star cli= ..." method and the fact that star deals with infinite path name length. - bsh/Bourne Shell/ved: The mapping system in */map.c now includes compiled in fallback mappings for "xterm" function keys to help people to work with these commands in case there are no configuration files in the users home directory. This is typically the case for "just installed" machines. - patch: The man page now contains a "SOURCE DOWNLOAD" section. - fifo: added a new program that is based on the fifo code from star. "Fifo" is a buffer that reads from stdin, writes to stdout and keeps the data in shared memory shared by the decoupled read and write processes. This is a revived version of how the fifo code has been developed 30 years ago. - SCCS TODO: - Convert sccs.c to use -NSCCS in the back end programs instead of converting g-file names from the command line into s.file names in the frontend in order to forward s.file names to the backend programs. Doing it this way allows us to have the SCCS history files "off tree" in a separate directory tree starting with ".sccs/" at the "project root" directory, as the related code to convert path names is in libcomobj::bulkprepare(). - Add code to admin(1) and delta(1) to retrieve the list of new or modified files in order to have all important code for a "sccs commit" in a single program that does not need to deal with ARG_MAX limitations. - Add code to admin(1), delta(1) and get(1) to maintain/understand the changeset file. - Add code to libcomobj to understand the changeset file, which is needed in order to e.g. know the file names and file specific SIDs/state that correspond to a project global SID. - Add code to all programs to maintain a lock (for the file $PROJECTHOME/.sccs/SCCS/s.changeset, that would be in the file $PROJECTHOME/.sccs/SCCS/z.changeset) to maintain a project global lock. - Find/verify a complete transactional model that allows to repair complex changes to the set of files for a project that have been aborted in the middle. The current idea is to create the file $PROJECTHOME/.sccs/changeset with the deltas to the changeset during a complex update operation. - Find a way to decide when to use SCCS v6 type history files. For the project mode, this is needed in order to prevent historic SCCS implementations to believe they could modify files without knowing about project global locks. Currently, SCCS V67 needs to be anebale manually for every history file using "admin". - Bourne Shell Missing features for POSIX compliance: - Support for $'...' quoting (this is not needed for the current version of POSIX but for the next POSIX version that will be named SUSv8). The development of SUSv8 will start in late 2016. We are now expecting the Bourne Shell to be fully POSIX compliant. - Bourne Shell further TODO list: - Finish loadable builtin support. - POSIX does not allow us to implement ". -h", so we will add a "source" builtin to be able to implement "source -h" - The following builtins (that are available in bsh) are still missing in the Bourne Shell: err echo with output going to stderr glob echo with '\0' instead of ' ' between args env a builtin version of /usr/bin/env The following bsh intrinsics are still missing in the Bourne Shell: - the restricted bsh has restriction features that are missing in the Bourne shell. - source -h read file into history but do not execute and probably more features not yet identified to be bsh unique. Author: Joerg Schilling D-13353 Berlin Germany Email: joerg@schily.net, joerg.schilling@fokus.fraunhofer.de Please mail bugs and suggestions to me.