|Example One||Just an example ...|
3.12 The Joys of NFS and OpenVMS
Lately I have been making extensive use of NFS on OpenVMS VAX coupled with Ubuntu Linux. Unfortunately, being on VAX means that there are no new versions of HP TCP/IP Service past V5.3 ECO4 and OpenVMS VAX V7.3 and its associated patches. This means that when you run into a problem, which doesn't happen too often, you're on your own. As luck would have it, this is exactly where I found myself with NFS.
The configuration I have is a system running Ubuntu Server 12.04 LTS (dockmaster) which exports a directory under my home directory, /home/tsneddon/vms. This is in turn is mounted by my OpenVMS VAX V7.3 system (bender) as DNFSn: via TCP/IP Services V5.3 ECO4 using the following command:
$ TCPIP MOUNT DNFS0: GIT DISK$GIT -
The TCPIP MOUNT process works without issue. The problems I ran into came about when I tried to create a version of a file, greater than ;9. Here is a snippet from my terminal session thats demonstrates the issue:
$ COPY NLA0: DNSF1:T.T
This issue is not just with the COPY command. It is just easy to demonstrate the problem in this way. Unfortunately, I discovered this problem after substantially editing a file and then trying to save it. As you can imagine, I lost everything. After discovering this I searched the 'net looking for someone who might have encountered the same situation. I also looked through reported problems in the release notes. However, it looks as though I might be the first to encounter this problem as I did not find anything. So I began investigating myself.
I tried a lot of different adjustments to the TCPIP MOUNT command and could not find a solution where I could create files with a version higher than ;9. I could not find any indication of what exactly was wrong with the create. There were no error reports in any log file I could find and nothing sent to OPCOM.
After spending way too much time messing around with this I decided I would try replacing the NFS client images installed on my system with those from V5.3 ECO3 (the only V5.3 kit I could find). Unfortunately the images in this kit had the same problem, so I went back further to the next latest kit, which was the V5.1 ECO5 patch kit. These worked!
So, in case someone else encounters this unfortunate problem I have packed up the necessary images, which can be downloaded at the bottom of this post and the following inscructions can be used to install them. Begin by downloading the ZIP file containing the images and unpack them. This directory is referred to as SYS$DISK: in the example below.
It's not a guarenteed fix (or even a very nice one). However, I have not had any further issues related to NFS when accessing an export served by dockmaster. I would like to hear whether it worked or not for others, so feel free to comment below.
The following ZIP file contains all the necessary files:
I would like to acknowledge the following sources in producing this blog post:
3.16 C-Kermit for Android
Recently I purchased an Acer Iconia Tab A200 with the intention of using it as my primary travel computing device, relegating my Compaq CQ41 laptop to pretty much a desktop role. I had looked around to ensure that there would be adequate software to do most of the things I needed and believed I had most bases covered. The most important was network communications. I do most of my work on OpenVMS and Linux servers from a command line and needed a telnet utility as well as serial communication software for hooking up to routers and things.
Initially I worked with ConnectBot and android-serial-api. Both reasonable, but not up to scratch for me. ConnectBot did not have decent keyboard support for my needs. Unfortunately the android-serial-api example application for basic serial comms is abismal. I needed something with a bit more.
The first thing I did was get Android-Terminal-Emulator, an open source VT100 terminal emulator for Android. While it gave me access to the command line and full use of the keyboard, as well as the ability to see /dev/ttyUSB0, I still didn't have an application for going much further. That is when I started looking at C-Kermit. For years I have used the software on both OpenVMS and UNIX (and its derivatives). I've even written some clever scripts for collecting and archiving router configurations and other stuff. However, I had never delved too deeply into its internals. The most I had done with the source code was build it using the procedures provided. Given that it always seemed to just work, there never was any need to. So, after a bit of looking around I couldn't find an Android port and so I decided I'd develop one. The instructions below detail what is needed and how to build C-Kermit for Android as well as providing pre-built images.
Firstly, there were not a lot of changes that needed to be made. A couple things needed to be #ifdef'd around, but largely the source code compiled without issue. To build your own C-Kermit for Android from source, you will first need to install the Android NDK and ensure that the procedure ndk-build is in your path. From here you can grab the source code from github:
$ git clone git://github.com/tesneddon/cka.git
or by downloading the source kit and unpacking it:
$ mkdir cka
Now, build the software using the Android NDK:
$ ndk-build NDK_PROJECT_PATH=./ APP_BUILD_SCRIPT=./Android.mk APP_ABI=all
To build the Android version of kermit for all ABI's I define APP_ABI to all. However, this is not necessary if you are only building for your own device. You can drop this definition to pickup the default or you can define it for a specific ABI.
As for installation, well I used ftp to download the kermit image to my tablet and used the following to install the image. I am assuming with the following instructions that the kermit archive has been downloaded from one of the links below. I must also point out that my tablet has been "rooted" and the following commands have been executed after assuming superuser access with the 'su' command.
$ cd <download-dir>
I have made sure that all of the commands above should work without having to install a shell other than the one that comes pre-installed, at least on the Acer release of Android 4.0.3 (Ice Cream Sandwich). If I am wrong, or there is a better way of doing it, then please feel free to comment below.
You should now be able to now start C-Kermit and see something similar to the following in your terminal emulator:
$ alias telnet="kermit -J"
Getting the Software
The source code referenced in this blog can be found as a project at github, tesneddon/cka. Please feel free to fork, comment and report issues related to C-Kermit on Android here. I have notified the maintained of C-Kermit, Frank Da Cruz, and hopefully these changes will be rolled into the official C-Kermit source code. I will post further updates on this to the comments of this post.
For those of you not interested in building C-Kermit yourself, I have provided a some pre-built images for you to download here:
A nightly snapshot of the source code, as well as pre-built binaries (linked above) can all be found here.
The following sites were instrumental in being able to build this software:
3.17 Configuring Emulated Ethernet for Charon-AXP
In the last month or so I have been involved in configuring an Alpha virtualisation solution for a client. They have an AlphaServer DS15A which they would like to move to an emulated CHARON-AXP DS10L. Until it came to configuring the network the whole process had been very simple and easy. However, once I started setting up the network interfaces I ran into plenty of trouble. After spending much time searching around the 'net and experimenting with the configuration I finally got something that worked. Unfortunately, finding all this information was not very straight forward (at least not to me) and hopefully everything I have documented here will save someone else stuck in the same situation.
Charon-AXP needs its own adapter to support an emulated device. The best way to load the drivers and take control of the card is to use the Network Control Center utility that comes with the emulationg software. I used version 1.26. If you don't have it you can download it here. Use this utility to allocate a network adapter for exclusive use by Charon-AXP and remove all other extensions and protocols. I used a Broadcom NetXtreme II as well as another Broadcom card for the storage and network adapters for host system. Something worth noting is that the host network configuration was using LACP teaming. These drivers will show up as being active on all the network adapters on the host system. You can safely ignore this, it does not appear to affect the Charon-AXP emulation.
The last thing that Network Control Center will do is to spit out a sample configuration snippet for the network adapter (similar to the one below). Use this only as a very rough guide. Trusting this output is partly what got me into trouble in the first place. You'll also notice the string "connection:EWA0" in there too. I renamed my network adapters to reflect their use by the Charon-AXP product.
1. NIC connection name (preferred).
If you are using a system that has built in ethernet, like a DS10L then you can ignore the line 'load DE500BA/dec21x4x EWA interface=EWA0'. The device EWA is already available as part of the default configuration. The next part, 'load ndis5_chpack_port/chnetwrk', is the tricky bit. There are a set of rules that determine the different driver names to be used. After reading through everything I could find I believe the following set of rules to be accurate, at least at the time of writing, and I request comments from anyone who finds different.
If your host system is...
So, here is a copy of the network adapter configuration for the system I was working on, an AlphaServer DS10L with two builtin DE500 network adapters.
You'll also notice in my snippet that I set the adapter_mode. According to the documentation I could find this has to be configured manually, as you see above. My experience has been that everything needs to be exactly the same. I configured the network adapter in Windows to be 100BaseT-FD as well as the switch port the system was plugged in to, not just in the configuration file. This is the only way I seemed to be able to get everything working reliably. Do not use autosense at all, it just doesn't seem to work.
In my quest to get the network configuration working correctly I discovered a few problems and have included the symptoms and solutions below in the hope that I might give a pointer to someone already well down this path.
When I first started out and was using ndis5_chpack_port (when I should have been using ndis6_chpack_port) I found that the LANCP utility within OpenVMS would report the following information:
$ MCR LANCP SHOW DEVICE EWA0/COUNTERS
I also discovered that although I could not get any traffic to move over the physical interface Charon-AXP was successfully changing the MAC address of the network card after DECnet started. So, ipconfig/all would report the new MAC address. The Cisco 2950 switch would also detect a change which would show up in the output of show mac-address-table briefly. The 'Bytes sent' counter would very slowly increase, but the 'Carrier check failures' would increment with astronomical speed.
Without the adapter_mode attribute configured I would also notice that the Cisco switch port would auto-negotiate the link speed to be 100BaseT-FD, but that LANCP would report the connection as being 10BaseT-HD. This would cause multitudes of carrier and frame check errors to be reported to the console by DECnet.
The content of this blog is based on my own personal experience and the following resources I managed to scour from around the 'net.
Lastly, here is the Advanced Networking Guide. I did not find this until I was checking all my links and references before publishing this article. It would have been a big help at the time.
3.24 asprintf for OpenVMS
For a long time the C Run-Time Library that ships with OpenVMS and the HP C compiler has been behind the curve. It usually takes a number of years to catch up with POSIX functionality, and in many cases it never catches up with GNU or *BSD extensions to the library. It took years to get an snprintf routine in the library, and then it was never backported to OpenVMS VAX or earlier versions of OpenVMS Alpha. Unfortunately [v]asprintf is no exception to this rule.
The short version of this story is that I was poking around in some OpenBSD code and came across asprintf. Curious as to what this *printf variant did I checked out the man page and was pleasantly surprised to find it was a version of sprintf that dynamically allocated its storage. So, after some looking around I discovered that it was not available on OpenVMS (not that I was particularly shocked) and decided to implement my own.
Of course, there are implementations around the 'net that I could probably hook up to sprintf, etc. However, there were a couple concerns that forced me to look elsewhere to solve this problem. The first was perfomance, and the second was that there may be issues with floating point representations and other output formats. It was important to me that this behaved just like the OpenVMS *printf routines. So, I started looking in the source listings.
After some quick poking around in the [ACRTL] facility the module that held the guts of the *printf engine was discovered as [ACRTL]F_DOPRINT.C. So, after reading through the listings (in [ACRTL.LIS]*_DOPRINT.*) I came up with the following (annotated) function definition:
** This global routine compiled only for narrow printf.
** Input Parameters:
** resbuf - address of result buffer
** fmtstr - address of format string
** arglist - address of argument list of items to be converted
** max_buflen - maximum length of result buffer
** fcb - Strictly speaking this is an FCB/WINDOW address for
** for a file, think fprintf() and friends (0 for SPRINTF).
** However, none of the code inside decc$$doprint() touches
** it. It is simply a pointer that is passed to the
** partial_printf routine. It could actually be used to
** provide context to the partial_printf routine.
** partial_printf - printf call back address. This routine is used to
** flush the buffer once it is filled. It returns 0
** for error and 1 for success.
** bytes_left - pointer to memory holding the number of bytes in the
** last partial filled buffer. For the VAX C RTL,
** this is actually returned in register R1.
** convert/formatted text - in buffer
** # of formatted bytes in left - bytes_left
** # of bytes transmitted - on success
** -1 - on failure
extern int decc$$doprint(char *resbuf, const char *fmtstr,
va_list arglist, int max_buflen, void *fcb,
int (*partial_printf)(void *fcb, char *buffer, int en),
... /* int *bytes_left */);
Next I had a look through the various libraries in SYS$LIBRARY: and found that this function was in fact publically available. So, I wrote my own implementation that supplied a partial_printf function that buffered the formatted output to a realloc'd buffer and returned it to the caller. The initial version of which only worked with G_Float format floating point numbers. After the additions of some #ifdef's based on the contents of [ACRTL]F_DORPINT.SRC the code now compiles correctly when compiling for all different floating point combinations. The code is shown below.
# ifdef VAX11C
# if CC$gfloat
# define decc$$doprint c$$gdoprint
# define decc$$doprint c$$ddoprint
# if __G_FLOAT
# define decc$$doprint decc$$gdoprint
# define decc$$doprint decc$$ddoprint
# if __X_FLOAT
# if __G_FLOAT
# define decc$$doprint decc$$gxdoprint
# elif __IEEE_FLOAT
# define decc$$doprint decc$$txdoprint
# define decc$$doprint decc$$dxdoprint
# if __G_FLOAT
# define decc$$doprint decc$$gdoprint
# elif __IEEE_FLOAT
# define decc$$doprint decc$$tdoprint
# define decc$$doprint decc$$ddoprint
Later on, after looking at some of the comments in the source listings it became clear that the *printf engine in use in the C Run-Time Library may have had its heritage in the the VAX C RTL. I checked this out and found that the VAX C routine was actually implemented in MACRO-32. However, the argument lists were almost the same, the only major difference being that the bytes_left parameter was actually returned in R1. So, after a couple quick changes, the end result was that this code will also work with versions of the VAX C RTL released after June 1986.
The only thing that really surprises me about all this is that it took until February, 2003 to get [v]snprintf into the CRTL and that [v]asprintf was never implemented, when it is not really a very hard problem to solve.
For each different combination of compilers and architectures, there is a different set of build instructions. Although these appear in the source module, they are also available below.
When using the VAX C Run-Time Library and compiler, it is important to use the following compilation command as compiling without the /OPTIMIZE qualifier will cause the code to ACCVIO.
$ CC/VAXC/OPTIMIZE=NOINLINE ASPRINTF.C
$ LINK ...,ASPRINTF,SYS$LIBRARY:VAXCRTL/LIBRARY -
There are no special qualifiers for the compilation under OpenVMS VAX, Alpha or I64 using the DEC/Compaq/HP C compiler. However, there are special linking qualifiers. For VAX it is necessary to use these qualifiers:
$ LINK ...,ASPRINTF,SYS$LIBRARY:DECC$CRTL/LIBRARY -
For Alpha it is necessary to use these qualifiers:
$ LINK ...,ASPRINTF,SYS$LIBRARY:DECC$CRTL/LIBRARY -
And finally, for I64 the following linker qualifiers are needed:
$ LINK ...,ASPRINTF,SYS$LIBRARY:SYS$LIBRARY:STARLET/LIBRARY -
The source code for [v]asprintf for OpenVMS can be downloaded here. If you are an OpenVMS engineer, feel free to take this code and add it to the C Run-Time Library. See the licensing comments below, it is all yours!
All the code mentioned in this blog post is released under an MIT-style license. There is no actual support, but I am generally happy to look into any discovered bugs. Beware, that given this code call into an undocumented and unsupported part of the CRTL, your particular issue may not be fixable. Only small, backward compatible changes have been made over the years, so the likelyhood of something going wrong and not being fixable is pretty small.
The following resources were instrumental in the production of this blog post.
3.26 This is the program that doesn't end...
Does anyone remember that song, The Song That Never Ends? I remember it from the end credits of Lamb Chop's Play Along when I was a kid. That was the first thing I thought of when I heard of a quine. Although, a quine is a whole lot easier to tolerate!
Of course, once I heard about these programs I started looking round for quines in my favourite languages. Of course there are plenty written in C. However, I was only able to come up with this one for PL/I. Unfortunately, according to the comments it will only compile using the OS PL/I V2.3.0 compiler with certain options set. Whatever the requirements, it wouldn't compile on OpenVMS. It also, erroneously, claims to be the smallest possible PL/I quine. So, in the spirit of innovation (at least that's what I'm going to call it), here is my contribution.
q:proc;dcl s char(94)init('q:proc;dcl s char(94)init();put list(substr(s,1,26)||byte(39)||s||byte(39)||substr(s,27));end;');put list(substr(s,1,26)||byte(39)||s||byte(39)||substr(s,27));end;
It will compile straight out of the box using the Kednos PL/I compiler on VAX and Alpha. It may compile using other PL/I compilers. The BYTE built-in function is an extension to the ANSI General Purpose Subset (ANSI X3.74-1987 (R1998)) so it may not work everywhere. Although the documentation for the IBM compilers indicates they support it.
The following is an expanded version to make it easier to understand what is going on. I should point out that this next sample is not a quine, it's just easier to read than the muddle of characters above.
dcl s char(94) init('q:proc;'
|| 'dcl s char(94) init(); '
|| 'put list(substr(s, 1, 26) || byte(39) || s || byte(39) || substr(s, 27));'
put list(substr(s, 1, 26) || byte(39) || s || byte(39) || substr(s, 27));
It works by storing most of the program as an initial value to the character string s. The only thing missing from this initial value is s itself and its surrounding quotes, which are inserted during execution with the concatenated input to PUT LIST.
Ahh, don't ya just love a good ol' fashioned waste of time?
For more mind-numbing quine madness have a look at these pages.
3.27 The VMS/L&T SIG Collection needs YOU!
I have announced (even tweeted) a call for submissions for the next VMS/L&T SIG Collection. That's right. It's a collection now, not a tape. The last release was all the way back in Spring (for those in the Northern Hemisphere) 2007. A lot has happened since then. The collection has a new maintainer (me) and the historical archives have moved to the new DECUSLIBrary.COMpendium.
The current plan is to have all submissions in by January 1st, 2010 with the "VMS/L&T SIG Collection 2009 H2" distribution ready for shipping by February 1st, 2010. I do intend to have the collections released half-yearly. Hopefully this release will make up for all those that have been missed.
If you are interested in contributing, please head on over to the SIG Collection page and make a submission. You don't even have to be the author. If you see something useful drop me a line and I'll chase up the author and handle the packaging requirements.
If you don't have any software to give and you still want to contribute, then maybe consider becoming a distribution point. The physical media is distributed by tree. So, the maintainer sends out a bunch of masters which are then copied and distributed to local (or at least semi-local) recipients. Although, in this day and age of broadband internet I imagine there won't be too many people looking for physical media.
To give you an idea of what to expect in the 2009 H2 release, the following packages have already been fetched for inclusion.
Please post more suggestions, whether it be below or at the SIG Collection page.
3.28 Jabbing the JIB...
Recently there was a post to comp.os.vms, JIB Jab. It asked of a way, from DCL, to determine the name of a process' job logical name table. I posted (and submitted to dcl.openvms.org) the following solution:
The procedure below accepts the process id to look up in P1. If no process id is specified then, like all process related queries, it defaults to the current process.
$ set noon
$ on warning then goto bail_out
$ on control_y then goto bail_out
$ say = "write sys$output"
$ p1 = f$edit(p1,"TRIM,UNCOMMENT,COLLAPSE,UNCOMMENT")
$ if (p1 .eqs. "") then p1 = f$getjpi("","PID")
$ pipe say "show process/id=''p1'" -
| analyze/system -
| search sys$pipe "JIB" -
| ( read sys$pipe result ; -
define/job/nolog pipe_result &result )
$ jibadr = f$element(5," ",f$edit(f$trnlnm("PIPE_RESULT"),"COMPRESS,TRIM"))
$ say "The job table for process ''p1' is LNM$JOB_''jibadr'"
$ exitt 1
This procedure works because a process' job logical name table is constructed from the string 'LNM$JOB_' and the 32-bit hexadecimal representation of the Job Information Block (JIB) address. The JIB of a process is accessible via the ANALYZE/SYSTEM utility and its SHOW PROCESS command. The output from SHOW PROCESS looks something like the following snippet:
Process index: 02CA Name: Tim at FTA40 Extended PID: 20404ECA
Process status: 020C0001 RES,PHDRES,HIBER,INTER
status2: 00000001 QUANTUM_RESCHED
PCB address 8227CEC0 JIB address 824817C0
PHD address 8462E000 Swapfile disk address 00000000
KTB vector address 8227D1B0 HWPCB address FFFFFFFF.8462E080
By searching for the string 'JIB' the address is extracted and the job logical name table is determined. The following test demonstrates the procedure works correctly.
The job table for process 2040500B is LNM$JOB_82483040
$ SHOW LOGICAL/JOB ASDF*
%S, no translation for logical name ASDF*
3.29 DOS file attributes on OpenVMS
A few years ago, four according to the source, I was asked if it was possible to set the DOS file attributes on an OpenVMS file that existed on an Advanced Server (aka Pathworks) share. I wasn't sure, but it took only a minute or so to check it out. Sure enough, from Windows Explorer it was easy enough done. A right-click and a couple left-clicks later and it was confirmed. Then came the real reason for the question, "Can we do it from VMS?" The story goes that, apparently a manufacturer required that a previous employer's software output data files to a Windows share. The manufacturer in question then used their own special tool to open the files and process the data. However, in order to be compliant with their requirements we needed to set the readonly attribute on the data files. Given the possibilities of ACLs and everything else available to Windows users it seemed crazy, but that wasn't the point. So, to solve this problem I came up with an OpenVMS version of the ATTRIB utility.
I started with looking at the output from a DIRECTORY/FULL of the file I had tested with and came up with something like the following output:
TEST.TXT;1 File ID: (142221,34,0)
Size: 0/0 Owner: [VMSUSER,TSNEDDON]
Created: 18-OCT-2009 22:45:19.94
Revised: 18-OCT-2009 22:45:19.94 (1)
Expires: <None specified>
Backup: <No backup recorded>
Effective: <None specified>
Recording: <None specified>
Accessed: <None specified>
Attributes: <None specified>
Modified: <None specified>
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 0, Extend: 0, Global buffer count: 0
No version limit
Record format: Variable length, maximum 0 bytes, longest 0 bytes
Record attributes: Carriage return carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RE, Owner:RWED, Group:RE, World:RE
Access Cntrl List: Suppressed 1 PATHWORKS ACE
Client attributes: Read-Only, System, Hidden
Total of 1 file, 0/0 blocks.
As you can see, the 'Client attributes' field lists the DOS file attributes. So I started looking at the source listings for the DIRECTORY command. In particular the source module [DIR]DISPLAY.B32 and the routines DIR$GET_CLIENT_ATTRIB and DIR$SHOW_CLIENT_ATTRIB. Together, these routines have knowledge of the Pathworks ACE that holds the DOS file attributes as they are responsible for the output of the field 'Client attributes' in the listing above.
Unfortunately I could not locate a routine to write these attributes and nothing public that could read them. So I searched through the source listings and eventually came up with the SDL module [F11X]PWACLDEF.SDL. This had more than enough information for me to work with. So, coupled with this module and the listings for DIRECTORY I wrote an OpenVMS ATTRIB utility. The complete source code and executable for all architectures are here.
Advanced Server stores the attributes as one of many Pathworks ACEs. These are suppressed (as shown above) in later version of the directory command, but a dump of the data can still be retrieved using either the SHOW ACL (obsolete) or SHOW SECURITY commands, like so:
freja_FTA9> SHOW SECURITY TEST.TXT
USER$:[TSNEDDON.SOURCE.ATTRIB.NEW]TEST.TXT;1 object of class FILE
Protection: (System: RWED, Owner: RWED, Group, World)
Access Control List:
The type of %X80 (128) is equal to PWRK$C_PATHWORKS_ACE indicating the ACE is a for Pathworks. The first word of the fourth longword (%X00000007) in the data contains the bitmask indicating which DOS attributes are active.
ATTRIB acts almost exactly like the DOS/Windows equivalent. It even supports the undocumented ',' operator. The HELP reported by the /? qualifier is shown below.
Displays or changes Pathworks file attributes.
ATTRIB [+R | -R] [+A | -A] [+S | -S] [+H | -H] [filespec[,...]]
+ Sets an attribute.
- Clears an attribue.
R Read-only file attribute.
A Archive file attribute.
S System file attribute.
H Hidden file attribute.
Specifies a file or files for attrib to process.
There is no support for the /S and /D qualifiers. These
can be performed using OpenVMS syntax.
, Specifying the comma immediately after the command
will cause all attributes to be cleared from all
files in the following filespec
However, I don't imagine that there will be too much use for this utility these days. The Advanced Server product has been placed into maintenance mode and will not be ported to the Integrity platform. Unfortunately the licensing arrangement between Microsoft, HP and AT&T fell apart and so the next step in keeping up with the ever evolving WIndows network on OpenVMS is the HP Common Internet File System (CIFS) product. It is an officially supported port of the open source Samba software.
The ATTRIB utility and complete (BLISS) source code can be downloaded below:
Fell free to use it to your heart's content. Maybe even post something below and let me know how you got on.
3.30 Serving Mercurial repositories from OpenVMS
I have just completed configuring the Mercurial repositories that hold the open source software served by Kednos. It is now possible to use the hg client to fetch and build MMK (The Halls Of ZK and MX will follow very soon) from http://hg.kednos.com. The following demonstrates a successful clone of the MMK repository.
hg clone http://hg.kednos.com/mmk
destination directory: mmk
requesting all changes
adding file changes
added 2 changesets with 44 changes to 44 files
updating working directory
44 files updated, 0 files merged, 0 files removed, 0 files unresolved
So, how did I get this all set up? Well that is the topic of this post. I have spent a bit of time tinkering around, reading the Mercurial documentation as well as some snippets of configuration files here and there. The first part was installing Mercurial. Mercurial has been ported by Jean-François Piéronne as part of his efforts in porting Python to OpenVMS. The Mercurial distribution is included on the Python logical disk and is not available as a PCSI (or other) kit. I chose to move the Mercurial distribution to our own physical disks and used the following commands to do so:
$ LD CONNECT <python-ld>
%I, Allocated device is $7$LDA5:
$ MOUNT/OVERRIDE=IDENTIFICATION LDA5:
$ BACKUP LDA5:[Mercurial...]*.* ddcu:[dir...]
As I said, this step is not necessary and entirely up to the user, I just like it this way. I then tweaked my login procedure to include the following commands to ensure I always have the relevant foreign symbols configured:
$ IF (F$SEARCH("PYTHON_ROOT:[VMS]SETUP.COM") .NES. "") THEN -
$ IF (F$SEARCH("MERCURIAL_ROOT:[VMS]SETUP.COM") .NES. "") THEN -
Once Mercurial had been installed and was running correctly, I began work on setting up the webpage that can be seen at http://hg.kednos.com. To serve the Mercurial repository we use (as we do for all our web facing needs) WASD. After some searching round on the 'net I found a couple snippets of WASD configuration file here and here. So, using these as a base I began to fiddle around with our setup.
Firstly, I needed somewhere to store the CGI procedure that handles all incoming connections as well as the repositories. I created the logical MERCURIAL_HOME. Although it seems to be the norm to use the abbreviation "hg" I found that to be troublesome in our setup as we also use Hunter Goatley's HGFTP for all our FTP needs. So, I created a directory and copied/created the relevant files and created the first repository like so:
$ CREATE/DIRECTORY ddcu:[MERCURIAL]
$ DEFINE/SYSTEM/EXECUTIVE_MODE/TRANSLATION_ATTRIBUTES=CONCEALED -
MERCURIAL_HOME ddcu:[MERCURIAL.] ! Don't forget to put this one in SYLOGICALS.COM too.
$ SET DEFAULT MERCURIAL_HOME:
$ COPY MERCURIAL_ROOT:[VMS]HGCGIPLUS.COM MERCURIAL_HOME:
$ CREATE LOGIN.COM
$ EXITT 1^Z
$ CREATE HGWEB.CONFIG
style = gitweb
mmk = mmk^Z
$ hg init mmk
$ CREATE [.mmk.^.hg]hgrc.
style = gitweb
description = MMK Make Utility
contact = Tim E. Sneddon <email@example.com>
name = mmk
allow_archive = gz zip bz2^Z
I ended up tweaking our version of hgcgiplus.com as we have a number of different version of OpenVMS running in our cluster. As a result of that we have three different PYRTE images. So, after a number of seemingly random HTTP 502 errors and some tinkering with WASD's WATCH facility I finally recalled what had been done for MoinMoin ( we used it before VWcms) and added in the pre-amble that picks up the correct image.
So, with all the files in the right place and a fresh repository waiting to be filled I began configuring WASD. First up was the configuration of the service. I added the following two lines to HTTPD$SERVICE.
That was no trouble, so now it's time to add the mapping rules to HTTPD$MAP, like so.
pass /static/* /mercurial_root/mercurial/templates/static/*
pass /mmk/static/* /mercurial_root/mercurial/templates/static/*
pass /*/-/* /ht_root/runtime/*/* cache=perm
pass /ht_root/runtime/* /ht_root/runtime/*
script+ * /mercurial_home/hgcgiplus* \
map=once ods=5 script=syntax=unix script=query=none \
script=as=mercurial throttle=1,,,30 \
This is a little more involved. The first couple lines ensure that all the static data references, like images and CSS for the template, are available. The next is a special rule added for the purpose of showing the Mercurial FastCGI procedure for this post. The next two lines ensures that the HTTP error message help pages all work correctly. Lastly, the line that allows everything to run.
The final rule ensures that everything not matched by the preceding pass rules is treated as a query string to be processed by Mercurial. All path names are translated to the POSIX/UNIX format. According to the WASD PyRTE documentation the final line causes a 4x improvement in speed.
The final step in the configuration is to set up the MERCURIAL user that the hgcgiplus.com script executes under. I simply copied the HTTP$NOBODY user that is created as part of the WASD installation. I used the following commands:
UAF> COPY HTTP$NOBODY MERCURIAL/OWNER="Mercurial Server"/UIC=[ggg,uuu]
With the MERCURIAL user created the protections can be set on the hgcgiplus.com procedure, like so:
$ SET SECURITY/ACL=((IDENTIFIER=WASD_HTTP_NOBODY,ACCESS=READ+EXECUTE), -
Now that everything is configured I told WASD to re-read its configuration and show me what I had created.
$ HTTPD == "$HT_EXE:HTTPD"
Voila! Now we have a working Mercurial repository. For the moment it is pull only. All modifications to the repository are handled locally. In the not to distant future I intend to get the push functionality up and running. When I do so, I'll post about that and how to configure the security necessary when providing the push facility.
Now, wouldn't it be wonderful if it had been that easy? Of course this is the shortened version with everything in the correct order and neatly written out. I spent a while playing around trying to get things working changing configurations, putting them back again and so on. Hopefully everything above will save somone the hassle of figuring it out for themselves also.
Did I miss anything? Post a comment below and let me know.
3.31 TECO has now entered the realm of the internet
So it's Friday and I feel like doing something a little fun although not really that useful. Yesterday I made a crack about getting TECO running in a Common Gateway Interface environment. Well, that's exactly what I decided to do. What follows is a set of basic instructions for getting your favourite editor/language up and running as a CGI handler on OpenVMS using the WASD web server.
First things first. The WASD_CONFIG_GLOBAL (or HTTPD$CONFIG in the pre-v10.0.0 releases) file needs a tweak to add support for the TECO interpreter. The snippet below shows how to tell WASD what to do when it encounters a file ending in .TEC.
It is sufficient to simply use the command
Seems easy, eh? Well, there are just a couple things that need to be mentioned. To fetch the CGI variables it is necessary to use the SYM built-in function. For example, to fetch the REQUEST_METHOD you would use:
This will leave the value of the REQUEST_METHOD in the file buffer register *.
For POST requests it is necessary to retrieve the POST data from the HTTP$INPUT stream. This can be opened and read just like a regular file using ER and the various Yank commands.
The last thing to remember is that TECO CGI procedures, just like regular MUNG'd procedures, have some simple requirements. The procedure must close the indirect input (like so,
So far there are no plans for a TECORTE to support persistent scripting, but who knows...? :-)
The following links may prove useful to anyone interested in writing some CGI TECO.
3.32 Improvements in VWcms
This Endless Software Solutions website, is maintained using the web content management system, VWcms (pronounced vee-dub). VWcms is a software packaged developed and maintained by Mark Daniel, the author of the WASD web server software. The software is of an excellent standard and Mark's support has always been great.
Recently I was putting together an upcoming blog entry that makes rather extensive use of the
After some fiddling around I discovered that the value for the CGI variable TEST would default to an empty string. So, even when the variable was not present in the URL it would always appear as if it had, just without a value. I corrected this and it is now possible to test for the presence of a CGI variable as decribed by the documentation.
While fiddling around in this code I decided that it might be a good idea to add support for some other operators to the "cgi=" parameter. In the official release it is only possible to specify "=" which actually performs a substring match rather than a test for equality. For example, the following piece of code will match when INDEX is both "1" and "11".
I have altered this behaviour for my patch. Although I have not noticed any erratic results there may be something lurking in the depths and your mileage may vary. However, it is now possible to compare the CGI variable value with a static string using the following operators:
*All comparisons are case insensitive.
Lastly, on Mark's suggestion I also applied a call to
I have constructed a patch based on the V0.94 release of VWcms.
This patch was created using the DCL command
$ EDIT/SUM HT_ROOT:[SRC.VWCMS]TEMPLATE.C/UPDATE=ddcu:[dir]TEMPLATE.DIF
3.33 A BLISS-ful family tree
A while back there were some posts on comp.os.vms regarding the history of Digital Equipment Corporation like, "Dave Cutler, Prism, DEC, Microsoft, etc." This particular post covered most of the DEC userland mythology. As usual though it is more than a little Lite on the facts. Thankfully, this post has nothing to do with PRISM, Dave Cutler or all the other conspiracy theories.
My gripe was with some comments regarding the history of BLISS. Those of you that know me will probably know that I have a sad, warped affection for the BLISS language. So, after responding to that particular post I decided to knock up a family tree/timeline of BLISS. It is much the same design as the UNIX family tree that can be found in "The Design and Implementation of the 4.4BSD Operating System".
The timeline does not show product version releases, just when the first version of that particular compiler became available. To find out what each variant did, what it targeted and where it ran, hover over the name with the mouse cursor.
This article has been composed of information gathered from the BLISS source code library history and the articles listed below.
The BLISS family tree has been developed with the help of the following websites and utilities:
3.34 Images from launch of Soyuz TMA-17
Yesterday, at 21:52 UTC I had the pleasure of watching the Soyuz TMA-17 spacecraft lauch from Baikonour Cosmodrome in Kazakhstan. This is the first time I have watched a Soyuz launch. It is also the second time I have watched a space craft launch via the streaming video feed offered by NASA TV. To cap it all off, later that night my eldest son and I watched the International Space Station fly by. To make the night complete we also caught sight of a shooting star.
It was quite an exciting day for me, as far as space travel goes.
While watching the launch I made a few screen grabs with my SnagIt "camera". They may not be the best quality (there are loads of high-resolution pictures around the 'net), but they are the only ones I could take (given my Olympus just doesn't have that kind of range).
Lastly, I spent some time looking around for Soyuz TMA-17 flight path information as I was hoping to see it also. I eventually came across a Java applet at the Russian Mission Control Centre that gave tracking information here. After some playing around I discovered that it could display other things too, so I setup my own tracking page that enables these here.
The following are a collection of links related to the Soyuz TMA-17 mission.
I would like to thank NASA and the Russian Federal Space Agency for their live coverage via NASA TV. Space enthusiasts are scattered all over the world and while all the noise is usually made in Florida or Baikonour, NASA TV ensures that the rest of the world can hear and see it when it happens.
3.35 Running Java Software on OpenVMS
Things have been pretty quiet on my blog for the last few months. Of course, this doesn't mean things have been quiet with me. On the contrary, the blog has been neglected because I have been quite busy. So, here is the first of many posts covering the things that have kept me away.
I don't know how many people reading this have ported Java applications to OpenVMS. However, if you haven't, it's generally a lot easier than you might think. Over time I have built up a bit of a "method" for getting Java software up and running quickly on OpenVMS. My biggest quibble with Java applications is that they often need many of the JAVA$* or DECC$* logicals to be configured. This often disrupts my environment, which I don't like. The only "POSIX-like" option I have configured in my login is
When porting any software the first thing I do is check out what has been done for other platforms. Normally if the package is available on both Windows and a UNIX derivative there is enough of a cross-section to see what sort of things might need changing. Most Java packages are shipped with both a shell script and a batch procedure for running the software. Initially I tried using a DCL procedure to do the same thing. It seemed like the logical step to take. Unfortunately this didn't work out.
While I don't specifically have a problem with DCL itself, it does have one rather large short-coming when attempting to write a procedure to start a Java application. The arguments,
I use the
$ mvn == "pipe def/u JAVA$DELETE_ALL_VERSIONS 1 ; " -
+ "def/u DECC$FILE_PERMISSION_UNIX 1 ; " -
+ "def/u JAVA$FORK_SUPPORT_CHDIR 1 ; " -
+ "def/u JAVA$FILENAME_CONTROLS 0 ; " -
+ "def/u DECC$ARGV_PARSE_STYLE ENABLE ; " -
+ "def/u DECC$EFS_CASE_PRESERVE ENABLE ; " -
+ "def/u DECC$EFS_CHARSET ENABLE ; " -
+ "java -classpath ""''classpath'"" ''f$trnlnm(""MAVEN_OPTS"")' "
+ "-Dclassworlds.conf=/m2_home/bin/m2.conf " -
+ "-Dmaven.home=/m2_home " -
+ """ org.codehaus.classworlds.Launcher"""
The example above is the method I used for some time to get Java software started on OpenVMS. This method has two drawbacks. The first is that it requires the executing process to have its
$ mvn == "spawn/nolog pipe set process/parse=extended ; " -
+ "@sys$manager:java$150_setup ; " -
+ "def JAVA$DELETE_ALL_VERSIONS 1 ; " -
+ "def DECC$FILE_PERMISSION_UNIX 1 ; " -
+ "def JAVA$FORK_SUPPORT_CHDIR 1 ; " -
+ "def JAVA$FILENAME_CONTROLS 0 ; " -
+ "def DECC$ARGV_PARSE_STYLE ENABLE ; " -
+ "def DECC$EFS_CASE_PRESERVE ENABLE ; " -
+ "def DECC$EFS_CHARSET ENABLE ; " -
+ "java -classpath ""''classpath'"" ''f$trnlnm(""MAVEN_OPTS"")' " -
+ "-Dclassworlds.conf=/m2_home/bin/m2.conf " -
+ "-Dmaven.home=/m2_home " -
An advantage of using the
$ cmd = "SPAWN/NOLOG PIPE SET PROCESS/PARSE_STYLE=EXTENDED ; " - 
+ "@<java-setup-procedure> ; " - 
+ "DEFINE <JAVA$-or-DECC$-logical> <value> ; " - 
+ "java <java-options-and-program>" 
It is important to remember that the above symbol works because of Extended DCL which is available on OpenVMS Alpha V7.3-2 and higher and all versions of OpenVMS I64. Prior to this DCL had the command line limit of 256 characters which would not allow this use of the PIPE command to work. Thankfully it is not really an issue as most Java software requires a recent version of OpenVMS and there is no Java for OpenVMS VAX. There are also no issues passing command line arguments as they tack right on to the end of the foreign command. Do also remember not to enclose the PIPE arguments in parentheses as any extra arguments will not be passed to the Java VM. The parentheses in a
Something else you might notice in the example is the definition of JAVA$FILENAME_CONTROLS. I don't like many of the JAVA$* logicals, but this must be my least favourite. From what I can tell, this logical hails from a time when Java under OpenVMS ran on ODS-2 disks and UNX-style file names often needed a bit of extra munging. Fortunately, those days are over. I find it best to set this logical to 0 and use the DECC$* logicals to define how file names are handled. Otherwise a lot of unexpected transformations can occur to the file name before it is actually used.
Maven on OpenVMS
At this time I am still working on the port. It is coming along exceptionally well. The following are a some of the more important tasks that have been acheived:
There is still much to be done, but a release of a largely working build system is imminent. To keep up to date with progress you can follow this blog (as you may already do) by signing up to the [RSS feed] as well as follow @tesneddon on Twitter for shorter, more frequent updates.
|As at: Thursday, 21 February 2019 17:27 Updated: Monday, 19 May 2014 11:41 (nearly 5 years ago)|