MSS Intro | Multiple MSSes | /public Data | Wrapper Scripts | Tips and Traps

Multiple MSSes

Over the recent life of GSD, data have been stored long-term on a number of systems:

  • 9-track tape (obsolete)
  • 8mm tape
  • UniTree/DSRS -- Data Storage and Retrieval System (the 'original' MSS)
  • FileServ/VolServ (the 'new' MSS)
  • StorNext (the 'newest' MSS)

A few salient facts about these systems are noted in the table below:

MSS Approximate start time Preferred user access method
9-track tape - -
8mm tape 11/94 operator request
UniTree/DSRS 05/96 DSRS commands
FileServ/VolServ 06/00 wrapper scripts:
/mss; /mss1
StorNext 11/02 wrapper scripts:
/mss; /mss2

9-track Tape

Early on, certain PROFS and GSD data were saved on 9-track tape stored in MDAL format. In the early 90's GSD migrated to the NIMBUS system wherein data were stored in more open and modern formats such as NetCDF and GRIB. As it was not cost-effective to reformat MDAL data into these other formats, the data and the media they were stored on were eventually obsolete and destroyed.

8mm Tape

With the advent of NIMBUS, data were stored onto 8mm tapes using the UNIX tar mechanism. An 8mm tape can hold ~5GB of data. While early on, nearly a week's worth of data could fit on a tape, in a few years a single day's worth of data required several tapes, and the system was becoming unwieldy.

These data are still available, albeit not very easily, as significant operator time is required to locate and retrieve data. These data have not been migrated to the next MSS.

UniTree/DSRS

It might be somewhat of a stretch in use of terminology to describe the previous systems as being 'mass store systems', at least by today's understanding of the term. By the mid 90's, however, GSD was able to acquire its first HSM-based MSS, although only the funds to purchase a 'starter system' were then available.

It became clear early on that the UniTree-based system suffered from several important limitations that needed to be addressed. The data usage pattern at GSD is rather atypical in the marketplace (a fact that affects us even in this modern era). Thus, DSRS was developed in-house to work around UniTree's limitations. Toward the end of the decade, though, UniTree/DSRS was clearly showing signs of advancing age.

The data stored on this MSS, which by now include generic user data as well as NIMBUS data, are still available via DSRS or through operator request. User data are currently being migrated to the next MSS, but NIMBUS data will probably not be migrated, although requests for the migration of certain data types will be considered.

FileServ/VolServ

The next MSS, based on FileServ/VolServ software, came as a major component of HPCS. It was hoped that at this stage the system would be robust enough to be presented to the users as is, but that proved not to be the case, and wrapper scripts again had to be developed, although they didn't have to be as complex as DSRS. But like all MSSes before it, problems arose that forced yet another upgrade.

The data stored on this MSS are still available via the wrapper scripts or through operator request.

StorNext

The most recent MSS is built on the StorNext suite of software, the next generation of FileServ/VolServ software. Although unfortunately slow in coming, the wrapper scripts were augmented to refer to the StorNext archive as well as the FileServ/VolServ one.

The End of the Road?

We've gone through a painful series of upgrades. Not being gluttons for perpetual punishment, and with the improvements in technology, can we now say we've reached the end of our tribulations? Well, no and yes. As this brief history has indicated, GSD's data storage needs continually push the envelope of MSS technology, at least that which we can afford. It seems very likely that our growth will eventually exceed what our current system can provide. It may well be that we'll need to move to a different MSS vendor, media, or technology.

However, from a user perspective, we're hoping that the trauma that accompanies changes in the MSSes will be significantly mitigated. The MSS wrapper scripts are approaching a level of completeness, robustness, and transparency where adding new MSSes can be accomplished in a straight-forward manner.