Interlibrary Loan Update

To:               New Hampshire Library Community

From:          Michael York

Subject        Interlibrary Loan

Date:           March 5, 2018

I believe this incident report about the failure of the equipment and software associated with the URSA  interlibrary module will answer some of the questions that librarians have regarding  the difficulty in replacing the system as quickly as we would have liked. NHAIS  Services has put this together we have been waiting for SirsiDynix to do the post-mortem  on the system  which they did last week and to review everything still in place for viability which they have also done.

Last week the SirsiDynix team which has been working on trying to revive our URSA ILL system had to admit defeat. The old ILL system cannot be recovered.

Our interlibrary loan program (URSA -- Universal Resource Sharing Application from SirsiDynix) resided on a SunBlade 2000 server at a Concord location managed by the NH Dept. of Information Technology.  The server had two hard drives containing the operating system and ILL program and these disks mirrored each other.  Transaction data was written to two separate hard drives on an attached disk array.  Transactions were also backed up to an 8mm tape. Over the years there have been network and power issues that have caused ILL outages but the only hardware issue we've had before was early in 2012 when the motherboard and central processing unit were replaced on the ILL server.  No data was lost that time as the hard drives were unaffected.

On the afternoon of December 6, 2017, the NHAIS Help Desk received a call from a library unable to connect to the ILL server.  We were unable to communicate with the server remotely so a trip to the server location was required.  The server was able to power up but couldn't boot.  Subsequent diagnostics found both of the server's hard drives had failed.  These were replaced with new drives on December 12.  NHDoIT assisted in formatting the drives and installing the operating system.  It took a few days to track down a patch which would allow SirsiDynix technicians to gain full remote access to the server but the patch was finally installed on December 20.  This was when SD first got a look at what was left.  No full copy of the ILL program and associated components was found on the remaining disks and transaction backup tapes so it was determined that a fresh installation of URSA was in order.

URSA 2.6 was first released in 2000 and the New Hampshire installation was the last one supported by SD so installing the program from scratch didn't prove to be straightforward.  One of the key components of the ILL program, the database engine Informix configured to work with URSA, could not be extracted from the old drives and is no longer available. (Informix was bought by IBM in multiple pieces in the early 2000s and the product lines changed considerably in the years since this occurred.) The final step in the salvage process was a full day of working with a technician from SD to examine the two failed hard drives to see if this software could be recovered from one of them.  The judgment at the end of the day was that the drives were unreadable and there was no way to rebuild our interlibrary loan program as it was.

This past week we also worked with SirsiDynix to complete a full review of our backup procedures regarding the current installation of the NHU-PAC which is on a server located at the NHDoIT server farm. The underlying software (SirsiDynix’s Horizon and the Sybase database engine) for that is currently available and supported software and our backup routines were determined to be appropriate to allow us to recover from a major hardware or software failure should one occur. We have no reason to expect such a failure, but it seemed prudent to double-check at this point.

Moving forward  we continue to evaluate other products to provide automated ILL to NHAIS libraries while we work through the procurement process for the next generation of NHU-PAC. We hope to have a new ILL solution ready to roll out as soon as both technical and contractual issues can be resolved.

Michael York

State Librarian

New Hampshire State Library

New Hampshire Department of Natural and Cultural Resources

20 Park Street

Concord, New Hampshire 03301

603 271 2397

Cell 603 497 7432

Fax 603 271 6826

michael.york@dcr.nh.gov