Over 40+ years, International Cooperators (ICs) have played a critical role in the mission’s success, acquiring Landsat imagery at numerous International Ground Stations (IGS). Until recently, most of the data held internationally were unique, relative to each station’s area of coverage, and were not duplicated in the USGS archive.
In 2010, the Landsat Global Archive Consolidation (LGAC) initiative began, with a goal to consolidate the Landsat data archives of all international ground stations, thus making the data more accessible to the global Landsat community and significantly increasing the frequency of observations over a given area of interest to improve scientific uses such as change detection and analysis. This initiative is possible only with the tremendous support given by the Landsat International Cooperators.
One critical component to successfully fulfilling the LGAC initiative was the ability to recover data from old, obsolete media for which newer media sources were not available, including Landsat data contained on thousands of High Density Digital Tapes, also known as HDDTs. Data recovery from HDDTs is a painstaking process that is dependent upon functional aged equipment and diligent operator support. In an effort to highlight the tremendous work performed by the data recovery technicians and to shed some light on the complexities of the data recovery process, this presentation will summarize the HDDT Landsat data recovery process. First, boxes and crates containing HDDTs arrive at EROS from ICs located around the world. HDDTs are not small or light, with each tape weighing approximately 14 lbs.
The tape contained on a standard-sized reel is about 2 miles in length and typically contains one satellite downlink. HDDTs labeled as Landsat Thematic Mapper (TM) are sorted out and processed first. HDDTs labeled as Multispectral Scanner (MSS) or tapes that show signs of mold are set aside initially. MSS tapes are processed at a later time by changing a setting in the Ingest software Moldy tapes, on the other hand, need to be evaluated, run through cleaners, and then can usually proceed through the data recovery process.
HDDT Data Recovery Process
Once the sorting has been completed, operators retrieve about 20 of the TM HDDTs and cart them to the Digital Camera Lab, which consists of a staging area for the HDDTs, four ovens for baking, BOW cleaners for cleaning tapes, drives used for exercising tapes, ingest drives used to read in the information, and a computer station displays information being captured. Some of the equipment used in information recovery process was graciously donated by the ICs and has been vital to the success of the project. There are four primary steps in the HDDT information recovery process: baking, cleaning, exercising, and reading. HDDTs from different countries require different variations of the information recovery process or recipe, and in some cases, multiple recipes for the same International Ground Station have been necessary. Identifying the proper recipe can be challenging and requires patience and persistence from the operators.
The first step in the HDDT data recovery process is baking. Baking is necessary because of two different but related scenarios. First, the tape is sticking to the rollers and guides on the drive, which causes damaged or broken tape, and second, there is a marginal or bad quality read of data from the tape. Each of the four ovens in the Digital Camera Lab has the same specifications. The ovens are 120 volts, 60 Hz, and 4 Amps.
The ovens consist of an enclosed unit with a door and contain a low-level heat source and a small fan for air circulation. The ovens are big enough to handle 10 HDDTs at a time and have adjustable settings for temperature and automatic shut off. The optimal number of tapes that can be loaded at once varies by International Ground Station based on trial and error. Once the HDDTs are loaded in the oven, the door is closed and the oven is turned on. The HDDTs bake in the oven for 24 hours at 130⁰ Fahrenheit (55⁰ Celsius), after which the operators either let them cool in the oven or set them on a wire rack to cool.
It is important that the tapes are read soon after the baking process as the tape tends to quickly revert back to the condition it was in before the bake.
The second step in the HDDT data recovery process is cleaning. Cleaning is necessary to remove dirt from the tape and is performed by using a BOW cleaner. The BOW cleaner contains a cloth ribbon that runs against the tape to help remove dirt off of the tape. Operators load the tape onto the BOW cleaner and run it through twice, with back-to-back cleanings.
The ribbon turns really slowly and lasts about two weeks before needing to be replaced. If a tape has not been baked, the ribbon will contain much more black dirt or soot on it when cleaning.
The third step in the HDDT data recovery process is exercising. To begin with, operators have to first clean the drive. They use alcohol, Q-tips, and wipes to clean the tape path.
Exercising the HDDT packs the tape better on the reel, which then leads to a better read? The optimal forward and rewind speeds can vary by International Ground Station.
The fourth and final step in the HDDT data recovery process is reading. Similar to the exercising step, the tape path is cleaned before reading a new tape. Next, an HDDT is loaded on the Ingest drive and the drive is set to run at the speed at which the data was recorded, also known as the downlink frequency.
The HDDT cannot be read at a faster or slower speed. As the tape is being read, there are different things operators need to watch for, including light activity. When the Status and Control display shows only a couple flickers of lights, it is an indication of a good read If the Status and Control display shows an abundance of flickering lights or if solid lights appear, it is an indication of a bad read. The read head contains 28 grooves and each groove on the head corresponds to a channel.
If a light is flashing for one of the channels, it is that section of the tape that the corresponding groove is reading that may have dirt or soot on it. At the same time, the operator must watch the computer monitors that are displaying the data capture. As the data is read off the tape, the image scrolls on the left monitor along with a data range. The right monitor displays statistics relating to the image quality If the data range rises and the data quality drops, that is an indication of a bad read.
When the read is not going well, the operator will stop the tape, rewind it 100 to 200 feet to when the read was going well, clean the tape path, and then retry reading the tape. Rewinding the tape and re-cleaning the equipment can make a huge difference in the read quality of the data. This can be seen very clearly in the before-and-after image screenshots. It is rare that tapes from some International Ground Stations can be read all the way through with no stops, and some tapes need to be exercised multiple times to try to repack the tape more smoothly. Particularly problematic tapes are set aside and returned to at a later time.
These tapes will need to go through the entire data recovery process again when they are reattempted. As tapes are completed, they are placed in a finished pile and later sent to the warehouse for disposal. Recovering data from HDDTs is a challenging process. Operators have to make frequent adjustments to recipes, equipment must be diligently cared for, and multiple read attempts are frequently required. While the process is labor intensive and requires tremendous patience and persistence, success results in the addition of new. Landsat data to the USGS EROS archive, in some cases, unlocking new regions of the world to the user community, and in all cases increasing the frequency of observations globally.