DATA PROCESSING IN THE NATIONAL GEODETIC SURVEY
by Maralyn Vorhauer

Since its inception in 1807, the National Geodetic Survey (NGS) and its predecessors, have always strived to produce the most accurate, highest quality geodetic data for its customers. Initially, and for well over 150 years, this meant painstaking, meticulous manual computation of coordinates and other geodetic quantities. It also meant that the quantity of data was limited and the time it took to produce results lengthy, since this oldest scientific agency in the U.S. Government was never large nor generously funded.

The past 10 years have seen what can only be described as an explosion in technology and a sharp increase in the amount of data flowing into the agency both for processing and dissemination. NGS has had to work ever more efficiently and take extensive advantage of new technology to even begin to meet these demands on its resources. This has never, however, lessened the commitment of NGS employees to produce products which users can depend on to meet their needs whether for land surveys, airplane navigation or Geographic Information Systems (GIS) applications. This commitment to excellence has required the development of extensive computer software and a set of procedures for use in-house and, increasingly, outside NGS, to assure the accuracy and reliability of its published data. It has also been necessary to limit the types of data which we can include in our databases and the formats which we can process and load into these databases.

This article deals with the general format, computations and checking of control data which is submitted for inclusion in the National Spatial Reference System as defined by the NGS data base (from which it is distributed to the public in one form or another). The data may come from NGS field parties, NGS contractors, the numerous federal, state, and county agencies or the private sector, all of whom submit field observations to NGS headquarters. But for all, the procedures and products are the same and rather stringently adhered to. This assures the consistently high quality of the data as it is published regardless of the source. All software programs mentioned in this article are available from NGS, either by calling the Information Center at 301-713-3242 or from the NGS home page on the World Wide Web - http://www.ngs.noaa.gov. The established standard for the submittal of data to NGS is found in the document entitled "Input Formats and Specifications of the National Geodetic Survey Data Base". This document, available from the Information Center, also details how and to whom to submit data or call for specific information.

This extensive document, usually referred to as the "Bluebook" because it was originally distributed in blue binders, defines every piece of information required for submittal of data to NGS and the format of this information. It defines formats for classically observed data (usually called the t-file), Global Positioning System (GPS) data (b-files and g-files), and descriptive data (d-files). Programs have been developed which expedite the production of the digital files in this format. Most manufacturers of GPS equipment have available software to produce the b-files and g-files necessary to meet NGS format requirements. NGS programs, MTEN- used to produce t-files- and DDPROC- used to produce descriptive data files- are available. Programs have also been written at NGS to check that the range and format of the data in the files are correct. Submittors may or may not use the programs to produce the files but all files must be checked using COMPGB (GPS data checking), CHKOBS (used to check classical t-files), OBSCHK (used to check b-files and g-files and to compare the information in each), CHKDESC (used to check d-files), and OBSDES (used to compare the b-file and d-file) and outputs of these programs are required in the data submittal package. Usually if the available software is used to produce the bluebook files, the programs will give few, if any, errors and expedite the submittal process. These programs have been written not just to check the bluebook files but to work in conjunction with the data base update software so that these routines will be free from time consuming errors when the project is completed and loaded.

Of course, the observational data are only the means to the end products. Geodetic coordinates, including latitude, longitude, orthometric height and where appropriate, ellipsoidal height, which result from these observations are the quantities which become the products used by consumers. Computation of these values formerly were handled by office staff with extensive geodetic backgrounds and years of experience. Today, facilitated by computers, sophisticated software, and ever increasing educational backgrounds, almost everyone involved with the collection of geodetic data have the means and resources to compute highly accurate and reliable values for geodetic quantities. These quantities are usually computed using least squares statistical methods-called adjustments'.

Many least squares adjustment software programs exist which can be used to accurately compute the coordinates. However, files of data submitted to NGS must have coordinates which are the output of the software program ADJUST. The question often arises as to why NGS requires that its own adjustment software, ADJUST, be used when results from others are quite similar or identical. The answer is that, to the best of our knowledge, ADJUST is the only program that uses the b-files and g-files. Given ever dwindling personnel resources in NGS, files in other formats cannot be accepted and the quality checking provided by using ADJUST with these files cannot be provided. Prudence dictates that if NGS is to provide the seal of approval' implied by publishing coordinates under its auspices, that we be confident that these coordinates are correct and, further, that supporting observations are available if future changes to these coordinates are required. ADJUST outputs can be easily reviewed for correct values and procedures. Submittal of these adjustments has been required for data submittal for a couple of years and has been the major reason that NGS can continue to provide the data base storage, distribution and quality assurance of its published data services currently available free of charge.

Before giving some details on NGS procedures for adjustments, especially GPS adjustments, it is probably important to outline what type of data is routinely accepted by NGS. None of the above discussion applies to leveling data. Although in principle, the same rules' apply and a bluebook is available defining the formats for submitting leveling data, portable software which can be used for level data digitizing, format checking (for conformance to the leveling formats and specifications similar to the horizontal data bluebook'), and adjustments has not been developed and is presently confined to NGS in-house machines. Users wishing more information on submitting leveled observations should contact NGS for more specific information. All horizontal data submitted must be observed to current first-order standards. Although not limited to GPS data, in fact no first-order data observed by classical methods have been collected by NGS for many years (since 1985, in fact) nor has any been submitted. This effectively results in all data submitted being GPS data. Therefore the following discussion will apply to GPS data; while the steps used also apply to classically observed data, the order of processing would be different or modified.

The first step in the adjustment process is to run a minimally constrained (free) adjustment holding fixed one position and one height. The ADJUST input and output is controlled by a file (A-file) which includes the parameters for the adjustment and the specific constraints to be used (CC records). If geoid heights have not been added to the b-file, then the height held should be an ellipsoidal height; if they have been added, it should be an orthometric height. ADJUST software is designed to expect orthometric heights to be provided in the A-file, and so in order for the program to produce valid results, the height used must match the presence or absence of geoid heights in the b-file. Without geoid heights, the ellipsoidal height equals the orthometric height and will be treated correctly in the software. The adjustment is run in three dimensions (DD3 record), without the option of scaling the results (MM record). It is usually helpful to include the options to print the full input b-file and g-file (PP record in the A-file) as a record of what information was included in the adjustment. Accuracies may be included (QQ records) to help determine that the internal consistency of the project has met the intended accuracy. At this point in the adjustment process, external (or network) accuracies are meaningless. (The internal accuracy of a line is the first accuracy given in the output; the external accuracy is the second.) The output residuals can be analyzed for blunders, redundant vectors that differ by more than about 5 centimeters (possibly pointing to the need for additional observations because of poor centering on the mark or atmospheric interference), and height residuals that are above about 8-9 centimeters ( possibly indicating a problem with antenna heights). Output positions can be compared with the published values to determine if the control is adequate. Most reduction software produces standard errors that are overly optimistic to assign to the vector components so a high variance is not always a good indicator of problems with the survey. High residuals throughout the free adjustment should be a cause for concern and a look at the reductions and possibly the field observations.

After a satisfactory free adjustment is run, program MODGEE is run to scale the standard errors assigned to the vectors by the standard deviation of the free adjustment (given near the end of the ADJUST output, before the listing of the adjusted positions). This results in all first-order projects having essentially equivalent standard error assignments when they may be combined in the future. Higher order projects are not scaled. A new free adjustment is usually then run to verify that the resulting variance is very close to 1.00 indicating proper scaling.

Two adjustments are run on the b-file to determine the final coordinates. What NGS usually refers to as a horizontally' constrained adjustment is run to determine latitude, longitude, and ellipsoidal height. A vertically' constrained adjustment is run to determine orthometric heights. This dual adjustment procedure has been the practice since NGS began to process GPS observations. It is necessary to ensure that both the orthometric and ellipsoidal heights are computed correctly. They are independent quantities and separate adjustments provide the mechanism to compute each one as such.

The horizontally constrained adjustment is run holding fixed NGS published North American Datum of 1983 (NAD83) positions. Marks which have published positions less than first order or positions not determined by GPS are usually allowed to float - such constraints are seldom as accurate as the GPS positions and observations and exert undue stress on the network. If shifts are found to be very small (less than 4-5 centimeters) for these points and the accuracies produced when holding them fixed meet project specifications, then the adjustor has the option of holding fixed the position simply to reduce the number of unnecessary positional changes at a point over time. This, however, is rarely the case for such marks. On the other hand, points which are B-order and above and first-order positions determined from these high accuracy positions can almost always be held fixed and produce the needed accuracies. These determinations are made by examining residuals in the adjustment, the external accuracies resulting from a free adjustment using with the final output positions from the horizontally constrained adjustment, and shifts of positions which are not held. Several adjustment runs may be necessary with various constraints before the adjustor is satisfied with the results. Not to be overlooked are the ellipsoidal heights. Their fit' with previous heights are not reflected in the accuracy listing and residuals must be examined manually to determine whether previously published values should be held. The older the HARN or the previous first-order survey, the more likely it is that the heights will result in higher (i.e. decimeter or more) residuals. Until the completion of the effort to reobserve the HARNs throughout the country using stringent methods developed to upgrade the height component of the vectors, this situation is likely to occur. The adjustor must determine the importance of the ellipsoidal heights to the survey to make a decision whether to extensively or selectively readjust these heights or leave them as currently published.

The orthometric height adjustment is run and analyzed similarly. The software program, GEOID96, should be used to add geoid heights to the bluebook. Usually only previously published, leveled elevations are held fixed. One horizontal position (latitude and longitude) is held fixed and the 3-D option used for the adjustment. If the adjustor or client is interested in using orthometric heights previously determined by GPS and since most GPS determined orthometric heights are only published to the nearest decimeter, NGS can be contacted to obtain the full two decimal place value stored in the database. Seldom are previously determined trigonometric values held. Studies are underway and criteria being formulated for the conditions under which a GPS survey can be expected to yield GPS derived orthometric elevations at the centimeter level. Until then, if such accuracies are desired for the orthometric heights, contact NGS or the local state NGS advisor for help in planning the survey to meet such specifications. Currently the decision regarding whether the heights meet the more stringent criteria is made on a case by case basis. The analysis of this vertical adjustment, however, is similar to the horizontal adjustment analysis, a review of the control, the residuals and the shift in elevation when it is not held fixed. The extent of this analysis often depends on the specifications for the survey but as the heights become more and more reliable, it can be expected that this phase of the project will take on more importance.

Since two files have now been produced, one with the final adjusted orthometric heights and one with the final latitudes, longitudes and ellipsoidal heights, and only one file can be used to update the data base, the files must be combined. Program ELEVUP provides a very easy means of accomplishing this task. Note that the output b-file from the vertical adjustment is used with the positions and ellipsoidal heights from the horizontal adjustment output b-file replacing those values. No format checking is performed and no other quantities are replaced or combined.

Final values for the accuracies can now be computed using this file. A minimally constrained adjustment is executed including the QQ records in the A-file. Once again as noted above, and, since geoid heights are present in the final file, an orthometric height must be held fixed. Holding a substantially different height (as would be the case with an ellipsoidal height) results in significantly lower accuracies and possibly incorrect conclusions about the results of the survey. The output of this adjustment is also used as input to the program ELLACC which gives an estimate of the order and class of the ellipsoidal height. This information must be entered in the final b-file; generally, one order and class is given to the entire survey based on which one most of the lines fall into. (This information is entered in the *86* record in the bfile.)

The final step, and probably one which is best implemented as the project is processed and analyzed, is an extensive writeup in the field report of the results of the adjustments, the analyses, and the conclusions reached. List published positions or heights not held fixed and why, error messages remaining in the format checking software outputs, unusual situations (an example might be an underground mark used for observations or a station which appears from the results to have been moved but no other evidence exists), vertical control which came from other than NGS sources (allowed but must be documented), etc. Often these details provide valuable documentation when the project is initially reviewed by NGS as well as in the years to come when questions arise about specific situations.

While this article could only deal with the general outline of the procedures used, the employees at NGS are more than anxious to provide answers to specific questions. NGS is committed to accepting the processing and decisions made by the submitter when satisfactory results are achieved and documented. In fact, it has been our experience that if these steps are followed with attention to detail, little additional effort is needed when the project is received by NGS to review it and update the database. The user can also be assured that the results will not change and can be used immediately and NGS can be assured that no better results could be achieved by in-house processing. Following the basic outline above provides both NGS and the data user with the quality of data they want and have come to expect from NGS.