Notifications
Clear all

Comments on Detection and Rejection of Outliers

2 Posts
2 Users
0 Reactions
3 Views
(@john-nolton)
Posts: 563
Registered
Topic starter
 

This field of study of outliers is vast and rich and many books and papers have been written on the subject matter.
In Surveying/Geodesy one can say that there are 2 stages in the process. The 1st stage is very rarely ever mentioned on this site but before any other statistical test is done one should look at "Standards and Specifications for Geodetic Control Networks, 1984 (see NGS web site).
Most surveyors that post on this site are doing Traverse and that would be on page 3-3 and 3-4 of the above manual. Standards and Specifications will give you the minimal number of angles to turn, distances to measure etc. to attain a certain order of survey (1st, 2nd or 3rd order).
There is a rejection limit from the mean. After you meet that you then calculate the Standard deviation of the mean and it is not to exceed a listed value. All of this up to this point is known as Field Procedures. The next stage in the process is "Office Procedures"(stage #2) "A minimally constrained least squares adjustment will be checked for blunders by examining the normalized residuals .....". See page 3-4 and 3-5 for the rest of the procedures.

What does all this mean for the private surveyor. Its a guide for your control surveying that helps you plan the control work you are going to do.

Identifying Outliers:
There are many different models used in trying to identify outliers. We are only going to discuss one here because it was brought up by a poster recently.

Chauvenet's criterion (1863).

William Chauvenet "criterion is to have a probability band (as the poster has stated) centered on the mean of a normal distribution"
It is simple to apply but does have some weak points and that is " it is too sweeping"." It does not allow sufficiently for the possible presence of a large number of small accidental errors of the same sign". [see The Combination of Observations by David Brunt, 1923, page 131].
(Too Sweeping- you could reject observations that should not be rejected.)

Now NGS has mentioned Chauvent in Special Publication #237, 1952; Manual of Geodetic Astronomy. See page 136 and page 137.
This is what NGS has to say about his process; " Evidently, it would be unreasonable to apply this process to a very small number of observations, since these would give no adequate determination of the probable error".

For the surveyor after he meets the standards set forth in Standards and Specifications or any other standard (state, county, local, ALTA)
a Student t test would be next as used in NOAA Tech. Mem. NOS NGS-10 Use of Calibration BASE LINES, pages 10-13. On page 12 states
"the student's t distribution, which is useful in analyzing small sample tests." Also chi-square distribution is used in modern times.
For a very good book on the subject matter see; Adjustment Computations Spatial Data Analysis by Ghilani and Wolf.

JOHN NOLTON

 
Posted : 10/11/2015 10:23 am
(@geeoddmike)
Posts: 1556
Registered
 

The web link for the "Standard and Specs..." Document is: http://www.ngs.noaa.gov/FGCS/tech_pub/1984-stds-specs-geodetic-control-networks.htm#3.3

Also on the NGS site is Allen Pope's "statistics of residuals and detection of outliers" available here:
http://www.ngs.noaa.gov/PUBS_LIB/TRNOS65NGS1.pdf

 
Posted : 10/11/2015 6:45 pm