I’ve been studying the calculations for 95% confidence ellipses around point positions, as produced by a least squares program, and I think Wolf and Ghilani have it wrong in their books.
The effect of their formula is to vastly overstate the size of the ellipses when there are few redundant measurements. It is good to encourage surveyors to make more redundant measurements, but I don’t like incorrect theory.
They have a section discussing the factor to get from standard deviation to 95% confidence ellipses, using critical values from the F-distribution, depending on the number of degrees of freedom (redundant measurements).
Reference: Wolf & Ghilani, Elementary Surveying, 11th edition, page 433, section 15.12 “Other Measures of Precision for Horizontal Stations.” The discussion in their Adjustment Computations book as found in a downloadeable copy was very similar.
I think the proper calculation uses the Rayleigh distribution, regardless of the amount of redundancy, and radius or ellipse axis for 95% confidence is at 2.448 times the post-processing standard deviation of that variable. That is the same number you get by their formula from the F value for infinite degrees of freedom (because any reasonable distribution averaged over many trials converges to normal).
I would argue that all least squares calculations are based on the assumption that the measurement errors are gaussian-normal distributed, the dependencies in a small neighborhood of the LS solution are linear, and any linear combination of normal variables remains normal. The Rayleigh tail is the proper distribution for confidence in a bivariate normal, and does not depend on the number of samples.
I noticed this puzzle because of the difference between the W&G discussion and what I had done with similar statistics in another field. I then set up a Monte Carlo simulation of a million random trials that confirms that the Rayleigh calculation does give the threshold which 95% of the trials do not exceed. I also ran Star*Net in demo mode and it agrees with me, not W&G.
Questions:
Has this been argued over before?
Does anyone here have enough understanding of the theory to either agree or argue with me? Is there some assumption or interpretation I shouldn’t be making?
What factor does your commercial software use to get from post-processing standard deviation to 95% confidence ellipses? Is it dependent on the number of redundant measurement observations?
Does anybody have a later book where Wolf & Ghilani have changed their discussion, or a book that disagrees with them?
-------------
Simple example problem (chosen because LS solution is a weighted average and thus easy to use in the million-trial simulations):
Assume perfectly located stations as follows:
1 (0,0)
2 (–1000,0)
3 (-1000,1000)
4 (0, -1000)
5 (+1000, -1000)
Assume imperfect measurements from various stations to station 1. The measurement values don’t affect the ratio of 95% ellipse to post-processing standard deviation, but the standard errors do determine the shape of the ellipse and have been chosen to maintain something close to a circular “ellipse” for easier checking.
The Star*Net input is:
# Study statistics of a least squares solution.
C 1 0 0 * * # initial guess
C 2 -1000 0 ! ! # fixed points
C 3 -1000 -1000 ! !
C 4 0 -1000 ! !
C 5 1000 -1000 ! !
D 1-2 998 1 # measurements
D 1-3 1416 1.4
D 1-4 1001 1
D 1-5 1413 1.4
#result is std dev 0.8138 each axis and semi-major 95%=1.9921
# ratio = 2.448
For the 1-redundancy case
D 1-2 998 1 # measurements, 1-redundancy case
#D 1-3 not measured
D 1-4 1001 1
D 1-5 1413 4
# ellipse not quite circular
This should be a very interesting topic for those of us who use least squares survey adjustment software to estimate positional uncertainty. It is to me. It doesn't at first impression make any sense to me that the random errors in redundant measurements would have other than the same Gaussian distribution characteristic of the process by which they were made.
I'm glad to read that Star*Net gives the correct answers.
BTW, here a diagram illustrating the method that Bill used to verify that Star*Net gives a consistent ratio of axes of Standard Error Ellipse and 95%-Confidence Error Ellipse. There are two pairs of distances taken in directions perpendicular to each other, either pair of which would be sufficient to generate an estimate of the position of Point 1 and its uncertainty. What Bill did was to verify that the ratio remained the same whether one pair or both pairs were used.
> Does anyone here have enough understanding of the theory to either agree or argue with me? Is there some assumption or interpretation I shouldn’t be making?
I do not know enough to argue, although I try to follow along. It interests me.
I met Professors Wolf and Ghilani once, for a two day seminar. Both were very personable and approachable. Professor Wolf isn't answering his mail anymore, but I'm pretty sure that Professor Ghilani would be tickled to hear from you. Drop him a line.
BTW, if you do get a letter from Professor Wolf, the stamp will be worth saving.
>
In effect, the argument he's making, if I understand him, is that the 95%-confidence error ellipse of Point 1 as determined from four independent meansurements of of 2-1 and 4-1, each with a standard error = s should be the same as the 95% confidence error ellipse of Point 1 from one measurement of 2-1 and of 4-1, each with a standard error of 0.5 x s, which only seems correct.
That is, it isn't clear at all why four independent measurements giving a mean with a error of s/2 would be statistically different than one measurement with a standard error of s/2.
I Disagree Bill
For a normal distribution of errors, considerably more observations at each point, not redundant observations, need to be made.
Without normal distribution, there is the probability that several gross errors could cancel out, lending one to believe that error ellipses are small.
Redundant observations are necessary to prove that the error ellipses are small thus allowing us to live with the non-normal error distribution.
In a traverse with several redundancies, the LS solution can often have 2-3 points with adjustments of 2-3 times the typical.
Least squares is too often used to spread error around thinly like gold gilt because of limited redundancies. A strongly redundant LS solution puts the majority of the errors in those few points where there is the highest probability of their occurrence.
Paul in PA
Okay, I'll bite
> For a normal distribution of errors, considerably more observations at each point, not redundant observations, need to be made.
Okay, so if the given is that the errors in observations are random errors from a well-characterized Gaussian distribution (i.e. the standard error is known with very little uncertainty), why do you feel the need to make lots of observations?
> Redundant observations are necessary to prove that the error ellipses are small thus allowing us to live with the non-normal error distribution.
Actually, if the random errors in observations are well characterized, the uncertainties in quantities derived from them are as well. Are you simply disputing the idea that a measurement process can have well-characterized uncertainties?
We Only Assume The Errors Are Random, They Are Not
We can have several random errors, all to one direction on a set of observations. We need many more observations to get those random errors on either side of the correct answer.
For a Gaussian distribution of errors we need quite a few repeat observations at each point.
Paul in PA
Properties of Random Errors
> We can have several random errors, all to one direction on a set of observations. We need many more observations to get those random errors on either side of the correct answer.
Well, "random" means, among other things, that the sign of the error can be either negative or positive. So, one property of random errors is that it is likely that you can in fact have a short sequence of measurements containing random errors of the same sign. What's the problem since the likelihood is that the magnitude of the errors of that same group will be small?
I trust (translation: hope) you aren't under the impression that it is possible to make any measurement of a physical quantity that does not contain random errors.
Yes, But Almost All Surveying Is Of Short Sequences
Wolf and Ghilani understand that, while Bill thinks they are exaggerating errors.
While the errors are small, they are also biased.
Paul in PA.
Very Little Surveying Is Of Short Sequences
> While the errors are small, they are also biased.
No, they would not be if they are random errors. That is a given. If the errors aren't random (i.e. are systematic), then there is some flaw in the technique or equipment. The whole point of professional surveying is to be able to make measurements free of significant systematic errors.
So Kent, Explain Your Survey Sequence At A Traverse Point
"So, one property of random errors is that it is likely that you can in fact have a short sequence of measurements containing random errors of the same sign. What's the problem since the likelihood is that the magnitude of the errors of that same group will be small?"
The above introduces the bias.
Paul in PA
Random means + or -
> "So, one property of random errors is that it is likely that you can in fact have a short sequence of measurements containing random errors of the same sign. What's the problem since the likelihood is that the magnitude of the errors of that same group will be small?"
>
> The above introduces the bias.
No, it doesn't. This can't be that difficult. If you flip a coin twenty times, you may very well have a run of heads or tails. The results won't be perfectly alternating heads or tails. So it is with the signs of the errors in random measurements.
A biased set of measurements is one that contains errors that can't be attributed to random processes. If you flip a coin and get heads ten times in a row, for example, the odds are very much against your coin being a fair coin.
I Disagree Bill
For a normal distribution of errors, considerably more observations at each point, not redundant observations, need to be made.
:good: :good: :good:
Random means + or -
> If you flip a coin and get heads ten times in a row, for example, the odds are very much against your coin being a fair coin.
What?
Please explain what you mean by "fair" coin.
If you flip a coin 10 times, and it comes up heads every time, you are (randomly) lucky.
What are the chances it will come up heads on the 11th try?
(hint) It's the same as the 1st time you flipped it...
Random Means All Over The Place
So you can flip a coin 20 times and it is a small sample?
If the results are not 50/50 that sample is biased.
A biased set of measurements can indeed result from totally random errors.
Does that mean you turn an angle 20 times? I think not.
I believe 2D-2R is too large a sample for most surveyors on this board.
Most of all "random" does not mean "all errors cancel out".
Paul in PA
I Disagree Bill
When you apply least squares adjustment, you are implicitly certifying to the algorithm that the measurement errors are samples drawn from an unbiased gaussian normal population. It's built into the mathematics. If not true, the results may not precisely reflect reality, although in most cases they will still be as good as any adjustment method will achieve.
The statistics generated by least squares algorithm are dependent on that assumption. If the assumption is not close enough to true, garbage in garbage out. An improper calculation of the confidence ellipses cannot be relied on to compensate for (unknown to the algorithm) assumptions that may or may not be defective.
What is important is that errors from measurement to measurement do not stack up against you, but rather mix in a "random" manner.
As an example of a random error, if your centering is sloppy each centering error is a sample from a random population (how much, which direction), and the resulting errors will probably not stack up, but just show up as a larger post-adjustment sigma, which then causes a proportional increase in the error ellipses. Each centering error is reasonably modeled as a sample from a population that fits the assumptions.
Some systematic errors that will stack up against you can be detected in the adjustment, even though not well modeled as random. If the prism pole bubble is out of adjustment and used only as the foresight so in a traverse you always measure off the same amount to the right of the actual point, then the adjustment on a closed traverse will show a poor closure and the adjustment will still do a decent job of distributing that error. The statistics will tell you that there is a lot of error, and you don't need any artificial factors to compensate for the fact that the errors weren't gaussian.
Some systematic errors stack up against you but aren't so easy to detect. If the bubble is out (or your EDM factors are wrong) so that you always measure short of the actual point, the closure may look pretty good despite the errors. So as usual, "it depends". Using the F-distribution instead of the correct Rayleigh distribution to get a larger ellipse won't match up to anything in the real world to give the right compensation for the bubble problem, and it would falsely penalize you if the bubble isn't out.
Regardless of how the errors actually occur, redundant measurements are good. They are best if they check the other measurements equally well. The mathematics of the least squares algorithm assume that the check measurements are well-distributed, and if not then the reported statistics are not representative of the real world.
If you close an N-point traverse with N distances and N angles, then you have three check measurements and they will be nearly equal in their ability to check all the other measurements of the traverse. If you measure across the traverse to an opposite point, that redundant measurement does a decent job of checking all other measurements. If you get check measurements by running repeated sets at one point, then you have not equally checked the other side of the traverse - you would need to run sets at all points to do that.
Random means + or -
> Please explain what you mean by "fair" coin.
A fair coin is one that when tossed has a 50% chance of showing heads (obverse side) and 50% chance of showing tails (reverse side).
>
> If you flip a coin 10 times, and it comes up heads every time, you are (randomly) lucky.
When you consider the chance of that happening as a random event (1:1024), most people would think that examining the coin to see whether both sides were identical would be the quickest route to the best explanation.
> What are the chances it will come up heads on the 11th try?
> (hint) It's the same as the 1st time you flipped it...
Well, if the coin isn't a fair coin (e.g. both sides are heads), the random chance model doesn't apply, and that was the point.
Random Means All Over The Place
> So you can flip a coin 20 times and it is a small sample?
>
> If the results are not 50/50 that sample is biased.
That is obviously not true as you will see if you bother to compute the chance of 20 tosses of a fair coin producing exactly 10 heads and 10 tails.
> A biased set of measurements can indeed result from totally random errors.
No, the estimate drawn from a set of measurements containing random errors with zero mean (such as the normal distribution) may not be a very good estimate, but it won't be biased.
> Most of all "random" does not mean "all errors cancel out".
You may be arguing with yourself. I would think it's understood that all estimates based upon measurements and observations with random errors will themselves contain random errors. The object of professional surveying is to characterize the inherent random errors in measurements, minimize those that are systematic in nature, and make reliable estimates of the resulting uncertainties in quantities derived from the measurements.