Comments on the Smi...
 
Notifications
Clear all

Comments on the Smith, Choi et al Report.

11 Posts
7 Users
0 Reactions
4 Views
(@paul-in-pa)
Posts: 6044
Registered
Topic starter
 

http://www.ngs.noaa.gov/PUBS_LIB/Analysis_TXDOT_RTN_and_OPUS-RS_from_the_Geoid_Slope_Validation_Survey_2011_Texas/Analysis_TXDOT_RTN_and_OPUS-RS_from_the_Geoid_Slope_Validation_Survey_2011_Texas.html

I dug a little deeper into the above report and will not back down on my comments.

For the above report the reporters chose to go against several of the limiting parameters of OPUS-RS. Their intent was to get good elevations and the did exactly what one should not do in elevation work.

1/ They elected to use 20 minute files, which while acceptable in a majority of OPUS-RS work will not yield 100% performance. OPUS-RS relies on good satellite geometry as well as good CORS geometry. In the extant case both expectations were not met.

Good satellite geometry depends on satellites being in the four sky quadrants. This cannot be determined by PDOP, since the PDOP algorithms assume more is also better. More does not help if the satellites are not well positioned. In the last few years I have seen numerous times with low PDOP that 6 to 8 satellites are almost in a straight line cross the sky view with very few sats normal to that line. Despite whatever the PDOP numbers are, that is poor satellite geometry. In addition despite the 30 to 31 PRNs in the constellation there are still short periods of higher PDOP throughout the day. Typically the poor period is 15-30 in length. When I was doing beta testing on RSGPS, the engine that drives OPUS-RS, I found the problems with those holes and convinced Charlie Schwarz to abandon the 5 minute minimum for a longer period. The current 15 minute minimum still gives the possibility of holes especially in light of non technical users to always expect the best and not know that things can go bad. I personally use OPUS-RS but will not bet my work on such short observations.

2/ By using OPUS-RS in coastal areas one is forcing almost every solution to be outside the polygon. OPUS-RS gives those quick precise positions because it uses the surrounding CORS to allow a very good interpolation of atmospheric correction inside the polygon at the Rover position. By going outside the polygon OPUS-RS must extrapolate the corrections. Extrapolation is inherently less reliable than interpolation. While it may be OK to stretch the box in a uniformly land based environment with a stable atmosphere, coastal work with the severe atmospheric changes between the land side and the sea side moves one away from a good approximation of corrections by orders of magnitude.

3/ Also by forcing solutions outside the polygon one opens up the opportunity for the CORS to warp the solution plane. OPUS solves positions especially an elevation by holding each CORS fixed and meaning the Rover. While OPUS-RS does not hold the CORS fixed it does LS each CORS to as close to fixed as possible. Again let us consider the coastal situation. The coastal CORS, especially those closest to the coast are not on hard terrain. Subsistence aside, the CORS are subject to earth tides. Earth tides will affect the most coastal CORS the most and the most coastal CORS will also warp the solution plane. This I learned early on in GPS avoiding as much as possible including the Sandy Hook Coast Guard CORS in any of my solutions. As I recall it was less than 15 miles from NJIT CORS to SHK1 or 2 and at times I could not even get a fixed CORS to CORS solution. NJIT on Newark's solid ground surrounded by terrain and SHK1 on a sandbar surrounded by ocean and bay for 350°. Not pretty at all.

Paul in PA

 
Posted : 15/04/2015 9:16 pm
 CSS
(@css)
Posts: 231
Registered
 

IDGI. Isn't the point of the OPUS-RS stuff to test how accurate the OPUS-RS is when used under "normal conditions" - which as you noted, 20mins is acceptable in the majority of OPUS-RX work.

You seem to be suggesting they should be avoiding all the likely pitfalls that normal users will face. That would certainly boost OPUS-RS accuracy, but make it a lot less useful to users who are trying to gauge the accuracy they may achieve under "the majority of OPUS-RS work".

Your argument seems to be that the authors failed to achieve the highest accuracy OPUS-RS is capable of. I'd say (based on a very brief read of the report) that is entirely the point. The goal wasn't to achieve the highest accuracy possible but to assess how it worked under normal conditions.

 
Posted : 16/04/2015 12:10 am
(@dave-ingram)
Posts: 2142
 

John Hamilton & others - Trimble

This paper also addresses an issue that I discovered a few months ago and John Hamilton commented on here a while back as well. With some Trimble receivers (in my case R8) if you turn on L2C it will quit receiving P2 for those satellites. OPUS-RS doesn't like this.

 
Posted : 16/04/2015 2:37 am
(@paul-in-pa)
Posts: 6044
Registered
Topic starter
 

Typical Guideline Is Double Occupation Time For Elevation

Plus they had anomalies but did not consider all possible causes for same.

The three problems I pointed out will also affect the RTN network to some extent.

Paul in PA

 
Posted : 16/04/2015 3:35 am
(@shawn-billings)
Posts: 2689
Registered
 

You make a good point about the points location relative to the polygon, Paul. It's possible that most of the sites noted with few CORS used in the solution were because of points outside the polygon. This probably isn't the case for all of them and unfortunately, this paper doesn't address the relationship of the point within the polygon, except for the note that solutions used were "without warnings". I don't know if this would include warnings about the polygon. The paper also says that they did remove points with poor CORS geometry. Still not an explicit "only solutions in the polygon were used" though.

 
Posted : 16/04/2015 5:56 am
 CSS
(@css)
Posts: 231
Registered
 

Typical Guideline Is Double Occupation Time For Elevation

How will it affect the network?

 
Posted : 16/04/2015 2:37 pm
(@paul-in-pa)
Posts: 6044
Registered
Topic starter
 

Typical Guideline Is Double Occupation Time For Elevation

PDOP gaps affect the satellites used to send data to your network rover. The network solution assumes the network CORS to be in a standard position, any thing that changes that stable position gets passed on to the rover position. If the network is sending atmospheric corrections to the rover that is outside the network those corrections may not really be applicable to that rover position.

Paul in PA

 
Posted : 16/04/2015 6:42 pm
(@drusmith)
Posts: 5
Registered
 

I couldn't help notice the attention our paper has received on this board. I'm not usually on here, and only happened to stumble upon this thread (and the comments in the earlier one) when looking for something else. As I don't really have time to engage in a long online back and forth, I thought I might post some of the relevant facts, as I thought it might be helpful for readers of these posts to see some additional information. I will begin with the basic chronology, and then intersperse with some of Paul's statements.

Chronology: The original plan for GSVS11 did not call for any use of OPUS-RS at all. The original plan called for using the TXDOT RTN to quickly, and to a few decimeters of horizontal accuracy, position the DIADEM marks (on shoulder mostly) relative to the official marks (off shoulder mostly). The two types of marks were always a few tens of meters from one another. When it came time to do the RTN survey, however, some logistical snags came up regarding equipment availability and we quickly adapted the plan, deciding to use OPUS-RS on the DIADEM points. After this data was collected and turned in, the RTN equipment became available so we decided (out of scientific thoroughness) to collect the RTN data anyway in hopes of having an OPUS-RS vs RTN paper come out of GSVS11 as a by-product.

(All of the above is documented in our paper.)

During the original 20 minute occupations (slated for OPUS-RS) we did not realize that the data we collected were suffering from "P2 off". We went so far as to write the first draft of the paper, using the results of OPUS-RS on the DIADEM marks and RTN on the same marks. One reviewer, however, noticed something odd and when we dug deeper we realized that all of our collected OPUS-RS data was missing P2 data (because the specific receiver we used automatically turned off P2 when L2C was turned on). This effectively invalidated all of our OPUS-RS data. But as we stated in the paper:

"It is a testament to OPUS-RS that this was still enough data to yield results accurate enough not to cause any immediate concern. Only upon deeper inspection did the below-average performance of OPUS-RS, caused by missing P2 data, come to light."

We were about to scrap the paper, but the reviewers were eager to see SOME form of OPUS-RS vs RTN comparison done, so we took their advice and parsed the existing OPUS-Projects data (~48 hours of data on each official mark which did sometimes suffer the "P2 off" problem, but only in a small percentage of the data) into 20 minute segments. These were pushed through OPUS-RS and we had tens of thousands of OPUS-RS runs to compare on the official marks against not only OPUS-Projects but also the RTN. The paper was saved, and the results published, not using the original 20 minute occupations on DIADEM marks but instead using parsed versions of 48 hour files on the official marks. As we say in the paper, regarding the original 20 minute files:

"No further analysis of these data is presented in this paper. However, an entirely separate analysis of OPUS-RS was performed, using subdivided versions of the long-session GPS data (as previously described)."

Anyway, on to some of Paul's comments....

> I dug a little deeper into the above report and will not back down on my comments.

I assume Paul means these comments:

1) "The OPUS-RS Issues With That Report were caused by not submitting properly collected data."
2) "They Had Sufficient Data, They Did Not Know How To Use It"
3) "I Read The Article, They Knew The Problem But made no effort to correct it. They may not have known how simple the correction was. Longer files could not make up for the lack of proper data. It did improver the solution but not to the extent possible."

Only myself and my co-authors can make any claim to the intimate knowledge of what we "knew" or what "efforts we made". I will therefore restrict myself to facts.

As stated above, and in the paper, the "P2 off" issue with our original OPUS-RS data was not found until deep into paper writing because OPUS-RS was doing so well even with such poor data. After that data was scrapped, and the parsing of 48 hour data done did we have the right data for a solid OPUS-RS vs RTN analysis.

> For the above report the reporters chose to go against several of the limiting parameters of OPUS-RS. Their intent was to get good elevations and the did exactly what one should not do in elevation work.

As stated above, the intent of OPUS-RS was to save GSVS11 when the RTN proved to be unavailable, and to horizontally position the DIADEM marks within a few decimeters of the official GSVS11 marks. Since we went ahead and got the RTN data afterwards anyway, the original intent of this paper was a complete by-product of having had both OPUS-RS and RTN data an the same marks. When we discovered the "P2 off" problem, we changed the intent of the paper to evaluate OPUS-RS by parsing 48 hour files, rather than using the originally collected 20 minute files. No intent to specifically do anything to "get good elevations" was part of the plan.

> 1/ They elected to use 20 minute files, which while acceptable in a majority of OPUS-RS work will not yield 100% performance.

It is unclear what "100% performance" means. We could just have easily parsed into 30, 45, 60 or any other size of files. We chose 20. That yielded up about 34,000 files, of which some 33,000 successfully ran through OPUS-RS. That's a 97% success rate, yielding up a sub-cm horizontal and 3 cm vertical agreement with OPUS-Projects. Only 1.6% of those successful runs yielded outliers greater than 5 sigma. Not "100% performance" but pretty good.

> OPUS-RS relies on good satellite geometry as well as good CORS geometry. In the extant case both expectations were not met.

The 33,000 or so 20 minute files spanned weeks temporally, and spatially everywhere from coastal Texas, where CORS are thinner, to inland Texas, where we had better CORS coverage (as evidenced by the number of CORS in each solution in Figure 12).

Our paper did not analyze OPUS-RS by any DOP criteria, so I can not expound on actual satellite geometry.

> 2/ By using OPUS-RS in coastal areas one is forcing almost every solution to be outside the polygon.

Very few solutions were outside of the polygon formed by the CORS used.

The paper did not make a geometric analysis, but Figure 6 and the statistics of the OPUS-RS runs tell the full story. Of the 34,000 original files, 1000 were rejected for a variety of reasons. Sometimes that reason was the station was "Too Far from Convex Hull" (an OPUS-RS error code, indicating such poor geometry). This happened on 39 of the 1000 rejected files, and was, in fact, generally always near the coast.

As stated in the paper:

"Finally, approximately 1,000 files simply caused OPUS-RS to fail to yield a successful result for one of a variety of reasons (large RMS values, poor CORS geometry, etc.)"

The coverage of CORS was good enough for the average number of CORS per solution to fall between 8 and 9, with every one of the 33,000 successful runs having a minimum of 5 CORS. Examine Figure 6. In almost every point on that line, at least a quadralateral of 4 CORS surround it, and that number grows monotonically once it gets just slightly north of Corpus Christi.

> 3/ Also by forcing solutions outside the polygon....

This simply did not happen.

---------------------------------

Most of the facts stated above are in our paper. But sometimes what seems clear to an author may not be clear to a reader. The intent of this post was simply to clear up any misunderstandings. So I appreciate the chance to emphasize the good work that my co-authors and I were able to put into that paper.

Should there future questions about NGS please feel free to submit your questions to our Information Center at:

ngs.infocenter@noaa.gov

They will make sure your questions are forwarded on to the proper people at NGS. Directly emailing NGS employees is not usually the most efficient way for our employees to handle the variety of questions that come in on a daily basis.

Dru Smith
Chief Geodesist, NOAA's National Geodetic Survey

 
Posted : 17/04/2015 10:30 am
(@goddsc)
Posts: 87
Registered
 

Comments on the Smith, Choi et al Report-Where's Paul now??

Thank you Dr. Smith for taking time to provide clarification and responses to what had been stated here and in the other thread where this paper was orginally posted.

Scott

 
Posted : 17/04/2015 3:37 pm
(@deleted-user)
Posts: 8349
Registered
 

Thanks for your comments and time,

 
Posted : 17/04/2015 3:41 pm
(@paul-in-pa)
Posts: 6044
Registered
Topic starter
 

Dru Smith

My 100% comment means that a short 15 minute occupation will not yield an top rate solution 100% of the time due to the possibility of it being within a DOP gap. The longer solution gives a probability that the DOP gap is spanned or that at either end of the occupation the data is sufficient to hold a good position.

1) "The OPUS-RS Issues With That Report were caused by not submitting properly collected data."

I believe that point is agreed.

2) "They Had Sufficient Data, They Did Not Know How To Use It"

Here I am referring to the point that some RINEX files were output with P2 for the older satellites and C2 for the newer satellites. OPUS-RS was not recognizing that as the comparably same data in two separate columns. Knowing that OPUS-RS substitutes C1 if P1 is not available I requested a change such that OPUS-RS automatically recognize this fact and substitute the C2 for the missing P2 as required. I got nowhere on that request. However the online teqc when requested to parse observations to the 4 key observables C1/P1, L1, L2 and P2, will merge the C2 data into the P2 column. Had you been aware of that you could have used all the data you had with no real fuss.

3) "I Read The Article, They Knew The Problem But made no effort to correct it. They may not have known how simple the correction was. Longer files could not make up for the lack of proper data. It did improver the solution but not to the extent possible."

I may have been too harsh here.

My outside the polygon comments were pure speculation on my part as to how you may have observed.

Also the link I followed for more on the report requested payment for downloading, should that not have been freely available on the NGS website?

Paul in PA

 
Posted : 17/04/2015 4:33 pm