How do you store your position/height data?

I keep my data from different sources separate, I collect robotic total station data in field coordinates and assumed vertical datum; my network RTK data in SPC and ortho vertical and my static GNSS data in SPC and elliptical heights. If the survey is being prepared for other users, the final drawing is in SPC and ortho so they can add additional data easy. If the survey is being prepared for a signed and sealed PDF; the drawing stays in field coordinates for X and Y, SPC for N and E, and adjusted ortho datum based on a real control monuments. Coordinates are easily converted when imported and control points have a blank, G or S in order for me to keep them straight. I AM SURE there is a better way, but this works extremely well for me. For stations heights; static GNSS is always fixed 2 Meter, networks RTK and robotic total station heights are adjustable and both heights are recorded in my raw data. I rarely change my network RTK heights during collection, but I am constantly changing my robotic total station height and try to be real careful.

Right now? We store point data in CAD drawings and CSVs, which are marginally better than on a scrap of paper in a drawer.

In the future? We're working on a framework for storing ECEF XYZ, plus any local coordinates, orthometric elevations and a rash of other attributes as needed/desired.

Raw field data is stored in the Field Data folder with sub folders named with the date.

TBC projects are in the TBC folder. After processing data is exported to a CSV in the local SPC and orthometric height because that is most useful in CAD and to the downstream users.

A lot of the point cloud software doesnƒ??t know anything except 3D grid coordinates.

If I told a Civil Engineer here is your spherical coordinates with ellipsoid heights but you can transform them using these formulas heƒ??d probably figured I have lost my mind and call the psych intervention people.

1. Every measurement I make is mine to keep. Therefore I monument my control with something more permanent than mag nails or wood hubs. Typically I use capped iron rods or Bernsten style brass plugs.?ÿ
2. Any job can turn into the starting point for a future job. Even when the job is a simple in and out lot job I figure that the neighbor 2 doors down might call me next year. For a very modest extra effort and cost I can arrive at that job already half done.?ÿ So even if the job at hand doesn't call for elevations I'm going to collect data 3d, because the next one might, although I don't always reference the work at hand to a benchmark (if the work is done by GPS getting on an objective elevation basis, via OPUS,?ÿ is easy).?ÿ Now that I'm working for a mid sized city and all my work occurs with a box about 4 miles on a side this goes x10.?ÿ
3. I live and work in a place that has defined Low Distortion Projections. Convergence angles on the order of 2 minutes or less and combined scale factors that start with five 9's, or better. Every project beyond the most trivial goes on that coordinate system.?ÿ
4. StarNet is my core software. Everything goes into the StarNet grinder and 3d results come out. Perhaps I should know more of the details of how StarNet handles things, but I don't. I have other fish to fry. 25 years now and it hasn't let me down.?ÿ

Any job can turn into the starting point for a future job. Even when the job is a simple in and out lot job I figure that the neighbor 2 doors down might call me next year. For a very modest extra effort and cost I can arrive at that job already half done.

It's for this reason that I cannot understand why so many of us store data under a dozen layers of folders, in an archived network or external hard drive, with minimal metadata (or metadata stored in another folder in a totally different format) and in a format that is difficult to retrieve and reproject or move forward/backward in time.

Time is money, data storage is insanely cheap and a geodetic database can be queried and transformed in an instant.

@rover83?ÿ

I agree. It's pretty easy if you're interested to learn just a few practices.

When I started into land surveying, I had already been doing database stuff, and was getting my GIS certification, and previous work in geophysics to boot.

My first response was "Why are we establishing new control, that house right there we worked on 4 months ago, the control is in isn't it?"....blank stares...that was 2014..

The heavy lifting people do is amazing to me when all you need to do is continue building your network and growing the dataset.

Using Leica now it's neat because we essentially are building an LDP for each project site based on location of monuments, corners and the calibration of the height to tie everything down to the project.

Technology is a tool.?ÿ using it is a decision.

?ÿ

?ÿ

Watching the most recent NGS Webinar on the upcoming revised SPCS I heard the presenter mention that most(?) users somehow retain height information when using SPCS.

I not a fan of the SPCS due to it being 2D (Northings and Eastings) ?ÿwith no provision for capturing height information.

Getting back to the post title question, how do you store your position data? If you use SPCs, how do you account for station heights?

How do commercial softwares address the issue of combining GNSS-derived coordinates with those resulting from conventional optical observations?

Any enlightenment is welcome. I have been retired for a long time and not worked with SPCS since the 1980s.

Is this a solution in search of a problem??ÿ

After rereading, I realized I only comprehended the data collection portion of your questioning statement. I think we WILL eventually go to a complete 3-dimensional system. Before now, the vertical degraded so much faster than horizontal, that a different system of measurement was required for each component. We had trig for horizontal and level for vertical; and both had a different control datum and control monumentation. Back then, a mapping plane was essential as a product for your work. The State Plane System worked really well then and is still useful today. Even with GNSS and software, we are stuck between where we were and where we will be. Vertical is still the weaker component, maps are still 2-dimensional and we still work off two datums. There are plenty of signs that it will not be long before your quest to collect, prepare and deliver survey products in a complete 3-dimensional system are ahead. 1) The next vertical datum will probably be delayed until they can produce a combined 3-dimensional datum just for the uses you have contemplated. 2) Data collection now includes data in cloud form where northern and easting are not a separate component from elevation. 3) Mapping is already trending to be screen-based instead of paper plan-based (my boat screen already calls down as side and side as down). 3) New post-processing software has the option to squeeze every bit data from each signal's size, rate, twist, deviation, polarization, noise and pulse, that the vertical component is not weaker than the horizontal. 4) Proposed geoids will be digital combinations of gravimetric and geopotential models using the proposed 3-dimensional datum. (And yes I had to look that up because I do not understand it nor could I remember it.)?ÿ

I have no plans to begin transitioning from 2-D with an elevation, but if I were younger I would probably already heading in that direction. I have probably not addressed GeeOddMike's concern s, but with little football today, I have wasted enough time until Army Navy kickoff.

?ÿ

?ÿ

@rover83 This is where geo databases and adding any and all data with the ability to assign the metadata in a variety of ways even adding it as attributes for various reasons. Whe in the USMC every point even a ground shot had everything assigned to it ECEF lat long ht utm mgrs etc etc also that was the good because no matter what I needed I could query years of data all over the world and bring it in to use any way i saw fit.

Everything is collected with GPS units or the robot. Both use the same DC flies. They are always on a projected system with underlying Lat, Long, Height data. Everything we do is NAD83 based.

So from the smallest to the largest jobs there is underlying geographic data available if needed. The points are stored into TBC, then sent to CSV files, then insert into Autocad; all the files are tagged to job numbers.?ÿ

All elevations are Geoid based, usually we are on Local Bench Mark data so that ellipsoid heights are slightly different from an OPUS or CORS solution. FEMA controls elevations which are based given local Bench Marks (at least their maps say so).?ÿ?ÿ

If we are in some remote location then elevations are strictly CORS/Geoid based along with the coordinates.?ÿ

If levels are run and adjusted the resulting control point elevations are added to the 3D coordinate, thus changing the orthometric and ellipsoid height for the point.?ÿ

The XY may be a modified state coordinate system or an LDP. We always try to work as close to the surface of the earth as possible. Unless there is a SPC request or the client doesn't have the ability to work outside a few canned coordinate systems.?ÿ?ÿ

All elevations are Geoid based, usually we are on Local Bench Mark data so that ellipsoid heights are slightly different from an OPUS or CORS solution.

And there it is. The smoking gun.

This is exactly one of the reasons that "storing data" versus "using data" is a vitally important question.

Current iterations of "geodesy" that most surveyors ("geo-professionals"?) are likely using, are geometric datums, meaning 3d all the time, which include ellipsoid height, because it's pinned to the ECEF origin.

The minute you break the connection from the actual observed measurements that deliver ortho heights via the geoid model, you're sunk. The geoid model is spongey, and going backwards won't get you back to the exact measured ellipsoid height. That's the error in the model.

So be careful what you're storing vs. what you're using?

It will be interesting to see how long NATRF2022 (or subsequent) will be around before professionals actually start using it as the sole method of reporting spatial data.

It would seem that only brand new projects, with no historical or spatial tie to an existing framework will be the best candidates for such endeavors.

Otherwise all projects (old and new) will have 437 different sets of coordinate values to choose from, most of which will be entirely fubar because the "land development/platting process example" someone gave earlier or in an earlier thread.

All elevations are Geoid based, usually we are on Local Bench Mark data so that ellipsoid heights are slightly different from an OPUS or CORS solution.

And there it is. The smoking gun.

This is exactly one of the reasons that "storing data" versus "using data" is a vitally important question.

Current iterations of "geodesy" that most surveyors ("geo-professionals"?) are likely using, are geometric datums, meaning 3d all the time, which include ellipsoid height, because it's pinned to the ECEF origin.

The minute you break the connection from the actual observed measurements that deliver ortho heights via the geoid model, you're sunk. The geoid model is spongey, and going backwards won't get you back to the exact measured ellipsoid height. That's the error in the model.

So be careful what you're storing vs. what you're using?

It will be interesting to see how long NATRF2022 (or subsequent) will be around before professionals actually start using it as the sole method of reporting spatial data.

It would seem that only brand new projects, with no historical or spatial tie to an existing framework will be the best candidates for such endeavors.

Otherwise all projects (old and new) will have 437 different sets of coordinate values to choose from, most of which will be entirely fubar because the "land development/platting process example" someone gave earlier or in an earlier thread.

It will be interesting to see how the roll out gets adopted for 2022 (maybe coming out in 2025?). I will guess the DOT will use it for new projects. The main issue will be the elevations, since FEMA controls much of that it could be years before it useful. New flood plain maps will need to be issued and then politically accepted.?ÿ

There is a HARN point on a hill in the city locally. It's a very stable NGS NAD29 monument that's also a first order benchmark. So it's been in the GPS system from the beginning, before CORS or OPUS. There is a data sheet for it with 4 ellipsoid heights. The 4 ellipsoid heights range from 1173.30m to 1173.18m or a difference of 0.4'. Of course 0.4' is a huge number vertically, we know the first order bench mark is stable from levels from it to other bench marks in the system. They all work exceedingly well to each other. The NAVD88 value for the mark didn't change at all through the years and no combination of ellipsoid height and geiod height ever combined to result in the published NAVD88 value. Occupying the point and sending the static file to OPUS or calculating it from the close CORS also never get very close to the NAVD88 number. However, the newest Geoid 18 model has made a difference and now you will be just under a tenth of a foot using it with the local CORS point. So accepting the NAVD88 number and letting the geoid model calculate the ellipsoid number was clearly the correct procedure.?ÿ

?ÿ

?ÿ

So accepting the NAVD88 number and letting the geoid model calculate the ellipsoid number was clearly the correct procedure.

I was right there with you. The whole way. Until that last sentence.

And I think that is the crux of the OP original question.

There is no direct link between 1d ortho heights and 2d/3d GNSS/remote sensing heights, etc. They are fundamentally different, and calculations used to go between the two are merely approximations. They are decent approximations, and seem to continually get better with subsequent iterations. And that's just NAVD88. There are countless other vertical datums in play that we haven't even mentioned.

Until one is defined in terms of the other, the disconnect will continue.

So, yeah, what is the best way to acquire/store data to preserve the benefit of native GNSS, remote sensing, and terrestrial instrument data when their native data formats are not homogenous?

Any station can have any combination of coordinate values, depending on its use. Maybe that's not such a good thing? Standardization does promote certain benefits.

Imagine if RTCM, or TCP/IP weren't standardized. Surveying, and most everything else would probably look much different, if it even worked at all.

So accepting the NAVD88 number and letting the geoid model calculate the ellipsoid number was clearly the correct procedure.

I was right there with you. The whole way. Until that last sentence.

And I think that is the crux of the OP original question.

There is no direct link between 1d ortho heights and 2d/3d GNSS/remote sensing heights, etc. They are fundamentally different, and calculations used to go between the two are merely approximations. They are decent approximations, and seem to continually get better with subsequent iterations. And that's just NAVD88. There are countless other vertical datums in play that we haven't even mentioned.

Until one is defined in terms of the other, the disconnect will continue.

So, yeah, what is the best way to acquire/store data to preserve the benefit of native GNSS, remote sensing, and terrestrial instrument data when their native data formats are not homogenous?

Any station can have any combination of coordinate values, depending on its use. Maybe that's not such a good thing? Standardization does promote certain benefits.

Imagine if RTCM, or TCP/IP weren't standardized. Surveying, and most everything else would probably look much different, if it even worked at all.

So which ellipsoid height do you use, there are 4 different ones, almost .5' different.?ÿ

?ÿ

So which ellipsoid height do you use, there are 4 different ones, almost .5' different.?ÿ

I use the one that yields the orthometric elevation in the desired datum when used in conjunction with the geoid model prepared for the purpose.?ÿ

I would use the ellipsoid height that I measured, which hopefully is the one within 0.1ƒ?? of published NAVD88.

My point wasnƒ??t that you did anything wrong. But rather, that by holding one to fix the other doesnƒ??t really work either.

The published NAVD88 height is the orthometric height. Which is separate from, and distinctly different from, the ellipsoid height.

Thereƒ??s no way around that.

I guess a better way to say it might be, if youƒ??re going to ƒ??store dataƒ?, you may want to include the measured ellipsoid height and a measured (leveled) ortho height, else you can run into the problem weƒ??re discussing.?ÿ

Both values are valid. Which one you use is a matter of circumstance.

All because geoid derived ortho heights are approximations.?ÿ

Otherwise all projects (old and new) will have 437 different sets of coordinate values to choose from, most of which will be entirely fubar

As with some others as described above, my approach has always been to maintain the observation data and adjustment criteria.?ÿ That way I can reprocess it at any time and in any way I need to in order to produced the desired end product.?ÿ Due to varying requirements I sometimes end up with more than one set of positional values for the observed points, but as long as the observation and adjustment information is available I have what I need to move forward.

As many have posted, there is a problem with how to deal with heights in a database of survey points. My preference remains 3D. Namely ellipsoidal coordinates: latitude, longitude and ellipsoid heights with appropriate annotations regarding datums to which they refer.

In particular, I remain unconvinced that the SPCS provides the solution as it is a 2D system.?ÿ

Given data in ellipsoidal coordinates it is simple to use NGSƒ?? NCAT to provide values to the clientƒ??s preference.

I recognize the difficulty dealing with the three differentheights types: ellipsoid, orthometric and the ellipsoid-geoid separation.

Even the NGS has problems dealing with this issue. See the case of AH1762 shown in the images below. Note that this monument had a first-order second-class differentially-determined NAVD88 height superceded by a number of ellipsoid-derived heights. Note as well the number of ellipsoid heights.?ÿ

?ÿ

?ÿ

We had two rather spectacular FUBAR's from people using OPUS/Geoid model derived elevations. Holding the processed ellipsoid height and adding on the Geoid height to get ortho heights. It was sometime after 2003 for both of these unrelated FUBAR's. One was a county road that was designed on the new county contour maps which were flown and controlled by NAVD88 elevations. The road was designed using the new mapping but the control points were placed by the engineering company using the XY coordinate system for the mapping but OPUS/Geoid for the elevations.

0.3' of a foot over 2 miles of road was a lot of dirt.?ÿ

The cost of that FUBAR was hiring us to cross-section the road using the new control (I know, I know, just lower or raise the control points, but engineers.....).

The second was worse. An out of state engineering company did a survey for a new building at a local campus. They didn't use local coordinates but coordinates from their state system. On top of that they did the OPUS/Geoid thing and produced the same 0.3' error. Well one building got designed to merge into an existing building and there was 0.3' difference in finished record floor elevations based on NAVD88 and their OPUS elevations. We got called after the building was half done and the construction company was trying to fit them together. It was too late. That was a terrible mess.?ÿ

So no, I'm not going to go with OPUS/CORS ellipsoids and Geoid Models. I'm going to continue to use NAVD88 monuments and apply the Geoid (unless I'm doing more remote work). I will say that the new versions will get you within .1' using an OPUS solution and the Geoid 18 model. If you use the local CORS and process a static file it's even better. Usually I will be about .06' "off" doing that. So Geoid18 was a big improvement over the really bad days of the early 2000's. We can all thank people like Bill93 for that.?ÿ

I recently had the PC sitting on the local HARN point and I went to a first order bench mark on the hill above the office and checked in with RTK,,,,,0.02' vertically using the HARN point, it's NAVD88 elevation, Geoid18 applied. Works well. Those bench marks are about 2.5 miles apart.?ÿ

The new 2022 elevations locally will move down about 2' putting them close to NGVD29 elevations. They will be totally GPS based so I'm told. It's good they are shifting so far so the confusion will be muted. But, they will be floating subject to movement at all times. So even more so it will be XYZT coordinates. When this will happen is up for debate, the FEMA maps aren't going anywhere for many years and they are NAVD88 based. Imagine lowering the elevations. Your house at 3502' will now measure 3500' and the BFE on the maps for your location is 3501'. This will be the exact situation for hundreds of houses and buildings near my office, including this office.?ÿ