I have been trying to work with a particular dataset of lidar information since about June. I have tried numerous ways of working with the data trying to get a surface or contours from it, to supplement our ground run survey for roads, creek channel cross sections, etc. This is at the request of our client. The lidar information is in an *.adf format if I remember correctly.
The data has been shared from our County's GIS department, and they have tried bringing the data into ArcGIS, exporting drawing files, shape files, etc., and either the elevations were correct, or the horizontal position was incorrect. I could never get the information in the right place horizontally with elevations from ArcGIS.
A new person with one of the members of the team (several firms are working on this massive project) was able to get a Civil 3D surface created in a *.grs file. We were not ale to do much with that, but after a few days of working with it, I was finally able to extract polyline contours which were in the correct place and with the correct elevations. Trying to create a surface from those contours (I know that's not the best way to do it), would crash the computers at the office. I was able to figure out with Carlson Tech Support (Thanks Lon Watson!) to reduce the polyline verticies, and then I was able to create a surface that I could work with. We ran into additional issues, and that surface was not in the correct place horizontally after the team member realized there is a shift in the data when e brought it into Civil 3D.
I know, this is kinda complicated, and unorganized.Carlson Tech Support has been right there with me trying to help me figure this out.
Realizing that I cannot work with this *.grs file in Civil 3D (I do not know much about it at all), I decided to go back to the source of the original lidar data, the *.adf files, and start over again. I found out that I could bring in the *.adf files into QGIS, and I was able to create a shape files of the project boundary, and then clip the lidar data, and export a GeoTiff.
I am using a demo version of Carlson Civil Suite, and tried importing that using the Grid File Utilities. That was okay, but the resulting surface and grid files were over 300MB in size, and it was crashing out. I even bought a new personal laptop because of the issues I was having. In talking with Carlson Tech Support, Lon told me that the Point Cloud software may work better, because AutoCAD was never meant to deal with the massive number of points in a GeoTiff, or a point cloud. I also spoke with Butch Herter with Carlson as well, and he pointed me to the online videos for Point Cloud. I downloaded a demo version of Carlson's Point Cloud last week, and tried it at the office. I kinda stumbled through it before watching the videos.
After watching the videos today at lunch, I was able to take the GeoTiff I extracted from the lidar data, and was able to create a point cloud project, clean the data, create a mesh, and export the tin into a file size that is manageable in about 20 minutes or less. I look at the number of hours I have spent working on this trying to get to this point, and I am frustrated with myself. I tried so many different approaches, and everyone I spoke with was just as lost as I was. I probably have about 100 hours of my own time tied up with this because I cannot stand not being able to figure something out. I feel like it was my responsibility to be able to do this efficiently, and I was not able to figure it out.
Carlson Point Cloud was definitely the right software for the job, and I will work it into my department's budget, however, we have to purchase a new base/rover setup at the moment, and that will take priority. The price tag is around $5,000, which I have that much lost billable time in this if I counted the number of hours I spent working on this on my own time.
Is there any point cloud tools that I may be able to utilize in the meantime that may get me by until I can get this worked into my budget and purchased? I am finding the strengths of QGIS, and that will end up being my main GIS software. The Grass tools are pretty strong, and I am easily able to export contours from the DEM/GeoTiff. I need to be able to take the DEM to a usable *.tin file. Lon said that any surface file over approximately 50MB will probably cause problems.
Thanks for reading this long post, and the ramblings. This has been an unbelievably frustrating project with the issues I have encountered. I have learned very valuable lessons, and have expanded my skill set tremendously, however, I feel it is my responsibility to be much more efficient in this process to increase our company's profitability, as well as provide the client with the end product in a more timely manner.
Any suggestions and recommendations are greatly appreciated.
Jimmy
A couple of thoughts. In Civil 3D, after you make a surface from a point cloud, are you then making a project so that this surface can be a data shortcut reference object in the "working" drawing? That way the point cloud data sits somewhere else.
Also, when working with drawings with big and many polylines in it, you should shut down the properties dialog box, as when you hover over a polyline it can be transparently selected (I think, from memory) which causes properties to analyse it, and if it's big, hang up - same (I think) if you pick it with the properties dialog box open.
If you can upload the LiDAR file I can have a crack at it.
If you want local assistance I can help. Send me an email and I will provide a link for you to upload data to.
Thanks Guys. I "think" I am good at the moment, but I definitely have to figure out the most efficient way to go about this procedure in the future. This will not be the last lidar data set that I work with. I use Carlson Survey 2016 OEM, and will probably be purchasing the Civil Module as well pretty soon. I do not know much at all about Civil 3D. I have only used it a handful of times, and it just seems overly complicated to me.
I have uploaded the data at lunch today. I have a few fires to put out this morning.
I am going through a similar track with Civil 3D. We observed a fairly large set of scans and have a point cloud. It crashes our best computers and work flows are all but nonexistent.
Perhaps when we are done a short white paper would be in order.
My short term solution is to extract 3d faces in TBC then generate a C3D surface from those. Every direct import to C3D ended with no or poor results. I am convinced the guy who wrote EDLIN now works for Autodesk.
Global Mapper LiDAR Module...
http://www.bluemarblegeo.com/products/global-mapper-lidar.php
Jimmy Cleveland, post: 455601, member: 91 wrote: The lidar information is in an *.adf format if I remember correctly
I'm not familiar with this file type. Is it a simple comma delimited value format containing XYZ & color values or something more complicated than that?
Cameron Watson PLS, post: 455672, member: 11407 wrote: I'm not familiar with this file type. Is it a simple comma delimited value format containing XYZ & color values or something more complicated than that?
It is a collection of files (dblbnd.adf, hdr.adf, prj.adf, sta.adf, w001001.adf, w001001x.adf) and they are all binary and form a grid. They typically are in Lat/Long or UTM and have to be projected to the target coordinate system.
Have you tried MeshLab? I haven't had a chance to play with it yet, but it is open source and a free download.
Cameron,
I think it is some sort of ESRI file that the "point cloud" information is stored in. The files are all 1.1GB in size.
Here is a link to the geotiff I created.The coordinate system is TN NAD83, NAVD88
https://www.dropbox.com/sh/te5tsnhkmmqfxua/AAD8FP6X4fhC9npgvWJBzNWQa?dl=0
I was able to generate a tin using the Point Cloud Module, and am working with that.
Thanks! I appreciate anyone looking at it, and offering any suggestions.
Here is a screenshot of a piece of the entire surface in Civil 3D, with contours at 2 and 10m. Point cloud processing time was about 2 minutes. The .dwg file is 8MB.
Yes I was dealing with something similar. I was pulling data from the TN LIDAR project and some genius decided to put the data up in zlas. Luckly somebody gave me a link to convert back to a las. After screwing with it for half a day. I get this all the time. People want to give me LIDAR data in a format other than .las or .laz. It drives me nuts. I would say the Global mapper would work good. Get a demo first. It even has some automated feature extraction. I use LP360 which works stand alone or inside arcmap. I also use terrascan, terrasolid, terraphoto that works inside of microstation. (I try to let others do that and I stick to LP360). Then I will classifiy model key points (thin the data) depending on what contour interval you need and export that class to a new .las . Then I bring that in Civil3d. One thing I found with working with large files is to turn off your automatic save and backup.
I did this recently using an *.adf file in Civil3D. It took a while to import the adf file (an ESRI format I think) and create a point cloud database, but once that is done creating a surface is pretty easy. Keeping the surface in a separate drawing and using a data shortcut/reference is the way to go with Civil 3D.
Check this link and video out. Slightly different approaches but one can glean good information.
https://knowledge.autodesk.com/support/autocad-civil-3d/learn-explore/caas/CloudHelp/cloudhelp/2017/ENU/Civil3D-UserGuide/files/GUID-4494DB95-04DE-42B6-BF18-68AF06935229-htm.html
[MEDIA=youtube]GottqJ5r4Cg[/MEDIA]
Civil3D is complicated, but it is also very powerful. In my experience, good workflows are forged by trial and error. I commend your efforts.
I would second the global Mapper - it is great at working with all the GIS data sets - I call it my lightweight GIS
global mapper for $500 +$500 for the lidar module - it is a great tool
Once you have a LAS cloud file - Autodesk's Recap is a great tool for cleaning up areas
