How to process my drone data the bestest way!
sorry for the lengthy title! so we have all this cool software. Pix4D, Autodesk recapture, and MicroStation, I’m just a simple surveyor so the drone data is a little intense for my processing data knowledge. Any comments would be greatly appreciated! at this point we are pretty good at flying our DJI Matrice 300, and we have had some success in seeing our image in pix4d, but after that we are completely lost!
- 16 Replies
Are you asking for help on a specific part of the processing workflow?
The big question is “What do you want to get out of the data?“
Methods for sUAS data processing can vary quite a bit depending on the platform (RTK or PPK, or no GNSS? IMU or no? LiDAR or imagery? GCPs or no GCPs?) and the goal of a project.
Are you looking to develop a surface from the dense cloud? DSM or DEM? Planimetric mapping? Manual or automated feature extraction? Orthorectified imagery? Or maybe just an approximate image to underlay in a survey?
Pix4D has a lot of training modules both online and in-person. Why not start there?
“…people will come to love their oppression, to adore the technologies that undo their capacities to think.” -Neil Postman
Pix4D training modules and YouTube, then ground truth your projects in high and low areas as well as areas of dense cover (grass, briars, bushes etc). Also take some spot checks on hard surfaces and any areas where the surface is uniform. The software seeks to match reflectance values within the pixels or cells of the image. Low flights over corn fields, sand or snow can be surprisingly inaccurate.
sorry for the lengthy title! so we have all this cool software. Pix4D, Autodesk recapture, and MicroStation, I’m just a simple surveyor so the drone data is a little intense for my processing data knowledge. Any comments would be greatly appreciated! at this point we are pretty good at flying our DJI Matrice 300, and we have had some success in seeing our image in pix4d, but after that we are completely lost!
We put out ground control points then fly the project using a mapping app. We do a grid (for example N-S flight lines then E-W flight lines) if practical.
Import the photos into Pix4D and, if necessary, correct the camera elevations to your ground elevation plus your flying height, convert to meters. +/- a few meters is close enough. This makes finding the targets a lot easier. Our dji Phantom elevations are usually way off. Our new Freefly Astro is within a few meters without PPK.
I set up a processing style that classifies the point cloud then does a reasonably good job putting contours on the ground.
Choosing feet it will automatically select your state plane zone.
Run Step 1 only. Then go into each target and pick the target in the photos, I prefer at least 10 picks. After the targets are picked -reoptimize. When you are happy with the statistics run step 2 and 3. This can take a long time depending on how many photos. Weekends are the time to process step 2 and 3 on big projects.
@rover83 So we are using Lidar Imagery and have a rtk base station. We are doing a topographic survey for a proposed road diversion around some existing homes. The majority of our topography is over empty fields with very little tree coverage so we figured it would be the perfect time to set up the drone program. (it’s been sitting in a closet for months which I think is a crime!) So far we have set control throughout the project. (Staggered every 1000 feet pk nails with painted L shape panel points). We have completed several fly over missions and have collected about 70% of our information. Today we will be attempting to download and process our points. My firm has purchased an expensive heavy-duty computer to help the processing time. Now the fun part.
Question 1- how do I get these points out of the controller? SD card? Cable? Email? All seem fairly similar to the process we use for our TSC7s.
I’m really not trying to give you a hard time here, but if your first specific question is “how do I get the raw data into my computer?” you’re much better off reading the manuals for the sUAS platform and sensor and following their instructions, then running through some Pix4D tutorials, and maybe processing a sample project with real-world data (perhaps a subset of the data being collected for this job) before coming back here with questions.
The Part 107 license only lets you fly the platform. It’s probably not your call, but I’d be wary of marketing professional sUAS surveying services until the firm moves from “completely lost” to at least “somewhat confident in our results”. There’s nothing wrong with jumping into a new area, but workflows need to be tested and evaluated prior to going after work.
“…people will come to love their oppression, to adore the technologies that undo their capacities to think.” -Neil Postman
Be prepared for bad data over “empty fields”. Pix4D, as well as others, are very weak with low contrasting images. I agree with most others here that you really need to get spun up on the basics.
I would say stop take a deep breath and grab a book or two on the science behind RSI or LiDar first. This will help you understand signatures and what is going on. Also maybe grab a basic book on photogrammetry so you get an understanding of that. No different than surveying with a total station and data collector. Anyone can learn what buttons to push. But when you have a understanding of what the buttons make the software do you will be miles ahead in the long run. If you have been surveying any length of time and have a good understanding of strength of figure you will master how to set GCP points. Learn the science at least the basic level then follow the above advice for sure. Now I have not run pix4d. But I have processed a lot of lidar. Low altitude high altitude stationary mobile. But learning the workflow is the easy part learning the why takes a few late nights reading and understanding. MSI RSI ll of it and more have pros and cons. Learning what type of platform performs in highly reflective environment vs say black fresh asphalt will save your bacon. Or over water. Clay dirt like mentioned above sand. Dead grasses weeds etc.
take for instance the plant okra which i love fried puts off a signature so close to the hemp plant they are often mis identified. Had the boys show up at my house and I laughed because I had seen the helicopter come over my garden. And watched unmarked cars watching the farm lol. I invited them to look. All the wanted. They laughed once they realized there processor made a rookie mistake . When you get to looking at the numbers and seeing areas you walked you will know which areas are the most suspect for error sources.
Now i am sure the technology that has hit the private sector has automated a lot of things that no one even checks now days. But I do know that knowledge helps. Don??t just learn software learn the science behind it. I need to read up and refresh my brain on all of this and dive into researching what would work best for us as well. Stl surveyor probably has very wise advice as i know he has probably been exposed to a lot of education and science on the subject beyond what most will ever get. I will have to look for an old book that??s probably outdated but still relevant. And post it here. As you might be able to find it cheap on used market. I will look this weekend and see if i can dig it out probably could speed read through it in a week with a hour or so every night to get some basics down.
It??s a tool like any other tool. If you need precise elevations then UAS is not appropriate for that.
You are relying on an automated software process to match pixels. Sometimes that goes off the rails, especially in vertical. Ironically uniform surfaces such as pavement are worse than soft surfaces with more texture.
For us the main advantage is the orthomosaic and the contours in cases where general lay of the land is needed but exact precision not so much.
We started our SUAS program having a third party perform photo processing. I found that was a great way to make the program profitable while we got our sea legs under us.
Plan on excessive ground truthing and experiment time while you learn. If you do not use value pricing (different thread) it will eat your lunch.
If you need precise elevations then UAS is not appropriate for that.
I mean, if we’re talking strictly SfM workflows with only RTK and no GCPs, or vice versa, sure.
But a good LiDAR platform (multiple returns, PPK + IMU + GCP checkpoints) will go toe-to-toe with the average terrestrial topographic survey, and outperform it when it comes to detail. We employ both methods, and each has its place.
Plan on excessive ground truthing and experiment time while you learn. If you do not use value pricing (different thread) it will eat your lunch.
This is absolutely true. We did a LOT of testing to ensure our collection and processing workflow was solid and we still do a great deal of ground truthing and check shots. I wasn’t involved when we started up our UAS program, but we also hired several experienced pilots and remote sensing experts, and now that I am getting more involved, it’s obvious that the up-front work has paid off.
The value-added pricing is critical. The quality LiDAR units are not cheap, but there are many clients who will pay good money for a large amount of high-precision data delivered quickly and efficiently.
“…people will come to love their oppression, to adore the technologies that undo their capacities to think.” -Neil Postman@rover83 What platforms and software are you using. You are correct no matter how you slice and dice it testing getting a workflow check routine and procedures will save a headache or two any day of the week. Prior Proper Planning Prevents P!$$ Poor Performance.
I worked for a guy. His idea of planning was take a few minutes and get a field package out on job x first thing x day morning. I ask what is the job where is it etc etc. Its a topo or alta or both. Just get the field folks started. That was his planning. Before they arrive go locate this locate that. We will traverse later. Next day go traverse. Start traversing stop i need this on datum. Every project every time. Always rushing with no plan of action. Me Once I know what and where my first thing is get control established and checked . Nothing else matters until that is done. No joke. On one large job they had 3 different coordinate systems assumed all on same job. Nothing tieng one system to the other. Just bring it to cad and rotate it. Always fighting the workflow. He always said planning doesn??t pay the bills. I said not planning is not paying either.
The value-added pricing is critical. The quality LiDAR units are not cheap, but there are many clients who will pay good money for a large amount of high-precision data delivered quickly and efficiently.
After working at one of the bigger operators in the manned fixed wing dealios, the cost of those versus drone when you compare the data quality and volume is pretty close, but it’s no comparison once you’re into anything bigger than a square mile. drones are a great bridge for the surveyors not wanting to become certified photogrammetrists. Lots of national companies are constantly flying to collect data for NICB etc and then the downloaded data is often paid for and supported by regional and local government. And just had to be requested. anywho,
Applanix, Global Mapper, TBC, the Hexagon software that supports their platforms, Reigel, etc etc etc all do great, but I’m not in that realm anymore so know little about the Pix4D etc which was a commercial but more like high end consumer version of what we played with. OMR has the best insight as he’s played in the secret squirrel world and they use lots of proprietary TS flavors of stuff that eventually trickles out to us.
Is is Friday yet….?
@olemanriver He always said planning doesn??t pay the bills.
Sounds like that guy took management training from my employer.
Planning always pays off, unless you don’t know what you’re doing. It pays even more when we add the cost of planning to the contract…which we should be doing anyways because that’s, uh, part of the project.
I just can’t wrap my head around the concept of having the authority and ability to charge for a valid part of the project, deciding to not charge for it, then complaining that said part of the project doesn’t pay the bills while also killing efficiency and screwing your own team. Sounds like surveyor business practice 101 to me though.
“…people will come to love their oppression, to adore the technologies that undo their capacities to think.” -Neil Postman@rover83 lol. Well I try and bite my tung because the Marine and Redneck in me can get all riled up on that stuff. Even when I first started surveying. One of the first things i was taught by a 62 year old LS who I worked by in the field was planning when we arrived at the site. He did all the other before and he was teaching me every day. By the time i climbed the ladder to I man. He would ask me what we should do and how and why. He always corrected me when i did not ask the right questions. He would listen and then say. That is all fine but you have not asked me the right questions yet.
He always said planning doesn??t pay the bills.
There is a line of thinking that says that a good plan acted on right now is better than a perfect plan later on. Get started and make adjustments as you go. But generally I agree with Rover83. Time spent planning pays off. As long as you don’t let it get to “analysis paralysis”.
Pix4D training modules and YouTube, then ground truth your projects in high and low areas as well as areas of dense cover (grass, briars, bushes etc). Also take some spot checks on hard surfaces and any areas where the surface is uniform. The software seeks to match reflectance values within the pixels or cells of the image. Low flights over corn fields, sand or snow can be surprisingly inaccurate.
Your input on this is one of experience. Ground truthing is more than simply shooting some points. If you don’t ground truth shots near the horizontal and vertical limits, you are so right–you will screw up. Also agree with your input about sand and snow or any smooth surface like that. Pix4D and some of the others are good but are dangerous if not properly ground truthed with a knowledge of what ground truthing really does.
Log in to reply.