What you are not told about accuracy

Accuracy, precision, relative accuracy, absolute accuracy, etc. Nowadays, everybody talks about accuracy and it feels like we have kind of forgotten what it is anymore. Let’s refresh up!

A few definitions

When dealing with point clouds, accuracy is the potential difference (also called standard deviation) between the measurement of a real-world object and its actual measured size in the point clouds. Let’s say that my relative accuracy is 1 cm. Now, I go outside and measure the height of the bench. It is 63 cm. Then, I put on my BMS3D backpack and scan the area (bench included of course!). Afterward, I open the data in my post-processing software, grab the measurement tool, and … I obtained 63,3 cm! It fits my relative accuracy which had made me expect a value between 64 cm (63+1cm) and 62 cm (63-1cm). Of course, this accuracy will depend on several parameters:
– the range at which the object is captured
– the intrinsic quality of my LiDAR scanner (also called measurement noise, we will get back to that)
– the calibration of my LiDAR scanner
– environmental parameters (reflectivity of the object, is the object wet? and so on…)

Now, let’s take about precision. By definition, precision is the ability to repeat reliably the same measurement over and over. Taking back my example with the bench, let’s say I go out and captured a value of 63,3 cm while making a first scan. I then make a second scan with my mobile backpack and this time obtained 63,4 cm. I do a third scan and obtain 63,1 cm. And so on… Should I repeat the scan a thousand times and obtain exactly the same measurement, my measuring device would be 100% precise. Not necessarily accurate (it could tell me 1 000 times that the bench is 63,3 cm while the actual measurement shows a value of 65,3 cm!), but precise.

Let’s finish by mentioning the term “absolute accuracy”. This only applies to those who are able to geo-reference their output point clouds. In other words, those who, thanks to the use of a GNSS sensor, are able to shift the local coordinates of their point clouds to a “projected” system. In geodesy, one aims at measuring the shape of the Earth and understand its geometry. By these principles, we basically want to be able to locate our point clouds on a map. And GNSS technology (unjustly called GPS) is the way to do it. We all know this technology. We all have used some driving app on our phone to go from point A to point B. The internal GNSS sensors in our mobile devices allow us to obtain a position on the globe (within a few meters), locate us on a map and provide us navigation guidance to our destination. A few meters are fine for transport applications. But when it comes to 3D measurements, surveyors usually need an absolute accuracy within a few centimeters. Without going to deep into GNSS technology (we keep this for another post) it means that, should they compare the projected coordinates of their point clouds with the absolute coordinates of the actual environment (based on a few physical points used as ground truth), the differences must not be more than a few centimeters (2-5 cm) in all dimensions X,Y and Z.

Now let’s talk real

Well, now that we have the basics down… let’s forget about everything! Indeed, when dealing with 3D scanning and point clouds, there is a catch. The catch is called “noise” (told you we would get back to it…).

When making a measurement with a ruler or similar, the noise is low or even non-existing as the device is passive; namely, it need not produce a signal to generate its measurement. The user completes the measurement without the extra energy used by the device. However, with 3D technology, whether stereo cameras in photogrammetry or LiDAR scanner, the device producing the measurement is active. It uses energy to send out a signal and, only once received, the signal can be analyzed and then used as a measurement input for the user. Therefore, like any electronic instrument, LiDARs (or cameras) have a noise measurement (or more commonly called image resolution for cameras).

Without going too deep into the underlying reasons, we can simply understand that a physical system is never perfect, and therefore its level of quality differs. For a LiDAR, the quality (among other parameters like reflectivity intensity, etc.) is defined by the noise measurement and is dependent on the fineness with which the optical system (lasers emitting and receiving pulses) has been put together. Usually (but not always!), the more expensive the LiDAR, the more carefulness is taken during the manufacturing process and thus the lower the noise of the component.

What does it all mean?

Translating this information into our point clouds applications, it means that when measuring an object in a point cloud, we are always going to have to deal with noise. The typical example is measuring a wall width. If scanned from only one side of the wall, in theory, the wall should be a perfect plane whose width should not be wider than the size of a reflected laser pulse (so a 1-2 millimeters or even less). In reality, this never happens and more likely you are going to end with a cluster of points giving a width of a few millimeters or a few centimeters (depending on the range, type of LiDAR, environmental conditions, etc.) even for static scanners!

Example of noise measurement

Remedies

Can we do something about it? Yes, we can. There are a few things that we can do to help the users making 3D measurements. First, we can potentially reduce the noise of the LiDARs we used by recalibrating them. At VIAMETRIS, we recalibrate ourselves every single LiDAR that we integrate into our end products. And the results are significant, especially at long ranges (top of buildings, etc.). This contributes to improving the relative accuracy of the measurement.

What else can we do to improve the relative accuracy of our point clouds? Yes, we could only integrate very high-end LiDARs into our systems. But unfortunately, they come with other engineering aspects (mostly weight and cost) that are not in adequacy with the needs of the market.

Before filtering

So our last resort, like other mobile mapping companies do is filtering. Filtering consists of removing “extremes” and smoothing the data. Picking up on our last example with the wall, it means that we will remove all the points that are too much above or below the clusters that are the most representative of the wall.

After filtering

Have we improved our relative accuracy? By now, you should confidently be able to say: “no, we did not”. Have we helped some users, drawers and BIM modelers to create their deliverables? Definitely. But beware. We just remove some information, we did not make the raw one more accurate. We know it and we will warn you. Make sure, others do too.

Conclusion

Now that we have understood all the different parameters that can influence the accuracy of our measurements, it is reasonable to assume that reaching certain types of deliverables (2D plans, BIM models, etc.) with a sub-centimeter (real not marketed) accuracy is a highly complex task. So beware of those who promise it! They may deliver a thin point cloud with a millimeter noise but who knows what lies under this filtered reality…?

Categories:

No responses yet

    Leave a Reply

    Your email address will not be published. Required fields are marked *