PhotoScan software, photogrammetry, and 3D modeling

[Charles Walbridge]

I've been following the progress our colleagues at the O'Keeffe Museum are making with RTI and photogrammetry, and as a separate project I've been learning about 3D scanning for art objects. Both paths have led me to the PhotoScan software, which looks as capable as anything for both photogrammetry and 3D modeling.

I'm thinking CHI has recommended PhotoScan to the O'Keeffe Museum, but I'm not seeing anything about it here in the forum: are there many cultural heritage organizations using it for photogrammetry and 3D scanning?

And is there something better for 3D scanning of art objects, assuming I want to start with photographs and not lasers?

And can I get by with the $200 version of the PhotoScan software, or should I budget for the $3500 version?

 

Useful links:

The O'Keeffe's blog about RTI and photogrammetry:

http://okeeffeimagingproject.wordpress.com/daily-documenting/video/

 

3D recording of archaeological remains, processed with PhotoScan:

http://www.academia.edu/1922635/Three-dimensional_recording_of_archaeological_remains_in_the_Altai_Mountains

 

From Agisoft's website, making a 3D model from a statue tutorial:

http://www.agisoft.ru/tutorials/photoscan/04/

 

 

[marlin]

hi Charles!

 

thanks for the post.

 

We recently upgraded from the AgiSoft standard edition to the 'professional' version. We ended up purchasing the 'Edu' license and paid 549 for the license. So far so good, although we have only recently started to send data into that package.

 

I'm not sure if CHI actually submitted a formal recommendation to use the AgiSoft package, but recently, when Mark and Carla were down there consulting with O'Keeffe Museum, I am certain that the subject presented itself. So perhaps this was when the knowledge passed hands.

 

I do know that Tom Noble, (USDI, Bureau of Land Management), has his finger on the pulse of the AgiSoft Photoscan Pro version — and gives it a thumbs up! And we listen to Tom, because he's an expert and professional.

 

His co worker Neffra Mathews, recently published "Tech Note 428: Aerial and Close-Range Photogrammetric Technology: Providing Resource Documentation, Interpretation, and Preservation". Neffra, is also a BLM photogrammetry expert. You can check out the technote here:

 

big PDF:

http://www.blm.gov/pgdata/etc/medialib/blm/wo/Planning_and_Renewable_Resources/coop_agencies/paleontology_library/paleo_publications.Par.69972.File.dat/TechNote428.pdf

 

I dont have much personal experience with laser scanners at this moment. But here's an interesting workflow from the NCPTT 3D Summit held in SF last year. This is a transcript of the presentation, its slightly harder to follow bc the slides are not included, but this is a comparison of laser vs photogrammetry. PhotoScan is part of Tom's workflow — using a simple Nikon DLSR and a 28mm lens getting very good results, just as good as the expensive laser gear. Its worth a look.

 

"A Comparative Study Using LiDAR Digital Scanning and Photogrammetry"

https://ncptt.nps.gov/blog/a-comparative-study-using-lidar-digital-scanning-and-photogrammetry/

 

cheers.

marlin.

[macsurveyr]

Hello Charles,

 

PhotoScan has come a long way in a couple of years. I have been encouraging them to improve their camera calibration and they have been steadily making strides. My most recent testing of the latest beta shows they are very very close. I do need to check a few more things and also see what is in the next actual release.

 

While the standard edition is very capable and can produce good looking models, only the professional version allows the camera calibration to be refined and the results improved - something that I feel is important. The professional version also adds the ability to scale the model and with more than one scale, get statistics on how good the solution is - something I also think is important.

 

I have done quit a few, fairly large subjects - rock art panel, rock art sites and others - I have only done a few smaller objects, some using a lazy susan just turning by hand. Here is a 3D pdf showing one capture

 

https://www.dropbox.com/s/pbh1wtpnvrfgrcc/Pitcher_tex.pdf

 

You must actually download and open the pdf in Abobe Reader to see the 3D content. I think the RGB texture being an integral part of the solution itself is a huge benefit to using photogrammetry. There are always going to challenges with complex subjects due to occlusion and limited depth of field but there are usually ways around many of those issues.

 

Tom

[GeorgeBevan]

Tom, 

 

How does the latest version of PhotoScan work with GPS data embedded in EXIF files from geotaggers? Is it able to use this lower quality GPS data to scale and orient the models? ADAM has really shied away from using this sort of control data, although I understand the new version will allow the integration of GPS data from UAVs. 

 

We're just starting to use a really nice geotagger for our work (Solmeta Geotagger Pro 2). The triple axis compass is very handy! 

 

George

[macsurveyr]

George,

 

PhotoScan reads the EXIF data automatically. It will scale and orient the model but since it is usually very low quality, you have to be careful using it when optimizing. CalibCam holds the photo alignment much more rigid even with bad control - assuming you assign a very loose error estimate. The latest version will import it from EXIF, but you can also import the camera positions from a text file. Actually both will do that quite easily.

 

Tom

[GeorgeBevan]

Interesting! So you've got decent orientation + scaling from GPS EXIF data? I've had lots of trouble even with a big sigma to get usable results in CalibCam. I'll be very interested to see how PhotoScan deals with control point data.  

[macsurveyr]

Only decent enough to put you in the right place - more or less - to assist in finding real control points if you have them. Still need some objects of known length or good control points to get a final, correctly scaled result.

 

PhotoScan deals with control point data very well. That does not always help me as I usually "survey" my own control with CalibCam and hopefully be able to do so with PhotoScan. Close to being there.

 

Tom

[GeorgeBevan]

So you're exporting the relative-only bundle points from CalibCam into PhotoScan? How much low-level access does PhotoScan give you to the matching points? I've only worked with PhotoScan briefly for some underwater work. It didn't give me the level of control I needed when working with low quality imagery, although I know others have used it with some success.

[macsurveyr]

I am exporting points on targets from CalibCam and using as control in PhotoScan. Prior to the last couple of beta versions, PhotoScan did not, in my opinion, optimize the camera without having high quality control points. As I have mentioned, that is something that is changing but I would like to test some more.

 

It is disconcerting to not have the level of control over point matching and picking points that you have with CalibCam. I am getting over it. While it is true that with low quality, marginal imagery, PhotoScan may fail to align and there is not much you can do about it - a couple of things that are not entirely obvious - but not much. It either works or it doesn't. Like I said, I am getting over it, as I have found that if it fails, it will be very difficult and time consuming to even get CalibCam to work with the images.

[marlin]

Really interesting thread!

 

I'd like like to chime in about the GPS -> EXIF data aspect of this topic. I'm interested in this EXIF / GPS related data because I want my photogrammetry input data to be as accurate as possible when imported into PhotoScan Pro and its ground control matrix.

 

Currently, the Canon 6D has a built in GPS functionality, but Canon states that the factory default (in-camera-gps) is only accurate to 98 feet (30m). (What's the accuracy of other GPS cameras on the market? hhmmmm.)

 

However, Canon also sells a separate device called the "GP-E2 GPS receiver" that gets a 'superior', far better result.

 

"When in use, the GPS Receiver GP-E2* will automatically record the geographical information (latitude, longitude, altitude, and Universal Coordinated Time (UTC)) of videos and photos. Detecting altitude is even possible from inside an airplane or helicopter."

(George — I have to wonder if this device will also tell you feet/meters below sea level?)

 

http://www.usa.canon.com/cusa/consumer/products/cameras/gps_receivers/gps_receiver_gp_e2#Features

 

Data from the GP-E2 can also be exported to a KZM file which might make it even more flexible for import into numerous apps in addition to PhotoScan.

 

Info about the 6D GPS:

http://www.learn.usa.canon.com/resources/articles/2012/eos6d_inside_gps.shtml

 

Does anyone have experience using the GP-E2? (and the Canon 6D and PhotoScan?) Comments?

 

thanks! Good thread!

 

Marlin.

[GeorgeBevan]

I can't speak to the Canon product, but I can say something a bit more general (I'm sure Tom will want to chime in on this). The best handheld GPS units with a 100ms clock will get you to within 3m horizontally. Height accuracy with GPS is considerably inferior (4.5 m or more; 1.5 x the horizontal accuracy). All of the geotaggers for cameras on the market essentially use these chip-sets. By surveying standards this is not very good. Generally though, geotaggers are becoming a must-have for those of us in the field. It provides great, "free" meta-data on how the photos were shot. As a nice bonus, the geotagger we're using also functions as a kick-ass shutter release for RTI

 

The problem Tom and I were discussing is how this low quality, absolute GPS positioning data can be used in high accuracy, close-range photogrammetry. It can certainly be used to put the models in the right place in the world, but can it be used to scale and orient the models accurately? At present, if I want to scale a model I can use scale sticks, or I can use a total station of differentially corrected GPS (accurate to 1cm horizontally) to put in four or more control points into the scene. Getting correct orientation on photogrammetry models without surveyed points gets a little bit trickier...the scale bars need to be aligned using a spirit level so that they are perfectly in line with the x and y axes of the model. In the software one then stipulates that the scale bars, as well as providing scale, also correspond with the  x and y axes (the z axis is then constrained to a unique direction). At least this is the way I deal with the issue in ADAM CalibCam...PhotoScan appears to handle camera location and pose data in a bit more relaxed fashion.  

 

This question is one I've been wrestling with viz. laser scanning. LiDAR always knows which way is down and what the absolutely scale is without adding anything to the scene being scanned. It would be ideal if a geotagging device added to a DSLR could provide high accuracy position and camera pose information to solve this problem.  For most close-range work, like rock art, precise absolute orientation isn't going to matter that much (scaling is more of an issue). Personally I'm more concerned about cases where I want to measure how far, say, a distant tall monument was leaning using photogrammetry. If I can't access the scene, I can't put in place the ground-control points to establish verticality. 

 

I've certainly been inspired by Tom's recent experience with PhotoScan to reprocess some of my data with ground control. How PhotoScan will deal with GPS data from UAVs with no surveyed GCPs is an urgent concern of mine for an upcoming project. 

[macsurveyr]

Even the best survey quality GPS systems - high quality antennas and multiple frequencies and all available satellites - are only capable of 2 mm xyz positions. I say only - that is incredibly good - but only possible with high quality "stuff" through out the entire process. Cameras or any add-on camera GPS will never achieve such accuracy  - I say never -  and that is a really long time - but I will still say it. It is very difficult to achieve those accuracies and at this time, it is not possible to get with the components that are available for UAVs. Someday? Maybe. Soon? Hmmm, well define soon, but I think still a while away. Inside buildings? Not with current frequencies - that is changing with new satellites and frequencies. Not sure when full deployment - have not really followed lately - perhaps others have? It won't be accurate enough either. Under water? No.

 

The bottom line is, I can survey using photogrammetry to better than the size of the pixels of the photos I take to document the subject -  I must have accurate scale to do so. Objects of known length, in good stereo, work the best - are the most accurate - but good, consistent, high quality control will work. Having some points in real world coordinates are always useful but they might not do it all - it just depends on how many, how consistent, how good.

 

We are working on some techniques to provide scale from UAVs with or without GPS, but need more testing. There is only so much time.

 

Tom

[GeorgeBevan]

It's an enormous frustration of mine that the "Open GPS" projects, GPL-GPS and GRINGO, of the early 2000s seem to have expired (http://gps.psas.pdx.edu/ and http://www.helenav.nl/). It isn't even possible to get a RINEX file (the raw data from the satellites) out of a hand-held to reprocess, although the technology would certainly allow it. With long enough measurement times, it is possible to get under a metre with a single frequency (http://www.geod.nrcan.gc.ca/products-produits/images/p1a_4_e.jpg) The publication from the GRINGO project in 2001  presented some preliminary results that it was possible to get decimetre accuracy from a 12 channel carrier phase GPS with software reprocessing of a measurement taken for as short as 30 mins: http://www.ingentaconnect.com/content/maney/sre/2001/00000036/00000280/art00006

 

It would be fantastic if we could reprocess the GPS data from a geotagger over the course of a long gigpan...sadly I don't think that will ever happen. Trimble and the others have the whole field sewn up tight. 

[Charles Walbridge]

Thanks for the links and info, guys. The NCPTT video is very informative. I'm going to find a project that needs PhotoScan.

 

Marlin, I've been making my RTIs with two spheres in the frame so the Builder can someday build me a 3D model from those same data sets; has the Builder progressed to that stage yet? 

 

And while I'm here on the forum, I'm going to go look for progress on the RTI to iPad front...

[GeorgeBevan]

Charles. I've seen PTMs viewed on an iPhone but specular enhancement is still slow! Last time I checked the programmers were still optimizing the code. We're hoping for a YouTube demo in a few weeks. I wouldn't be at all surprised if there are other groups doing this as well...they might be further ahead.

 

Is a PTM viewer for iPad something you'd be interested in for a museum display?  

[Charles Walbridge]

George, I think we here at the Minneapolis Institute of Arts could make good use of an RTI viewer for iPad. I've made RTIs of saints in tempera and gold on panel like St. Sirus here - https://collections.artsmia.org/index.php?page=detail&id=1610 - that show the artist's techniques better than you can see them with the naked eye.