KurtH Posted July 12 Report Share Posted July 12 I have a model that is causing me a number of headaches. It's something I shot before thinking about Kintsugi but I captured polarized and non-polarized ring-flash images. Additionally it was shot on a turntable but with a square wood platform (not built as part of the model) that sticks out and occludes part of the sculpture for the circuits with the lowest angles. Additionally this object we were not able to capture the bottom so I filled the hole and brought it back into Agisoft. To make sure the issues I've been having weren't due to me somehow mis-aligning the model while hole-filling, I re-decimated the model in agisoft, and built a new set of textures in agisoft to confirm the images are aligned in Agisoft. I then exported the model, cameras (only the non-polarized ones), and triple-checked that I grabbed the folder of non-polarized JPGs that were used in this last round of Agisoft. I am unsure where the issue is coming from but wondering if the occlusion (that is masked out in Agisoft) combined with the hole filling is causing problems. Attached is an image with one view showing the wood platform that causes the occlusions. Attached is an image showing the object on the turntable with the platform occluding part of the object along with two screen captures in the initial state in Kintsugi showing the images misaligned (proceeding anyway and trying to render textures does not improve things, I tried incase it was just an issue with the initial render). I can package up the images, object, and cameras. If you want the whole agisoft project, I can probably do that but it might be next week so I can get a colleague to send it via a file transfer service. Quote Link to comment Share on other sites More sharing options...
Michael Tetzlaff Posted July 12 Report Share Posted July 12 If you could send be the files, Kurt, that would be helpful. My best guess is that this might have something to do with the lens distortion parameters that Metashape estimates. The images, model, and camera XML file should be enough, although if I don't get a lead for what is wrong from that, the next step would be to look at the Metashape project. Quote Link to comment Share on other sites More sharing options...
Charles Walbridge Posted July 15 Report Share Posted July 15 Kurt, a general question that I don't think is the problem here: is Kintsugi using the masked images as its source images? (As a review for this Forum post) You would have to export new images from Metashape using the masks that you made via Mask From Model; then you'd need to rename the images to match the image names in the XML file that specifies the camera locations; and for that you may need to take the exported PNGs with their masks and save them as TIFs or JPEGs, whatever your source images are. I don't know if Kintsugi 3D Builder sees alpha-channel masks, so that we'll put that on the list of things to test. But in the near future, Kintsugi 3D Builder will be able to read Metashape PSX files directly - we're testing that now - but I don't know if Builder is going to be aware of any masks if the masks aren't part of the source images (instead of the masks being stored in Metashape). We'll talk more and test this. But as I mentioned, with the example image you've shared here, I don't think that's what's happening in this case. I'd like to see that data too. Quote Link to comment Share on other sites More sharing options...
KurtH Posted July 16 Author Report Share Posted July 16 Thanks for the idea Charles. I ran all the image masks to create JPGs with the object floating in white space to see how it would behave, and the images were still mis-aligned. When I have a moment I'm going to completely re-align the project and see what I can get and probably run the model through at a few stages... 1) Straight out of metashape, holes and all. 2) After MeshLab cleaning and rebuilding textures in Metashape. 3) Bottom hole filled in blender and retextured in Metashape. Knowing at what stage it breaks will give more clues to where the problem is. Quote Link to comment Share on other sites More sharing options...
Michael Tetzlaff Posted July 17 Report Share Posted July 17 Regarding masks: Kintsugi should see alpha channel masks. We could also add the ability to import masks stored as separate files -- that feature was in the code a long time ago and got removed in a UI update (predating even the Kintsugi development push I think). If we do that, it should be able to handle different file extensions - TIF vs JPEG vs PNG. We should look into also importing the masks from the PSX. Let me know if you discover anything, Kurt. I'm on vacation for a couple weeks with my family, so I'm not sure when I'll get around to looking at your bugged dataset -- it is one of my top priorities when I find some time to work on Kintsugi. Quote Link to comment Share on other sites More sharing options...
Charles Walbridge Posted July 19 Report Share Posted July 19 @KurtH , this week I had some image alignment issues that I investigated and solved, and it led me back to a mistake (or a 'variation in the workflow') that I had made early in the process. Our normal turntable photogrammetry workflow is to use Canon CR2 or CR3 raw files in the capture process, then add data in Adobe Bridge and process the images to DNG and full-resolution (but sometimes cropped) JPEGs in Adobe Camera Raw. We use lightly-compressed JPEGs in our process because we used to transfer Metashape projects over our networks and the JPEGs are about a third as big as the equivalent TIFFs. I don't think it makes Metashape processing any faster. With our turntable photogrammetry process, I often crop the source images before I put them into a Metashape project, because there are a lot of unused pixels in the source images - but that's not the way CHI teaches, so don't do that. (But let me know if you want to see a thread about that, because I want to talk to Carla about it some more.) I have a Metashape project that's the lid of a shiny silver censer. I had good alignment and mesh-building in Metashape but of course I want the improved color, specularity, and normal maps from Kintsugi 3D Builder. When I ran the model through the Kintsugi 3D Builder process the images didn't align properly, and it looks like the offset that you've noted in your photos here. I could see the misalignment in both in the early Kintsugi workflow and in the built textures. Looking back at the whole Metashape project I realized I had mistakenly used the DNGs as the source images and not the cropped JPEGs. I could see in the image information in Metashape that Metashape doesn't see the crop that you've put on the DNG - and I've since learned from Carla that Metashape also doesn't use the color balance or any other adjustments that you might make to a DNG in Camera Raw - it just loads the TIFF that's stored in the DNG (and I assume that TIFF is like the quick preview JPEG that a camera makes right when it takes a photo). When I asked Kintsugi 3D Builder to make textures, it used the cropped JPEGs, and the images didn't align. But I went back and made new uncropped JPEGs with the same filenames and used those, and now everything has aligned well. I don't know if that's the issue you're having with the model you show here, but maybe there's something like that going on. Let us know if any of this helps -- 1 Quote Link to comment Share on other sites More sharing options...
KurtH Posted August 19 Author Report Share Posted August 19 Had another model where the images were not being aligned properly. This was using the newer version Kintsugi builder and the Metashape import. I THINK the problem was that the Metashape project had the JPG files spread across multiple folders (one for 24mm on a turn table, one for 50mm on a turntable, one for 24mm with the object lying down to get the bottom, one at 50mm for the bottom). I copied all the JPGs to one folder and used the manual OBJ/Cameras.XML/JPG folder and then it worked fine. Quote Link to comment Share on other sites More sharing options...
Michael Tetzlaff Posted August 19 Report Share Posted August 19 Thanks, Kurt. In the original Metashape project, were there images with the same name and folder but different file extensions? (i.e. IMAGE0001.png and IMAGE0001.jpg in the same directory)? That's my best guess as to the source of the issue -- if you had two capture sessions differentiated only by file extension in the same folder, Kintsugi strips the file extension and tries all recognized extensions in a predetermined (arbitrary) order. Quote Link to comment Share on other sites More sharing options...
KurtH Posted August 19 Author Report Share Posted August 19 The files all had unique names that denoted if it was on the side or circuits (or details) from a turntable set. Only JPGs were used in this model, no PNGs or TIFFs. The only thing I could think of is the Agisoft file was referring to the JPGs spread across 4 different folders. Knowing I could only reference one JPG folder when using the Camera.XML/Model.OBJ manual set up, I copied all the JPGs to one folder before using the manual mode, and then it worked fine. 17661-20240726-24mmOCF-OnesideCalSet-001.jpg 17661-20240726-24mmOCF-c01-01.jpg 17661-20240726-50mmOCF-OnesideCalSet-001.jpg 17661-20240726-50mmOCF-c01-01.jpg Quote Link to comment Share on other sites More sharing options...
Michael Tetzlaff Posted August 19 Report Share Posted August 19 Interesting -- if you are willing to zip up the project that's causing problems and send it to me, I or one of my students could look into what's causing the problem for the full project import method. And I still need to look at the other dataset you sent me assuming you didn't figure out what the problem was there. It's been on my to-do list but just haven't had the time. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.