Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu 24.04 DensifyPoincloud Buffer overflow #1176

Open
jsnetz opened this issue Oct 4, 2024 · 18 comments
Open

Ubuntu 24.04 DensifyPoincloud Buffer overflow #1176

jsnetz opened this issue Oct 4, 2024 · 18 comments

Comments

@jsnetz
Copy link

jsnetz commented Oct 4, 2024

Have a new PC with Ubuntu 24.04, more RAM and the RTX 4060 TI.
after building openMVS 2.3 and testing with the South Park example files i get a Bufffer overflow.
It doesnt matter using --cuda-device 0,-1 or -2. After camera model loaded and loading the images there is an error (line 66 in DensifyPointtCloud_Cuda_0_log.txt or line 60 in DensifyPointtCloud_Cuda_-2_log.txt) These files are terminal output with "-v 4"

[App ] error: does not seem to be a tower: X(0.08), Y(66.69), Z(630.57)

After that it runs until "selecting images and then crashed with Buffer overflow in "cuda-device -2" or no output when using "cuda-device -1" or "cuda-device 0".

So I don't know is it a Problem with Interface COLMAP i build openMVS 2.3 with InterfaceVisualSFM. Now I get a Buffer overflow at while running InterfaceVisualSFM.
With Ubuntu 22.04 everything running well.

Have somebody build openMVS 2.3 under Ubuntu 24.04 without problems? What are the settings building it?
I tried to build version 2.2 and 2.1 but that doesn't work (compile errors)

greetings

joerg

DensifyPointtCloud_Cuda_0_log.txt
DensifyPointtCloud_Cuda_-2_log.txt

@cdcseacave
Copy link
Owner

pls build and run in debug mode

@jsnetz
Copy link
Author

jsnetz commented Oct 4, 2024

Hello cdcseacave

Here the Files
greetings joerg

debug_DensPointCloud_cuda_0.txt
debug_DensPointCloud_cuda-2.txt

@jsnetz
Copy link
Author

jsnetz commented Oct 5, 2024

Hello cdcseacave
good morning,
and here the terminal output cmake configure and build

greetings
joerg

openMVS_cmake_build_out.txt
OpenMVS_cmake_config_out.txt

@cdcseacave
Copy link
Owner

the cmake config and build look fine, but there is a runtime error in cuda, some cuda call fails, but the output does not say which function; this seems a problem that can be reproduced only on your machine, so if you can, pls do a bit of debugging, put a breakpoint in the function that reports the error and see from where is called

@jsnetz
Copy link
Author

jsnetz commented Oct 8, 2024

Hello
Sorry but I'm new what debugging and so on does. So i had to read and look first.
Here I have 3 files, maybe this is what you need. First is the normal output, second the gdb logfile and thrid the Terminal output.
If you need something additional plese give me a liitel help how to create.

Greetings
joerg

DensifyPointCloud-2410081213508B54CB.log
gdb_DensifyPointCloud_20241008.txt
gdb_DensifyPointCloud_terminal_20241008.txt

@cdcseacave
Copy link
Owner

in order to get any meaningful debug info from gdb you need to build the project in Debug or RelWithDebInfo

@jsnetz
Copy link
Author

jsnetz commented Oct 9, 2024

ok, I got these files with with debug build and no other additional settings.

@jsnetz
Copy link
Author

jsnetz commented Oct 15, 2024

A weekend later with testing I had some news. The buffer overflow is gone since I use don't use any power strip. Connecting to the wall solve it.
Bad news it doesn't work under Ubuntu 24.04 with great datas. The South-Park examples working but when I test with over 400 Images (20MP) DensifyPointCloud hang with SIGSEGV. It's the same using or not Cuda.
Logfile attached.

Maybe there are additional dependencies under Ubuntu 24.04 nessassary.

sorry about my english it's a lot of years ago I learned and I wasn't good at school

greetings Joerg

gdb_DPC.log

@cdcseacave
Copy link
Owner

pls try the latest develop branch, I fixed there a string print problem on gcc

@jsnetz
Copy link
Author

jsnetz commented Nov 5, 2024

Hello
My Machine was 2 weeks down, needed new ram-modules. It's the Problem with inter Z790 Boards.

Now its back and I start with a new installation of Ubuntu 24.04. So when installing the additional software is there something I had to look for?
Could you tell me which gcc and g++ Versions is to install? Colmap needs gcc-10 and g++-10 to install since 22.04.
Which Cuda Version and driver do you use? at the moment I have 12.6 installed but I can only install the 550-driver.

and at least a Question from a really beginner: How can I download the develop version?

Lot of Questions but I think to find the problem I need the same software base to search the Problems.
Last Problem I got with gdb was

Thread 16 "DensifyPointClo" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffb2a00000 (LWP 16398)] Download failed: Das Argument ist ungültig. Continuing without source file ./string/../sysdeps/x86_64/multiarch/strlen-avx2.S. __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:76

Greetings

Joerg

@cdcseacave
Copy link
Owner

No special installs, any gcc and cuda version
only make sure to use the develop branch of OpenMVS (see internet how to use git)

@jsnetz
Copy link
Author

jsnetz commented Nov 9, 2024

Hello
do you changed something in source? Cloned this mornig to build a Release but when Running InterfaceCOLMAP I got a crash. Build as Debug and the here the gdb-log and Terminal output.

Yesterday i got a step forward. I found my 2 years old batchfile based on Peter Falkinghams example (on my Mac). He used InterfaceVisualSFM. so I removed the "colmap image_undistorter " part and reactivate the Interface VisualSFM in openmvs. It runs perfect (build Debug). So, I think its a problem with the data comming from colmap to openMVS.

My about 460 images have a size of 5148x3888 pixel. So, I set in colmap image_undistorter --max_image_size=2000 and everything works perfect. a second test run with --max_image_size=4000 runs also perfect. Now i have the following question. Is the colmap setting of --max_image_size in a combination with the amount of images and the image size?

Thanks a lot

joerg

gdb_IFColmap.log
gdb_IFColmap_terminal_out.log

@jsnetz
Copy link
Author

jsnetz commented Nov 24, 2024

Hello
today I installed develop and got a new error from InterfaceCOLMAP. Terminal output and gdb log added
So, i can't test the DensifyPointcloud.

Tested with Colmap example person_hall

Thanks

Joerg

gdb_IFC23_Terminal.txt
gdb_IFC-23.log

@cdcseacave
Copy link
Owner

the colmap scene you are trying to import seems invalid

@jsnetz
Copy link
Author

jsnetz commented Nov 29, 2024

ok, reinstalled colmap and checked batchfile.
InterfaceCOLMAP works.
TestImages Colmap south_park (only 43 images)
But now I get

ASSERTION FAILED: (ISEQUAL(norm(normalMap0(nx)), 1.f)) Norm = 1

after "Selecting images ... completed" on DensifyPointCloud.

20241129_gdb_DPC.log
20241129_gdb_DPC_noCuda_Terminal.txt

@cdcseacave
Copy link
Owner

Build in release mode

@jsnetz
Copy link
Author

jsnetz commented Dec 4, 2024

Hello

first - it works.

build version 2.3 develop Release works but DensifyPointCloud is not so easy to handle.
I've running 2 terminals with htop and nvtop to see what happens.

With a smaller range of images, here Colmaps South Park, it runs perfect.
Testing with a greater Project as Colmaps Person Hall it's not so easy to run

running DPC with --cuda-device 0 it crashed with Cuda Error
running DPC with --cuda-device -1 it crashed with Cuda Error
running DPC without --cuda-device it crashed with Cuda Error

build RelWithDepInfo to get gdb and Terminal output crashed without error message so I tested these settings in Debug mode to see why it breaks at this poing.

I had to set explicit 2--cuda-device -2" to get it working.

All parts of openMVS working without Cuda when cuda-device option is not used. The problem is that DPC without "--cuda-device" settings uses cuda in the first parts include "estimated depth-maps" and it looks this crashed with the Cuda Error. You can see it monitoring with nvtop.

@jsnetz
Copy link
Author

jsnetz commented Jan 15, 2025

After some additional test now some results. Tested under Ubuntu 24.04 with RTX 4060 TI.

Using CUDA 12.4.0, 12.5.x and 12.6.x DensifyPointCloud has a Problem with running out of memory. With CUDA 12.4.1 its much more stable running with --cuda-device -1. Installed Ubuntu without addditional drivers. installed the nvidia-driver-550, the dependenciies and the CUDA 12.4.1 deb local.

The results are not the same as I get under Ubuntu 22.04 but thats another Thing

greetings

Joerg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants