Possible memory leak in the Positional Tracking module

[rgaignouxdilepix]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues.
• This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

Hello,
I am experiencing unexpected memory increase when using the ZED SDK with the following configuration:

• Positional tracking module: enabled
• Pose tracking mode: GEN_3
• Area memory: disabled
• Mapping module: disabled

When replaying an SVO2 file (1080p, 30fps, with camera movement) or using the camera directly (and moving it), memory usage increases significantly over time:

• Initial memory usage: ~5.4%
• After 10 minutes: ~21.6%

Screenshots showing this progression (using htop) are attached below:

Given this configuration (area memory set to false and mapping module disabled), I would not expect memory to increase, so I'm reporting this as a potential memory leak. Please enlighten me if this memory usage increase is a normal behavior.

Steps to Reproduce

Compile the following code (area_memory_test.cpp):

#include <sl/Camera.hpp>
#include <iostream>

int main(int argc, char **argv)
{
    if (argc < 2)
    {
        std::cerr << "Usage: " << argv[0] << " <svo-file>" << std::endl;
        return EXIT_FAILURE;
    }

    std::string svo_name = argv[1];

    sl::Camera zed;

    sl::InitParameters init_params;
    init_params.input.setFromSVOFile(svo_name.c_str());
    init_params.depth_mode = sl::DEPTH_MODE::NEURAL;
    init_params.coordinate_units = sl::UNIT::METER;

    sl::ERROR_CODE state = zed.open(init_params);

    if (state != sl::ERROR_CODE::SUCCESS)
    {
        std::cout << "Error Opening Camera: " << state << std::endl;
        return EXIT_FAILURE;
    }

    sl::PositionalTrackingParameters tracking_params;
    tracking_params.enable_area_memory = false;
    tracking_params.mode = sl::POSITIONAL_TRACKING_MODE::GEN_3;

    state = zed.enablePositionalTracking(tracking_params);
    if (state != sl::ERROR_CODE::SUCCESS)
    {
        std::cout << "Error Enable Tracking: " << state << std::endl;
        zed.close();
        return EXIT_FAILURE;
    }

    zed.disableSpatialMapping();

    sl::RuntimeParameters runtime_params;

    while (true)
    {
        state = zed.grab(runtime_params);

        if (state == sl::ERROR_CODE::END_OF_SVOFILE_REACHED)
        {
            std::cout << "End of SVO file reached." << std::endl;
            break;
        }

        if (state == sl::ERROR_CODE::SUCCESS)
        {
            sl::Pose pose;
            zed.getPosition(pose, sl::REFERENCE_FRAME::WORLD);

            sl::Translation translation = pose.getTranslation();
            sl::Orientation orientation = pose.getOrientation();
        }
    }

    zed.close();
    return EXIT_SUCCESS;
}

I personally used: g++ area_memory_test.cpp -o area_memory_test -I/usr/local/zed/include -I/usr/local/cuda/include -L/usr/local/zed/lib -L/usr/local/cuda/lib64 -lsl_zed -Wl,-rpath,/usr/local/zed/lib -Wl,-rpath,/usr/local/cuda/lib64

Run it. Preferably use a long SVO file (with the camera moving) to observe the increase.

Open htop and monitor the process memory usage over time.

Expected Result

Memory usage should remain stable and not increase over time.

Actual Result

Memory usage increases over time.

ZED Camera model

ZED2i

Environment

Issue observed on my laptop, as well as on Stereolabs Orin NX 16GB.

OS: Ubuntu 22.04
CPU: Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
GPU: NVIDIA GeForce GTX 1650 Ti
ZED SDK version: issue observed on both 5.0.7 and 5.1.2
ROS version: ROS2 humble

Anything else?

The issue appears to be specific to the pose tracking mode. When set to GEN_1, memory usage remains stable over time.

This behavior was also observed with the ROS2 wrapper, in which i loop the SVO files. I observed that memory increases during the first SVO replay and then stabilizes at that level for all subsequent replays (no further increase on replay 2, 3, etc.).

Also, this behavior is not observed when the camera is not moving (no increase in memory in either case: SVO playback or camera connected to a laptop).

[JujuDel]

Hi @rgaignouxdilepix,

Thanks for the report!
We've just released the 5.2.0 version today. Can you try this one out?

Among other things, it includes quite some improvements to PositionalTracking's GEN_3 mode.

And with our latest tests, we have not seen such behaviors. If the issue persists with 5.2.0, please forward an SVO to us (via the support email) so that we can investigate.

[rgaignouxdilepix]

Hi @JujuDel,

Thanks for your response. I'm still experiencing the issue with ZED SDK 5.2.0. I've sent you an email with an SVO file for reproduction

[JujuDel]

Thank you for the confirmation!

The support team will forward the SVO to the relevant team, and we'll take it from here and come back to you

[Myzhar]

I've sent you an email with an SVO file for reproduction

To what email address have you sent the SVO?

[rgaignouxdilepix]

I sent it to support@stereolabs.com

[rgaignouxdilepix]

Hi @Myzhar , do you have any news on this issue? Were you able to reproduce the memory increase with the file? Thanks!

[JujuDel]

Hi @rgaignouxdilepix,

The behavior you described with enable_area_memory = false has been internally addressed and will be part of the next patch (5.2.1)

The patch is currently running our last validation steps and will be published soon

[JujuDel]

Hi again @rgaignouxdilepix,

5.2.1 has been released (see https://www.stereolabs.com/en-fr/developers/release#521-b35bdf54f33238c75ea27a24a1a21a4e).

Can you confirm your issue is fixed when using it?

[rgaignouxdilepix]

Hi @JujuDel,

Sorry for the late reply. I can confirm that memory usage has been significantly reduced with 5.2.1. I ran a longer test with 5.2.1 and a very long SVO, using the same code I previously provided and tracking the process memory:

Memory does continue to grow slowly over longer sessions, up to 5GB. I was therefore wondering if this is the level of memory usage expected with area_memory=false and GEN_3 mode?

Thank you for your quick response in resolving the issue!

[vls303]

With ZED SDK 5.2.2 and ZED-ROS2-Wrapper 5.2.0, the problem still persists. When Position Tracking is enabled, the frame rate drops to only 5–7 Hz immediately after startup, while the total processing time is already significantly too high from the beginning. At the same time, memory usage is high and continues to increase strongly during operation. This behavior occurs on multiple systems, including Intel-based PCs as well as the Jetson AGX Thor.

When Position Tracking is disabled, the system returns to the expected normal state with a frame rate of 14–15 Hz. Another notable point is the difference compared to the previous version (SDK 5.1.1 / Wrapper 5.1.0): in that version, the system also started at 14–15 Hz with Position Tracking enabled, before the frame rate reproducibly dropped to 5–7 Hz after about one hour. With the current version, this reduced frame rate now occurs immediately after startup. No changes were made other than updating the SDK and wrapper.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days