Skip to content

Local manager does not limit the number of running tasks #279

@RaphaelRobidas

Description

@RaphaelRobidas

When running a flow (for example, this one), the manager configuration seems to be read correctly, but the hardware limits are completely disregarded.

I use the following manager.yml file:

qadapters:

- priority: 1
  queue:
      qtype: shell 
      qname: localhost

  job:
      mpi_runner: "mpirun"
      mpi_runner_options: "--bind-to none"
      omp_env: {"OMP_NUM_THREADS": 1}

  limits:
      timelimit: 1:00:00
      max_cores: 1

  hardware:
      num_nodes: 1
      sockets_per_node: 1
      cores_per_socket: 1
      mem_per_node: 1 Gb

When launching a flow with abirun.py FLOW_DIR scheduler, the initial printout contains this, confirming that the specifications have been ready correctly:

AbiPy Manager:
[Qadapter 0]
ShellAdapter:localhost
Hardware:
   num_nodes: 1, sockets_per_node: 1, cores_per_socket: 1, mem_per_node 1024,
{'OMP_NUM_THREADS': 1}
Qadapter selected: 0

However, more than one task will run at once, although only one core should be available. This is very problematic for larger flows where the number of tasks is much greater than the number of cores, causing the machine to crash.

Abipy version: git+https://github.com/abinit/abipy@59e3816168ea11ab90622531948d0dac4cd31df9
OpenMPI version: 4.1.6
OS: Debian Trixie

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions