Dev deeplearning Integration (nnLM and new models)#106
Open
mahmoud1yaser wants to merge 3 commits intomainfrom
Open
Dev deeplearning Integration (nnLM and new models)#106mahmoud1yaser wants to merge 3 commits intomainfrom
mahmoud1yaser wants to merge 3 commits intomainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Notes
This pull request introduces major enhancements to the AFID detection workflow, adding support for a new nnLandmark (nnLM) detection mode, improving configuration flexibility, and optimizing inference for both GPU and CPU environments. The changes enable users to select between three detection strategies (prior-based, sliding-window without prior, and nnLM single-pass), provide more granular control over inference parameters, and restructure the Snakemake rules to support both parallel and sequential inference workflows.
Key changes include:
Detection Mode and Configuration Enhancements:
snakebids.ymlto select detection mode (--detect_with_prior,--detect_without_prior,--detect_with_nnlm), and to configure nnLM-specific parameters (fold, plans, checkpoint, device), as well as sliding-window inference overlap and batch size.afids_inferencesection for per-AFID checkpoint configuration, patch size, device, and overlap.enable_sequential_inferenceconfig option to control whether inference is run sequentially (recommended for GPU) or in parallel (recommended for CPU).Workflow and Rule Structure:
Snakefileto dynamically include the appropriate rules (cnn.smkornnlm.smk) based on the selected detection mode, and to automatically enable sequential inference for GPU.rule allinput and descriptor logic to select the correct FCSV output based on the detection mode, ensuring output files are labeled appropriately.CNN Inference Rule Improvements:
cnn.smkrules to support both sequential (single job for all AFIDs) and parallel (one job per AFID) inference for both prior-based and no-prior detection, including new gather rules to combine per-AFID outputs. [1] [2]pytorch.yaml) and improved parameter passing for model checkpoints and inference settings.Environment and Dependency Updates:
pytorch.yaml) and nnLM (nnlm.yaml), specifying compatible Python and library versions and including the nnLandmark package for nnLM inference. [1] [2]Quality-of-Life Improvements:
These changes make the workflow more flexible, scalable, and ready for integration with the new nnLandmark model while maintaining backward compatibility with existing CNN-based modes.