Updates.

Author: pichuan

README.md

@@ -1,9 +1,52 @@
 # DeepConsensus
 
-DeepConsensus uses deep learning for correcting errors in  Pacific Biosciences
-(PacBio) Circular Consensus Sequencing data.
+DeepConsensus uses gap-aware sequence transformers to correct errors in Pacific
+Biosciences (PacBio) Circular Consensus Sequencing (CCS) data.
 
-See: https://github.com/google/deepconsensus
+![DeepConsensus overview diagram](docs/images/pipeline_figure.png)
+
+## Installation
+
+### From pip package
+
+```bash
+pip install deepconsensus==0.1.0
+```
+
+You can ignore errors regarding google-nucleus installation, such as `ERROR:
+Failed building wheel for google-nucleus`.
+
+### From source
+
+```bash
+git clone https://github.com/google/deepconsensus.git
+cd deepconsensus
+source install.sh
+```
+
+(Optional) After `source install.sh`, if you want to run all unit tests, you can
+do:
+
+```bash
+./run_all_tests.sh
+```
+
+## Usage
+
+See the [quick start](docs/quick_start.md).
+
+## Where does DeepConsensus fit into my pipeline?
+
+After a PacBio sequencing run, DeepConsensus is meant to be run on the CCS reads
+and subreads to create new corrected reads in FASTQ format that can take the
+place of the CCS reads for downstream analyses.
+
+See the [quick start](docs/quick_start.md) for an example of inputs and outputs.
+
+NOTE: This initial release of DeepConsensus (v0.1) is not yet optimized for
+speed, and only runs on CPUs. We anticipate this version to be too slow for many
+uses. We are now prioritizing speed improvements, which we anticipate can
+achieve acceptable runtimes.
 
 ## Disclaimer
 

deepconsensus/scripts/run_deepconsensus.py

@@ -126,7 +126,6 @@ def create_all_commands(directories: List[str], input_subreads_aligned: str,
   --out_dir={directories[3]} \
   --checkpoint_path={checkpoint} \
   --inference=true \
-  --params=deepconsensus/models/model_configs.py:transformer_learn_values+ccs \
   --max_passes=20'
 
   command5 = f'python3 -m deepconsensus.postprocess.stitch_predictions \

deepconsensus/scripts/run_deepconsensus_test.py

@@ -57,7 +57,7 @@
 
 
 ***** DRY-RUN ONLY:*****
-python3 -m deepconsensus.models.model_inference_with_beam   --dataset_path=output_directory/3_write_tf_examples/inference   --out_dir=output_directory/4_model_inference_with_beam   --checkpoint_path=checkpoint   --inference=true   --params=deepconsensus/models/model_configs.py:transformer_learn_values+ccs   --max_passes=20
+python3 -m deepconsensus.models.model_inference_with_beam   --dataset_path=output_directory/3_write_tf_examples/inference   --out_dir=output_directory/4_model_inference_with_beam   --checkpoint_path=checkpoint   --inference=true   --max_passes=20
 
 
 ***** DRY-RUN ONLY:*****

docs/images/pipeline_figure.png

@@ -13,12 +13,17 @@ Memory: 126G
 
 ## Download data for testing
 
-This will download about 162 MB of data and the model is another 244 MB.
+This will download about 252 MB of data and the model is another 245 MB.
 
 ```bash
 # Set directory where inputs will be placed.
-INPUTS="${HOME}/deepconsensus_quick_start/inputs"
+QUICKSTART_DIRECTORY="${HOME}/deepconsensus_quick_start/"
+# This will soon have 3 subfolders: inputs, model, and outputs.
+
+INPUTS="${QUICKSTART_DIRECTORY}/inputs"
+MODEL_DIR="${QUICKSTART_DIRECTORY}/model"
 mkdir -p "${INPUTS}"
+mkdir -p "${MODEL_DIR}"
 
 # Download the input data which is PacBio subreads and CCS reads.
 gsutil cp gs://brain-genomics-public/research/deepconsensus/quickstart/v0.1/subreads.bam "${INPUTS}"/
@@ -28,7 +33,7 @@ gsutil cp gs://brain-genomics-public/research/deepconsensus/quickstart/v0.1/ccs.
 # gsutil cp gs://brain-genomics-public/research/deepconsensus/quickstart/v0.1/subreads_to_ccs.bam "${INPUTS}"/
 
 # Download DeepConsensus model.
-gsutil cp gs://brain-genomics-public/research/deepconsensus/models/v0.1/checkpoint-50* "${INPUTS}"/
+gsutil cp gs://brain-genomics-public/research/deepconsensus/models/v0.1/* "${MODEL_DIR}"/
 ```
 
 ## Prepare input files
@@ -51,34 +56,16 @@ samtools view -h "aligned.subreads.bam" | \
 
 ## Install DeepConsensus
 
-First go to a parent directory where you want to install DeepConsensus, then
-follow the steps below to install DeepConsensus.
-
-```bash
-git clone https://github.com/google/deepconsensus.git
-cd deepconsensus
-source install.sh
-```
-
-You can ignore errors regarding google-nucleus installation, such as:
-
-```
-  ERROR: Failed building wheel for google-nucleus
-```
-
-(Optional) After `source install.sh`, if you want to run all unit tests, you can
-do:
-
 ```bash
-./run_all_tests.sh
+pip install deepconsensus==0.1.0
 ```
 
 ## Run DeepConsensus
 
 ```bash
 # Set directory where outputs will be placed and set the model for DeepConsensus
-OUTPUTS="${HOME}/deepconsensus_quick_start/outputs"
-CHECKPOINT_PATH=${INPUTS}/checkpoint-50
+OUTPUTS="${QUICKSTART_DIRECTORY}/outputs"
+CHECKPOINT_PATH=${MODEL_DIR}/checkpoint-50
 
 # Run DeepConsensus
 python3 -m deepconsensus.scripts.run_deepconsensus \

docs/quick_start.md

@@ -34,13 +34,16 @@
 """
 
 import pathlib
+from setuptools import find_packages
 from setuptools import setup
 
 here = pathlib.Path(__file__).parent.resolve()
 
 # Get the long description from the README file
 long_description = (here / 'README.md').read_text(encoding='utf-8')
 
+REQUIREMENTS = (here / 'requirements.txt').read_text().splitlines()
+
 setup(
     # To support installation via
     #
@@ -50,11 +53,11 @@
     description='DeepConsensus',
     long_description=long_description,
     long_description_content_type='text/markdown',
-    url='TODO',
+    url='https://github.com/google/deepconsensus',
     author='Google LLC',
-    keywords='TODO',
-    packages=['deepconsensus'],
+    keywords='bioinformatics',
+    packages=find_packages(where='.'),
     package_dir={'deepconsensus': 'deepconsensus'},
-    python_requires='==3.6',
-    install_requires=[],
+    python_requires='>=3.6,<3.7',
+    install_requires=REQUIREMENTS,
 )