.. _configfiles:

Config file format
==================

Each time you run a MARTi analysis on a sequencing run, you need to specify a config file which provides the details of the analysis to be performed.

This config file is generated by the MARTi launcher front-end (Desktop) or GUI (cluster/HPC).

The following table specifies the meaning of the parameters in the file. Keywords in **bold** are mandatory, others are optional.

Sample and global settings
--------------------------

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table1.csv
   :delim: tab

Pre-filtering settings
----------------------

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table2.csv
   :delim: tab

LCA classification settings
---------------------------

These Lowest Common Ancestor settings apply to BLAST results (see below).

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table3.csv
   :delim: tab

BLAST processes
---------------

You can run multiple BLAST processes. Each begins with the Keyword BlastProcess.

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table4.csv
   :delim: tab

Diamond processes
-----------------

Diamond can be used to classify reads against a Diamond database that is built with taxonomy information. For example, to build a compatible diamond database using NCBI taxonomy, use the command

``diamond makedb --threads 8 --in nr.gz -d nr.diamond-2.0.9 --taxonmap prot.accession2taxid.FULL.gz --taxonnodes nodes.dmp --taxonnames names.dmp``

The fields ``--taxonmap``, ``--taxonnodes``, and ``--taxonnames`` must be specified for the database to be compatible with MARTi.

Diamond processes are a subset of BLAST processes, with the ``Program`` field set to ``diamond``. All compatible fields from the BLAST process are passed through to Diamond. Diamond processes have an additional ``options`` field to specify the sensistivity mode (or any other options). See below for an example.

Centrifuge processes
--------------------

You can run multiple Centrifuge processes. Each begins with the keyword CentrifugeProcess.

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table_centrifuge.csv
   :delim: tab

Kraken2 processes
-----------------

You can run multiple Kraken2 processes. Each begins with the keywork Kraken2Process.

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table_kraken2.csv
   :delim: tab

Metadata
---------------

Metadata blocks are optional blocks that contain data describing the collection of samples. A metadata block could describe the whole run or a subset of barcodes.

.. csv-table::
   :header: "Keyword", "Example", "Meaning"
   :file: table5.csv
   :delim: tab

Example
-------

Example file::

 SampleName:BAMBI_1D_19092017_MARTi
 RawDataDir:/Users/leggettr/Documents/Datasets/BAMBI_1D_19092017_MARTi
 SampleDir:/Users/leggettr/Documents/Projects/MARTiTest/BAMBI_1D_19092017_MARTi
 ProcessBarcodes:
 BarcodeId1:SampleNameHere
 
 Scheduler:local
 LocalSchedulerMaxJobs:4
 
 InactivityTimeout:10
 StopProcessingAfter:50000000
 
 TaxonomyDir:/Users/leggettr/Documents/Databases/taxonomy_6Jul20
 LCAMaxHits:20
 LCAScorePercent:90
 LCAMinIdentity:60
 LCAMinQueryCoverage:0
 LCAMinCombinedScore:0
 LCAMinLength:50
 
 ConvertFastQ 

 ReadsPerBlast:8000
 
 ReadFilterMinQ:9
 ReadFilterMinLength:500
 
 BlastProcess
     Name:nt
     Program:megablast
     Database:/Users/leggettr/Documents/Databases/nt_30Jan2020_v5/nt
     TaxaFilter:/Users/leggettr/Documents/Datasets/bacteria_viruses.txt
     MaxE:0.001
     MaxTargetSeqs:25
     BlastThreads:4
     UseToClassify
 
 BlastProcess
     Name:card
     Program:blastn
     Database:/Users/leggettr/Documents/Databases/card/nucleotide_fasta_protein_homolog_model.fasta
     MaxE:0.001
     MaxTargetSeqs:100
     BlastThreads:1
 
 Metadata
     Location:52.62170,1.21900
     Date:31/10/23
     Time: 11:41
     Temperature:21.7C
     Humidity:49%
     Keywords:bambi


Different classification processes can be performed in the same MARTi process (but only one classification process can have the "UseToClassify" field). The example below shows a config file that classifies reads using Kraken2, and searches for AMR hits using BLAST and the CARD database. Note that if a BLAST/CARD process is used, a walkout analysis giving the putative host taxa for AMR genes is only performed if a BLAST process is used to classify the reads. ::

 SampleName:BAMBI_1D_19092017_MARTi
 RawDataDir:/Users/leggettr/Documents/Datasets/BAMBI_1D_19092017_MARTi
 SampleDir:/Users/leggettr/Documents/Projects/MARTiTest/BAMBI_1D_19092017_MARTi
 ProcessBarcodes:
 BarcodeId1:SampleNameHere
 
 Scheduler:local
 LocalSchedulerMaxJobs:4
 
 InactivityTimeout:10
 StopProcessingAfter:50000000
 
 TaxonomyDir:/Users/leggettr/Documents/Databases/taxonomy_6Jul20
 LCAMaxHits:20
 LCAScorePercent:90
 LCAMinIdentity:60
 LCAMinQueryCoverage:0
 LCAMinCombinedScore:0
 LCAMinLength:50
 
 ConvertFastQ 

 ReadsPerBlast:8000
 
 ReadFilterMinQ:9
 ReadFilterMinLength:500
 
 Kraken2Process
     Name:refseq_16
     Database:/Users/leggettr/Documents/Databases/kraken2/k2_standard_16gb_20231009/
     Kraken2Threads:4
     UseToClassify
 
 BlastProcess
     Name:card
     Program:blastn
     Database:/Users/leggettr/Documents/Databases/card/nucleotide_fasta_protein_homolog_model.fasta
     MaxE:0.001
     MaxTargetSeqs:100
     BlastThreads:1


To classify using Diamond and a compatible database, use a ``BlastProcess`` with the ``Program`` field set to ``diamond``. For example ::

 SampleName:BAMBI_1D_19092017_MARTi
 RawDataDir:/Users/leggettr/Documents/Datasets/BAMBI_1D_19092017_MARTi
 SampleDir:/Users/leggettr/Documents/Projects/MARTiTest/BAMBI_1D_19092017_MARTi
 ProcessBarcodes:
 BarcodeId1:SampleNameHere
 
 Scheduler:local
 LocalSchedulerMaxJobs:4
 
 InactivityTimeout:10
 StopProcessingAfter:50000000
 
 TaxonomyDir:/Users/leggettr/Documents/Databases/taxonomy_6Jul20
 LCAMaxHits:20
 LCAScorePercent:90
 LCAMinIdentity:60
 LCAMinQueryCoverage:0
 LCAMinCombinedScore:0
 LCAMinLength:50
 
 ConvertFastQ 

 ReadsPerBlast:8000
 
 ReadFilterMinQ:9
 ReadFilterMinLength:500
 
 BlastProcess
     Name:diamond-nr
     Program:diamond
     Database:/Users/leggettr/Documents/Databases/diamond/nr.diamond-2.0.9
     MaxE:0.001
     MaxTargetSeqs:100
     BlastThreads:2
     options: --sensitive --range-culling



Processing Barcodes Example
----------------------------

The following example demonstrates how to configure MARTi to process multiple barcodes.

::

   RunName:Sample_Name
   RawDataDir:/path/to/data/reads
   SampleDir:/path/to/marti_output/Sample_Name

   ProcessBarcodes:01,02,03,04,05,06,07,08,09,10,11,12
   BarcodeId1:Kessingland1
   BarcodeId2:Kessingland2
   BarcodeId3:CarltonMarshes1
   BarcodeId4:CarltonMarshe2
   BarcodeId5:ThetfordForest1
   BarcodeId6:ThetfordForest2
   BarcodeId7:CityCentre1
   BarcodeId8:CityCentre2
   BarcodeId9:Brancaster1
   BarcodeId10:Brancaster2
   BarcodeId11:FoxleyWood1
   BarcodeId12:FoxleyWood2

   Scheduler:local
   MaxJobs:64
   InactivityTimeout:10
   StopProcessingAfter:0
   TaxonomyDir:/path/to/databases/taxonomy/taxdump_2024_03_09
   ReadFilterMinQ:8
   ReadFilterMinLength:150
   ConvertFastQ
   ReadsPerBlast:10000

   BlastProcess
   Name:nt
   Program:megablast
   Database:/path/to/databases/blast/ncbi/nt_20240305/nt
   NegativeTaxaFilter:/path/to/results/marti/exclude/other_sequences_taxids.txt
   MaxE:0.001
   MaxTargetSeqs:25
   UseToClassify

   LCAMaxHits:100
   LCAScorePercent:90.0
   LCAMinIdentity:75
   LCAMinQueryCoverage:0
   LCAMinCombinedScore:0
   LCAMinLength:150

The ``ProcessBarcodes`` line specifies which barcodes MARTi should analyse during the run. The lines following ``ProcessBarcodes`` (e.g., ``BarcodeId1:Kessingland1``) are used to assign custom names to each barcode. If these lines are omitted, MARTi will assign default names using the run name followed by the barcode number (e.g., ``Sample_Name_bc01``).

Users can also rename barcodes after running MARTi. This can be done through the GUI or by creating an ``ids.json`` file in the MARTi output directory. For this example, the file would be placed at ``/path/to/marti_output/Sample_Name/ids.json``.

Here is an example of an ``ids.json`` file to rename two samples after the analysis has been completed::

    {
        "Kessingland1": "Kessingland1_Autumn24",
        "Kessingland2": "Kessingland2_Autumn24"
    }
