mgkit.workflow.filter_gff module

Filters GFF annotations in different ways.

Value Filtering

Enables filtering of GFF annotations based on the the values of attributes of a GFF annotation. The filters are based on equality of numbers (internally converted into float) and strings, a string contained in the value of an attribute less or greater than are included as well. The length of annotation has the attribute length and can be tested.

Overlap Filtering

Filters overlapping annotations using the functions mgkit.filter.gff.choose_annotation() and mgkit.filter.gff.filter_annotations(), after the annotations are grouped by both sequence and strand. If the GFF is sorted by sequence name and strand, the -t can be used to make the filtering use less memory. It can be sorted in Unix using sort -s -k 1,1 -k 7,7 gff_file, which applies a stable sort using the sequence name as the first key and the strand as the second key.

Note

It is also recommended to use:

export LC_ALL=C

To speed up the sorting

The above digram describes the internals of the script.

The annotations needs first to be grouped by seq_id and strand, forming a group that can be then be passed to mgkit.filter.gff.filter_annotations(). This function:

1. sort annotations by bit score, from the highest to the lowest

2. loop over all combination of N=2 annotations:

   1. choose which of the two annotations to discard if they overlap for a the required amount of bp (defaults to 100bp)

   2. in which case, the preference is given to the db quality first, than the bit score and finally the lenght of annotation, the one with the highest values is kept

While the default behaviour is the same, now it is posible to decided the function used to discard one the two annotations. It is possible to use the -c argument to pass a string that defines the function. The string passed must start with or without a +. Using + translates into the builtin function max while no + translates into min from the second character on, any number of attributes can be used, separated by commas. The attributes, however, must be one of the properties defined in mgkit.io.gff.Annotation, bitscore that returns the value converted in a float. Internally the attributes are stored as strings, so for attributes that have no properties in the class, such as evalue, the float builtin is applied.

The tuples built for both annotations are then passed to the comparison function to be selected and the value returned by it is discarded. The order of the elements in the string is important to define the priority given to each element in the comparison and the leftmost one has the highesst priority.

Examples of function strings:

• -dbq,bitscore,length becomes max((ann1.dbq, ann1.bitscore, ann1.length), (ann2.dbq, ann2.bitscore, ann2.length) - This is default and previously only choice

• -bitscore,length,dbq uses the same elements but gives lowest priority to dbq

• +evalue: will discard the annotation with the highest evalue

Per Sequence Values

The sequence command allows to filter on a per sequence basis, using functions such as the median, quantile and mean on attributes like evalue, bitscore and identity. The file can be passed as sorted already, saving memory (like in the overlap command), but it’s not needed to sort the file by strand, only by the first column.

Coverage Filtering

The cov command calculates the coverage of annotations as a measure of the percentage of each reference sequence length. A minimum coverage percentage can be used to keep the annotations of sequences that have a greater or equal coverage than the specified one.

Changes

New in version 0.1.12.

Changed in version 0.1.13: added –sorted option

Changed in version 0.2.0: changed option -c to accept a string to filter overlap

Changed in version 0.2.5: added sequence command

Changed in version 0.2.6: added length as attribute and min/max, and ge is the default comparison for command sequence, –sort-attr to overlap

Changed in version 0.3.1: added –num-gt and –num-lt to values command, added cov command

Changed in version 0.3.4: moved to use click for argument parsing reworked the values, sequence commands

Changed in version 0.4.4: overlap command: added option to not use the strand information and added an option to make multiple passes of overlap for each sequence

mgkit.workflow.filter_gff.filter_eq(annotation, attr=None, value=None, conv=None)

mgkit.workflow.filter_gff.filter_gt(annotation, attr=None, value=None, conv=None, equal=None)

mgkit.workflow.filter_gff.filter_in(annotation, attr=None, value=None, conv=None)

mgkit.workflow.filter_gff.filter_lt(annotation, attr=None, value=None, conv=None, equal=None)

mgkit.workflow.filter_gff.find_comparison(comparison)

mgkit.workflow.filter_gff.make_choose_func(values)

Builds the function used to choose between two annotations.

mgkit.workflow.filter_gff.perseq_calc_threshold(annotations, attribute, function, func_arg=None)

mgkit.workflow.filter_gff.setup_filters(str_eq, str_in, num_eq, num_ge, num_le, num_gt, num_lt)

mgkit.workflow.filter_gff.validate_params(ctx, param, values, convert=<class 'str'>)