### Gene report: AN1_CONRAD_23 vs. AN1_CALLREFERENCEGENES_21

report version 26079+26104 generated 2007 Mar 13, 08:24:33

To evaluate the accuracy of our gene predictions for Aspergillus nidulans Assembly 1, we created a set of reference gene models exclusively from EST data. We then compared the two gene sets using a variety of metrics. In the tables below, we refer to the final, published gene set as the query and the EST-based gene set as the reference.

The Feature comparisons and Splice analysis sections only report on the subset of query genes that overlap reference genes. Although a substantial number of predicted genes overlap EST alignments, the majority do not. Because we use EST data to improve our gene calls, we expect lower accuracy in regions that lack supporting EST evidence, on the order of 5–10% (smc). Therefore, while they are a useful measure of gene prediction accuracy, the numbers reported in those two sections and immediately below do not apply evenly to all predicted genes.

- Brief summary
- Assembly overview
- Overview of query genes
- Overview of reference genes
- Feature mapping
- Feature comparisons
- Splice analysis
- Possible problems

#### Brief summary

Transcripts with no splice problems: 1213, 67.6%
The table below is a subset of the tables shown in Feature comparisons, which see for more details.

| sn | sp | smc
nucleotides | 0.9772 | 0.9915 | 0.9725
splice sites | 0.8851 | 0.8342 | 0.7527

#### Assembly overview

Aspergillus nidulans Assembly 1 (AN1): 30068514 bp
30068514 bp in contigs (100.0%), 0 bp in captured gaps (0%)
super/contigs: 0/248

| min | median | mean | n50 | max
bp per contig | 2005 | 54382 | 121244 | 282268 | 1114266
bp per covered gap | - | - | - | - | -

#### Overview of query genes

13620 genes (8281 on '+' strand, 5339 on '–')
13620 transcripts (9615 spliced, 4005 unspliced)
37084 exons, 23464 introns

| len | %cov | %gc | %at
genic | 15886803 | 52.84 | 53.02 | 46.98
intergenic | 14181711 | 47.16 | 47.31 | 52.69
exonic | 14159060 | 47.09 | 53.81 | 46.19
intronic | 1727743 | 5.75 | 46.50 | 53.50
coding | 14159060 | 47.09 | 53.81 | 46.19
5′ UTR | 0 | 0.00 | - | -
3′ UTR | 0 | 0.00 | - | -
alt. spliced | 0 | 0.00 | - | -
genomic | 30068514 | 100.00 | 50.32 | 49.68
| min | median | mean | n50 | max
total length (incl. UTR + introns) | 9 | 971 | 1166 | 1720 | 21811
coding length | 9 | 840 | 1040 | 1560 | 21462
exons per transcript | 1 | 2 | 2.72 | 3 | 18
exons per spliced transcript | 2 | 3 | 3.44 | 4 | 18
bp per exon | 5 | 190 | 382 | 801 | 4975
bp per intron | 16 | 59 | 74 | 66 | 600
5′ UTR bp | - | - | - | - | -
3′ UTR bp | - | - | - | - | -

#### Overview of reference genes

2427 genes (1221 on '+' strand, 1314 on '–')
2535 transcripts (1626 spliced, 909 unspliced)
5989 exons, 3454 introns

| len | %cov | %gc | %at
genic | 2357945 | 7.84 | 51.52 | 48.48
intergenic | 27710569 | 92.16 | 50.22 | 49.78
exonic | 2112388 | 7.03 | 52.18 | 47.82
intronic | 241222 | 0.80 | 45.77 | 54.23
coding | 1748613 | 5.82 | 53.49 | 46.51
5′ UTR | 141739 | 0.47 | 50.25 | 49.75
3′ UTR | 236360 | 0.79 | 43.54 | 56.46
alt. spliced | 4335 | 0.01 | 47.84 | 52.16
genomic | 30068514 | 100.00 | 50.32 | 49.68
| min | median | mean | n50 | max
total length (incl. UTR + introns) | 398 | 840 | 974 | 940 | 3133
coding length | 204 | 705 | 716 | 735 | 2103
exons per transcript | 1 | 2 | 2.36 | 3 | 11
exons per spliced transcript | 2 | 3 | 3.12 | 5 | 11
bp per exon | 5 | 281 | 368 | 622 | 2342
bp per intron | 42 | 59 | 76 | 68 | 918
5′ UTR bp | 1 | 54 | 76 | 146 | 879
3′ UTR bp | 1 | 105 | 130 | 228 | 1212

#### Feature mapping

Searched for mappings from 13620 query features to 2427 reference features.

Reference genes farther than 200bp from any query gene: 93
Containing 93 transcripts: 58 unspliced, 35 with canonical splices.
These may indicate missed genes.

128 query genes (0.9%) hit 108 reference genes (4.4%) on the opposing strand. Hits to opposing strands are considered misses in the following comparisions.

Strand mismatches such as these may be orientation problems in the reference set, or incorrect predictions in the query set. Unspliced reference transcripts are difficult to orient correctly. 74 reference gene(s) did not contain a single canonical splice site. These may easily have been placed on the wrong strand; queries hitting these genes are likely to be oriented correctly. 34 reference gene(s) had at least one canonical splice site; everything else being equal, their orientation is more likely to be correct than the queries hitting them.

The table below shows how query genes (rows, roman type) map to reference genes (columns, italic type). We cluster genes into overlapping groups on the same strand and record eight types of relationship. To the left of each arrow is the number of query genes in each type of cluster; the number of reference genes is to the right.

| none | reference one | many
| none |  | - |  | 0 | ↔ | 240 | 0 | ↔ | 0
query | one | 11825 | ↔ | 0 | 1265 | ↔ | 1265 | 419 | ↔ | 846
| many | 0 | ↔ | 0 | 70 | ↔ | 35 | 41 | ↔ | 41
| counts per cluster type
| none | reference one | many
| none |  | - |  | 0 | ↔ | 9.9 | 0 | ↔ | 0
query | one | 86.8 | ↔ | 0 | 9.3 | ↔ | 52.1 | 3.1 | ↔ | 34.9
| many | 0 | ↔ | 0 | 0.5 | ↔ | 1.4 | 0.3 | ↔ | 1.7
| percentages per cluster type

one↔one clear, unambiguous mappings from one query gene to one reference gene. none↔one reference genes that do not map to any query gene (misses). none↔many reference genes that overlap other reference genes but do not map to query genes. None↔X mappings may represent underpredictions. one↔none query genes that do not map to any reference gene. many↔none query genes that overlap other query genes but do not map to reference genes. X↔none mappings may represent overpredictions or deficiencies in the reference gene set. one↔many single query genes overlapped by multiple reference genes (merges). These may occur if the reference set contains partial genes. many↔one single reference genes spanning multiple query genes (splits). These are often annotation errors. many↔many complex clusters where multiple query genes map to multiple reference genes.

#### Feature comparisons

Predicted genes that hit reference genes on the wrong strand, or hit reference genes containing only non–canonically-spliced transcripts, are ignored in the following calculations. When scoring genes with multiple transcripts, we select the single best transcript↔transcript comparison. For clusters in the one↔many and many↔many categories above, each query is compared to as many references as possible. The best non-overlapping comparisons are used to score these loci.

| #comp | TP | TN | FP | FN |
nucleotides | 1666923 | 1435147 | 185970 | 12320 | 33486 |
splice sites | 6663 | 5015 | - | 997 | 651 |
| sn | sp | smc | acp | ac | cc
nucleotides | 0.9772 | 0.9915 | 0.9725 | 0.9385 | 0.8770 | 0.8762
splice sites | 0.8851 | 0.8342 | 0.7527 | - | - | -

| #comp | TP | TN | FP | FN |
introns | 2991 | 2359 | - | 242 | 54 |
exons | 4378 | 3326 | - | 30 | 47 |
| sn | sp | sn_sp | wrong | miss |
introns | 0.8380 | 0.7887 | 0.8134 | 0.0192 | 0.0809 |
exons | 0.7694 | 0.7597 | 0.7645 | 0.0109 | 0.0069 |

#comp the number of subfeatures compared. TP true positives: nucleotides predicted as exonic in both query and reference; or, splice junctions with exact agreement, in position and type (donor:donor, acceptor:acceptor), between query and reference; or, introns or exons with both splice sites in exact agreement with the reference. TN true negatives: nucleotides predicted as intronic in both query and reference; not defined for splice sites, introns or exons. FP false positives (overpredictions): nucleotides marked as exonic in the query but intronic in the reference; or, splice junctions identified only in the query*; or query introns/exons that do not overlap a reference intron/exon. FN false negatives (underpredictions): nucleotides marked as intronic in the query but exonic in the reference; or, splice junctions identified only in the reference*; or reference introns/exons that do not overlap a query intron/exon. sn sensitivity, TP ÷ (TP+FN). sp specificity, TP ÷ (TP+FP). smc simple matching coefficient, (TP+TN) ÷ (TP+FN+FP+TN). acp average conditional probability. ac approximate correlation. cc correlation coefficient. These last three terms are described in detail in the citation below, and are not well defined for splices, introns or exons. *In the very rare cases where the query predicts a donor site exactly where the reference has an acceptor, or vice versa, we count it as both FP and FN. Terminology is from Burset M and Guigo R, "Evaluation of gene structure prediction programs." Genomics, 1996 Jun 15;34(3):353-67.

The table below briefly summarizes the relative coverage of overlapping query and reference genes. The first column shows the amount of sequence (in bp and percent) in each gene set that is overlapped by sequence in the other gene set. The next three columns show the overhanging sequence from transcripts that overlap but are offset. Overhangs are categorized as 5′ (one transcript begins upstream of the other), 3′ (one transcript extends downstream past the other), and mixed (indeterminate). Finally, the last column, "complete miss", counts the number of bp in queries that did not hit a reference at all, and vice versa.
A high degree of overlap can indicate a good match between the query and reference sets. A high "missed" score may simply indicate a small reference set.

| overlap | overhang, 5′ | overhang, 3′ | overhang, mixed | complete miss
query | 1666923 bp | / | 10.5% | 445300 bp | / | 2.8% | 260288 bp | / | 1.6% | 382400 bp | / | 2.4% | 13131892 bp | / | 82.7%
reference | 1666923 bp | / | 69.0% | 142247 bp | / | 5.9% | 245435 bp | / | 10.2% | 164501 bp | / | 6.8% | 198140 bp | / | 8.2%

#### Splice analysis

5015 splice agreements, 1648 disagreements.
4333 ignored: 78 due to EST misalignment, 4255 due to partial initial/terminal exon coverage.
perfect exon:exon/intron:intron matches: 1206/2359
0 query transcripts contained noncanonical splices.

transcripts with no splice problems | 1213 | 67.6%
... with complete reference coverage | 423 | 23.6%
explainable by alternate splicing | 336 | 18.7%
... with spliced reference | 278 | 15.5%
clashes | 246 | 13.7%
... with spliced reference | 212 | 11.8%

Transcripts that have a splice site disagreement with an overlapping reference gene are placed into two categories, depending on the severity of the clash. If all splice disagreements could be explained by well-known types of alternate splicing, we call the transcript a "possible alternate splice." If the two transcripts cannot be reconciled in this way, we label the query a "clash." In partitioning splice disagreements into two categories, we are not asserting that 18.7% of this genome shows alternate splicing. We do this as a form of triage: genes in the "clash" category are manually inspected before release.

| in ref. | in query
cassette exons |  | 12 | 7
retained introns |  | 190 | 42
early 3′ splices |  | 40 | 92
late 5′ splices |  | 28 | 30

cassette exon an exon that falls completely with an intron of a variant transcript. Such exons may represent alternative splice forms but are more likely instances of exonic over- and under-prediction. retained intron an intron that falls within the exon of a variant transcript. These introns may indicate alternative splicing but usually are over- and under-predicted introns. early 3′ splices two introns agree on their 5′ splice site but differ on the 3′ side, relative to the affected intron. In other words, differing 3′ splice sites lie on the leading edge of an exon. late 5′ splices two introns agree on their 3′ splice site but differ on the 5′ side, again relative to the affected intron. Most terminology is from Matlin AJ, et. al. Understanding alternative splicing: towards a cellular code. Nat Rev Mol Cell Biol. 2005 May;6(5):386-98.

#### Possible problems

| [no evidence feature] | 13620
short proteins < 50aa | 1754 | 1754 | ← not tallied in problems
shorter proteins < 30aa | 840 | 840 |
very short proteins < 10aa | 76 | 76 |
initial exon ≤ 6bp | 73 | 73 |
internal exon ≤ 6bp | 25 | 25 |
terminal exon ≤ 6bp | 0 | - |
≥ 15 exons | 3 | 3 |
intron ≥ 1000bp | 0 | - |
intron ≤ 20bp | 34 | 34 |
first codon not Met | 68 | 68 | ← not tallied in problems
first codon not xTG | 66 | 66 |
first codon not known START | 68 | 68 |
last codon not STOP | 81 | 81 |
contains in-frame STOP | 210 | 210 |
coding length not modulo 3 | 90 | 90 |
non-canonical splicing | 0 | - |
has ≥1 good BLAST hit | 0 | - | ← not tallied in problems
≤1/3 as long as BLAST hit | 0 | - |
≥3× longer than BLAST hit | 0 | - |
contains ≥1 N in exon | 0 | - |
contains low-quality sequence | 0 | - | ← not tallied in problems
touches gap(s) | 0 | - |
spans contigs | 0 | - |
within 1kb of contig edge | 369 | 369 | ← not tallied in problems
any overlap (UTR or CDS) | 0 | - |
CDS overlap only | 0 | - |
CDS overlap > 50bp | 0 | - |
CDS overlap > 100bp | 0 | - |
CDS overlap > 200bp | 0 | - |
has predicted UTR | 0 | - | ← not tallied in problems
UTR ≥ 50% length | 0 | - | ← not tallied in problems
UTR is spliced | 0 | - | ← not tallied in problems
one or more problems | 1274 | 1274 |

#### Name counts

[none] | 13620