Table 2.
Methodologies Involved in the Evaluation ofDIAN: Strengths and Weaknesses
| Approach number | Approach type | Description | Strengths | Weaknesses |
| 1 | Manual verification of assignments made to selected proteins. | In-depth review by domain experts of assignments made to well- understood proteins. | Extensive human expertise can confirm assignments made by method and substantiate its effectiveness. | Suffers from lack of comprehensiveness; biased in favor of well-understood proteins. |
| 2 | Comparisons with other assignment data sets using a test set of sequences. | Evaluation of sequence assignments made to cross-referenced ontologies using different methods. | Presence of extensive shared assignments for numerous proteins lends credence to the method under evaluation. | Assumes that the reference ontology can be treated as a standard of comparison; in practice, this is not the case. Results in the identification of weaknesses in both the test and reference ontology. Manual review is required to evaluate unbalanced assignments. |
| 3 | Comparisons between orthologs. | Verification that assignments made to closely related orthologs are balanced, (i.e., nearly identical). | Strong expectation that balanced assignments will be made. | Although orthologs share functions, even orthologs share functions, even orthologs from closely related species don't necessarily have identical functions, resulting in unbalanced assignments; manual review is required to evaluate unbalanced assignments. |
