Inferring direct regulatory targets of a transcription factor in the DREAM2 challenge

In the DREAM2 community-wide experiment on regulatory network inference, one of the challenges was to identify which genes, in a list of 200, are direct regulatory targets of the transcription factor BCL6. The organizers of the challenge defined targets based on gene expression and chromatin immunoprecipitation experiments (ChIP-chip). The expression data were publicly available; the ChIP-chip data were not. In order to assess the likelihood that a gene is a BCL6 target, we used three classes of information: expression-level differences, over-representation of sequence motifs in promoter regions, and gene ontology annotations. A weight was attached to each analysis based on how well it identified BCL6-bound genes as defined by publicly available ChIP-chip data. By the organizers' criteria, our group, GenomeSingapore, performed best. However, our retrospective analysis indicates that this success was dominated by a gene expression analysis that was predicated on a regulatory model known to be favored by the organizers. We also noted that the 200-gene test set was enriched only in genes that are upregulated, while genes bound by BCL6 are enriched in both upregulated and downregulated genes. Together, these observations suggest possible model biases in the selection of the gold-standard gene set and imply that our success was attained in part by adhering to the same assumptions. We argue that model biases of this type are unavoidable in the inference of regulatory networks and, for that reason, we suggest that future community-wide experiments of this type should focus on the prediction of data, rather than models.