Systematic Comparison of False-Discovery-Rate-Controlling Strategies for Proteogenomic Search

Abstract

Proteogenomic searches are useful for novel peptide identification from tandem mass spectra. Usually, separate and multi-stage approaches are adopted to accurately control the false discovery rate (FDR) for proteogenomic search. Their performance on novel peptide identification has not been thoroughly evaluated, however, mainly due to the difficulty in confirming the existence of identified novel peptides. We simulated a proteogenomic search using a controlled, spike-in proteomic dataset. After confirming that the results of the simulated proteogenomic search were similar to those of a real proteogenomic search using a human cell line dataset, we evaluated the performance of six FDR control methods—global, separate, and multi-stage FDR estimation, respectively coupled with a target-decoy search and a mixture model-based method—on novel peptide identification. The multi-stage approach showed the highest accuracy for FDR estimation. However, global and separate FDR estimation with the mixture model-based method showed higher sensitivities than others at the same true FDR. Furthermore, the mixture model-based method performed equally well when applied without or with a reduced set of decoy sequences. Considering different prior probabilities for novel and known protein identification, we recommend using mixture model-based methods with separate FDR estimation for sensitive and reliable identification of novel peptides from proteogenomic searches.