https://www.biostars.org/p/198405/

Quick question is: I have some mapped reads in bam file which have good read quality, but they have sam flag 0x200 which means they didn't pass the vendor check. Should I include them or not in downstream analysis?

Long question is: what' s the relationship between read quality score and Chastity score?

First, everybody may know read quality score:

Reads quality score(phred score) is calculated by -10*log(P(error_base)), P(error_base) represents the probability that the base is incorrect.

Second, I want to talk about Chastity score during the vendor check:

For reads in fastq format, there is a header field 'Y/N' which indicates whether the read pass filtering step. And the corresponding sam flag is 0x200, indicating "not passing filters, such as platform/vendor quality controls". How does Illumina set the filtering criteria?

As far as I know, read filtering by Illumina Real Time Analysis (RTA) happens during the run, and filtering is determined by Chastity score. Chastity Score is calculated by “the ratio of the highest of the four (base type) intensities to the sum of highest two”. Illumina described the vendor check as follows:

"To remove the least reliable data from the analysis, the raw data can be filtered to remove any clusters that have “too much” intensity corresponding to bases other than the called base. By default, the purity of the signal from each cluster is examined over the first 25 cycles and calculated as Chastity = Highest_Intensity / (Highest_Intensity + Next_Highest_Intensity) for each cycle. The new default filtering implemented at the base calling stage allows at most one cycle that is less than the Chastity threshold. The higher the value, the better. This value is very dependent on cluster density, since the major cause of an impure signal in the early cycles is the presence of another cluster within a few micrometers."

So, to my understanding, every cycle the Sequencer scan a cluster, there would be 4 kinds of signals from 4 bases(am I right?) the most significant base would the final choice. The bigger the signal intensity divergence is the better for base calling. For the first 25 cycles, Illumina allow at most one base with smaller signal intensity divergence, otherwise, Illumina would set the read as vendor failed. Is my understanding right so far?

But what is the relationship between the Phred score and the Chastity score? if they really have. Can I still use vendor failed reads if they have high phred score?

Thanks! Tao

ADD COMMENT • link •

Not following 

modified 10 months ago by ablanchetcohen • 1.1k • written 10 months ago by Tao • 110

 
1

Curious. Why are the vendor failed reads in your dataset?

ADD REPLY • linkmodified 10 months ago • written 10 months ago by genomax2 ♦ 26k
 

I downloaded the bam file from GTEx (dbGaP). The bam file contains all the reads, including mapped, unmapped, vendor failed reads. For a sample with ~100M reads, ~12M are labeled as vendor failed including both mapped and unmapped reads. Part of the vendor failed reads have read good quality. So, I'm not sure if I should include them.

ADD REPLY • linkmodified 10 months ago • written 10 months ago by Tao • 110
 
0
 
10 months ago by
Canada

I second this comment. You should contact your vendor. I have never seen reads failing the filtering step indicated in the header field of a FASTQ file being given to a client. Why include these reads? They just take up storage space, and are likely to induce errors in the downstream analysis. There was either an error in the setting of the flag, or a mistake in giving you the reads.

ADD COMMENT • linkwritten 10 months ago by ablanchetcohen • 1.1k
 
1

I checked the GTEx Project FAQ. The alignment was probably done in 2012, since TopHat v1.4.1 was used. This was the very dawn of RNA-Seq. The analyses dating back to this period are often suspicious since bioinformaticians were not yet familiar with RNA-Seq, and the software programs contained bugs more often than not. My recommendation is always to treat with suspicion any analysis results dating back to this period. Most likely, those preparing the data were not aware yet that these reads should be filtered out.

I would filter out all the "vendor failed reads", and redo the alignment using a more recent aligner, genome, and annotation. At least, that would be my recommendation based on my knowledge. To get a definitive answer, you could contact the staff at the GTex project.

ADD REPLY • linkwritten 10 months ago by ablanchetcohen • 1.1k
 

Thanks, your comments are very helpful!

ADD REPLY • linkwritten 10 months ago by Tao • 110
 

thanks for your comments. The sample is downloaded from a public project GTEx. I'm also confused why they deposit so many(10M vendor failed for a 100M sample) vendor-failed reads on dbGaP. In my study, I didn't realize this problem at first, which causing a big problem now. In your opinion, such reads should be removed without considering reads quality?

ADD REPLY • linkwritten 10 months ago by Tao • 110
 
1

Short answer yes.

They were "failed" by Illumina pre-processing software for a reason (e.g. mixed sequence from one cluster, phasing issues etc).

ADD REPLY • link

Question: Should I use reads with good quality but failed-vendor flag?--biostart for vendor quality的更多相关文章

  1. 去除外显子低质量reads时弹出错误“Invalid quality score value (char '#' ord 35 quality value -29) on line 4”和“Invalid quality score value (char '.' ord 46 quality value -18) on line 12”的解决方法

    楼主跑以下程序时分别弹出了“fastq_quality_filter: Invalid quality score value (char '.' ord 46 quality value -18) ...

  2. Quality assessment and quality control of NGS data

    http://www.molecularevolution.org/resources/activities/QC_of_NGS_data_activity_new table of contents ...

  3. XSplit Quality, VBV-Buffer, VBV-Maxrate and Preset Settings

    XSplit uses the x264 encoder, so let's start off by saying that parameters mentioned in the title, w ...

  4. Quality 是什么?

    Quality 是什么? 通常,我们谈及 Quality(质量)时,最常见的问题就是:Quality 是什么? 有很多业界先驱和研究人员已经回答了这个问题,我在这里并不会再给出一个新的答案.在学习总结 ...

  5. 视频主观质量评价工具:MSU Perceptual Video Quality tool

    MSU Perceptual Video Quality tool是莫斯科国立大学(Moscow State University)的Graphics and Media Lab制作的一款视频主观评价 ...

  6. software quality assurance 常见问题收录

    1. What is Quality? Quality means, “meeting requirements.” ..Whether or not the product or service d ...

  7. ITU-T G.1080 IPTV的体验质量(QoE)要求 (Quality of experience requirements for IPTV services)

    IPTV的服务质量(QoE)要求 Quality of experience requirements for IPTV services Summary This Recommendation de ...

  8. unity 质量设置 Quality Settings

    Unity allows you to set the level of graphical quality it will attempt to render. Generally speaking ...

  9. samtools常用命令详解

    samtools的说明文档:http://samtools.sourceforge.net/samtools.shtmlsamtools是一个用于操作sam和bam文件的工具合集.包含有许多命令.以下 ...

随机推荐

  1. 转载-解决使用httpClient 4.3.x登陆 https时的证书报错问题

    今天在使用httpClient4.3.6模拟登陆https网站的时候出现了证书报错的问题,这是在开源中国社区里找到的可行的答案(原文链接:http://www.oschina.net/question ...

  2. 实用的IOS应用程序框架

    实用的IOS应用程序框架 目录 概述 概述

  3. go练习3 --map的操作

    func T2_1() {     // 键值string , 值 int 类型     m1 := map[string]int{}     //添加一个元素     m1["str1&q ...

  4. js中的颜色对应的常量代码code

    颜色的对照表 颜色 英文代码 形像颜色 HEX格式 RGB格式   LightPink 浅粉红 #FFB6C1 255,182,193   Pink 粉红 #FFC0CB 255,192,203   ...

  5. 最大网络流(Dinic)

    hdu1532 Drainage Ditches Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: 65536/32768 K (Java ...

  6. 详探TextRange对象--查找与选择(转载)

    TextRange对象是动态HTML(DHTML)的高级特性,使用它可以实现很多和文本有关的任务,例如搜索和选择文本.文本范围让您可以选择性的将字符.单词和句子从文档中挑选出来.TextRange对象 ...

  7. CH1809匹配统计【KMP】

    1809 匹配统计 0x18「基本数据结构」练习 描述 阿轩在纸上写了两个字符串,分别记为A和B.利用在数据结构与算法课上学到的知识,他很容易地求出了“字符串A从任意位置开始的后缀子串”与“字符串B” ...

  8. 我见过的最完善的log4net配置

    Log4net的优点: 几乎所有的大型应用都会有自己的用于跟踪调试的API.因为一旦程序被部署以后,就不太可能再利用专 门的调试工具了.然而 一个管理员可能需要有一套强大的日志系统来诊断和修复配置上的 ...

  9. iOS定位原理和使用建议(转)

    原文:http://ibbs.91.com/thread-1548870-1-1.html 看到很多网友讨论iOS设备定位的问题,这里将我们所了解的关于iPhone.iPad.iPod等的定位原理做详 ...

  10. 理解Java异常处理机制的机理

    重看异常机制的时候觉得抓到了点机理上的精髓,所以来说一下,对初学者应该会有些帮助   JAVA中的异常机制 从机制上由[产生异常][抛出异常][捕捉异常][异常处理]组成 从形式上又分为四种: 运行时 ...