The label on our food does not always reflect the what is in a product. This became very clear in Europe in 2013 were horse meat was found in many frozen meat products, without this component being declared on the package. The whole magnitude of the scandal was revealed by DNA testing, mostly by confirming DNA origin by PCR and Sanger sequencing (Walker et al. 2013). While private companies started to conduct their own testing once they realised the scale of the issue, the whole scandal was unveiled by testing done and / or contracted by government agencies.
Last week, the startup Clear Labs (California, US) released a report as part of their Clear Food initiative, claiming 14.4% of the tested hot dogs were problematic and especially vegetarian products prone to contamination and hygienic issues, compared to the meat hot dogs. This of cause was picked up by the media, leading to quite a buzz and fueling the companies kickstarter campaign (which they probably only do for marketing / media reasons, as they just secured $6.5M funding).
So what is all this about? Do we have another food scandal coming up? Is Clear Labs the new friendly company which want to help you, the consumer, or are they more friends with the food industry? And what methods / did they use, is the study scientifically solid? Lets find out!
Clear Labs “secret” sequencing workflow
A total of 345 samples were tested in the Clear Food Hot Dog Report, using a “Clear Labs’ proprietary next-generation genomic sequencing workflow”. I reached out to Clear Foods on twitter for further information on laboratory method and protocol, but they are not willing to share their “company secrets”. A company that want’s more food transparency, but keeps a culture of secrecy and unverifiable results?
Well let’s see what information they are sharing concerning the hot dog report:
- They tested 345 samples, from 75 different brands, from 10 retailers (No further details on sample processing and replicates are given).
- They Don’t provide a list of exact products tested (which should include expiration dates and production time / information often included on packages).
- They use a “proprietary next-generation genomic sequencing workflow” and detect “major, medium, minor & trace substitution” (Semi quantitative substance abundance).
- They might only be sampling a small pice of the product, as they state “foods might not be homogenous”. (Why not homogenize a larger proportion of the product then? From personal experience I recommend the TissueLyser LT and ULTRA-TURRAX® Tube Drive control).
That’s it. No word on detailed laboratory protocols, bioinformatics pipelines or statistics. It’s a black box, there is no way we as scientists could validate the methodological accuracy of the study. However if you dig deeper on the Clear Labs website, you will find a video: Behind the Scenes at Clear Labs giving more details about what they are actually doing!
What can we learn from the video about the technology Clear Lab has developed (Notice that the things shown in the video might not reflect how they are handling actual samples, but this is the best guess we have):
- They process food samples in duplicates (Unfortunately, they don’t tell us if both duplicates are sequenced and how sequences are treaded which are only present in one of the duplicates).
- The lab does not look like it has ancient DNA standards, no clean benches are shown. They use filter tips, which is good to reduce contamination. Tissue sample is taken on a kitchen paper towel with a plastic knife. (As the hot dogs are heavily processed, DNA is potentially very degraded and present in small quantities, making it difficult to detect and prone to cross contamination with DNA and PCR products present in the laboratory, which is problematic).
- “We purchased every single food item in the US market [and] sequenced it” to create a reference database. (I seriously doubt that! That would have been millions of samples they would have sequenced within a few years).
- “After DNA extraction, we are talking our samples to two steps of PCR, in order to amplify the DNA region of interest and in order to give each sample an unique tag.” (So they are doing standard metabarcoding, which is targeted amplicon sequencing. This has nothing to do with genomics or metagenomics, which is the study of genomes).
- They pool ~1.000 samples and sequence them on the Illumina MiSeq system. (Depending on the version of the chemistry they are using, that would give them 12-25 M sequences, with ~25.000 sequences per sample).
So in conclusion, Clear Labs is using a standard metabarcoding protocol, on a HiSeq system. Nothing wrong with that, but a PCR based approach can introduce biases that you have to account for in your study. For example, not all animals are amplified equally in metabarcoding samples, as we have shown in our own research (Elbrecht & Leese 2015). Additionally, when working with trace DNA, a lot of factors like DNA degradation and cross contamination between samples can introduce additional biases (see review by Barnes & Turner 2015 for a good overview).
Until they publish their exact sampling lists, laboratory methods, bioinformatics and raw sequencing data, the report they provide is absolutely meaningless. It is not verifiable and the shown effects could be artifacts or at least biassed by sampling, laboratory methods and sample handling.
Misleading infographic, no clear data
Despite the shortcomings, lets take a look at the data and what it actually could mean. The report presents it self with colourful infographics and diagrams, which would be “OK” but if you take a closer look at the data you will find data cherry picking. Lets try to convert the data into a more meaning full “scientific” graph.
In only a few products unlabeled meat was found, and those are probably traces (Clear Labs does only give presence absence data). So your hot dog should be fine right? Lets look at some claims from the study, which did lead to wrong conclusions and hysteria in mainstream media when taken out of context.
“10% of Vegetarian products contained meat.” – Yes, 10% = 2 samples. This could have easily been cross contamination or tag switching in your library construction / sequencing reaction (Carlsen et al. 2013). We are talking about only two samples, you need a sample size larger than 21 and do independent analysis in at least 2 independent laboratories before you make those claims. Because media will not check the data / study from Clear Labs and just copy the suggestive headline this might cause panic and food scare. (Media can’t really check on the report because not enough details are shared in the study).
Also, what interest would an vegetarian hot dog producer have in adding meat, which is more expensive than grain or milk (which is commonly used as a meat substitution)? To me it looks like Clear Labs is not confident in their own data, as they don’t release the names of the manufacturers wich have meat contaminated hot docs.
“Vegetarian items accounted for 67% of the hygienic issues found in the report.” – With hygienic issues being defined by detection of human DNA.
Again, we talking about a tiny sample size here: only 4 samples! But this actually does make sense; Vegetarian products should contain no meat, which makes it easier to amplify the trace amount of human DNA which might be present in the sample. I am sure the meat hot dogs are equally “contaminated” with human DNA, but as they contain probably 99.9999% DNA derived from chicken, pork and other farm animal meat the trace amount of human DNA is just not picked up in the PCR amplification.
Additionally, as also the National Hot Dog and Sausage Council writes: DNA is every were! Finding trace amounts of DNA is not an issue, and does not necessarily reflect the hygienic conditions at the production facilities. There was a case in Germany were police hunted a female phantom killer, which was present at over 30 crime scenes. It turned out that the cotton swabs the police used to collect DNA samples were sterile but not certified DNA free, and the phantom of Heilbronn was finally found: A woman who did work at the factory producing the cotton swabs.
Over all, the hot dog report is nothing you should worry about. As long as the Clear Labs don’t publish methods and raw data, authenticity of the report is questionable. Even if there are really trace amounts of other meats in your hot dog, this can also be due to production logistics (no one will clean industrial hot dog making machines to a DNA free level, and the kind meat processed might be changing from day to day). As long as no one finds half your hot dog is an unlabelled meat, there is really nothing to worry about. The hot dog report is a rather clever marketing gack and not the next big meat scandal.
Metabarcoding: The wrong approach!
Now as we established what laboratory methods Clear Labs is using, and their report is nothing to worry about lets look at another aspect of the study: It it even the right molecular approach to test food ingredients and authenticity?
We discussed the advantages of metabarcoding and metagenomics on this blog before. In short, a metagenomics approach has the advantage to be more detection sensitive and having to potential to estimate abundance of beef species in a hot dog, because unlike metabarcoding it does not rely on PCR amplification of a partial gene but rather sequences every pice of DNA that is present in the sample directly. Metagenomics are not widely used, because the complete reference genome of each target organism is needed to map the identify the sequenced pieces of DNA.
There are complete reference genomes of Chicken, Pig (Pork), cattle (Beef), Turkey, Lamb and also Human available for free on NCBI. Usually genomes of economically relevant plants and animals are the first which are sequenced. I have no idea why Clea Labs has dictated not to use this resource and used the metabarcoding method instead, which is more complicated and error prone than a metagenomics approach in hot dog research project. @Clear Labs, pleas let me know why you did use metabarcoding instead of a metagenomics approach! I can’t see a reason why, but I’m really curious.
Clear Labs needs you to freak out!
So can we make sense of all of this? What does Clear Labs want? What is their bossiness model? In my opinion they are following a quite clever but also dangerous strategy.
- They branch of a part of their company to create Clear Food, a litte friendly project that wants food transparency and “fights” the evil food industry which is ling to you on there food labeling! Clear Food is your friend, they also need your money on kickstarter.
- They create nice compelling videos, and color full infographics telling you all the evil in the food industry (and your hot dogs). Media picks up on the project, creating a buzz and getting the public concerned. Journalists often don’t understand scientific jargon, so they don’t even bother to provide study details.
- The public gets concerned / freaks out forcing retailers and food producers to adopt and embrace the “Clear List” and put little score stickers on their products, so the consumer knows the product is save and free of human contamination.
- Now companies have to pay Clear Labs to monitor their products and supply chains, and Clear Labs holds a monopole on monitoring and protocols, as those are a trade secret to “protect [their] intellectual property from competitors” as they outline in their FAQ.
Major companies will not care about genetic testing, until the public is concerned and scarred about DNA being every were. Don’t get me wrong here, genetic testing is really use full in cases like the horse meet scandal in Europe. But the hot dog report is nothing like that. Clear Food needs you to be scared, so they can make a profit with Clear Labs. They want to be friends with you, the consumer, but also friends with the food industry (why else don’t they publish the data on which hot dogs brands are supposably contaminated?). Clear Labs / Food want to be friends with both sides… this can’t end well.
I think DNA based monitoring in the food industry is very valuable, but it should come from an independent institution like governmental agencies or non profit organisations. A company which first scares the public to force food producers to buy their DNA testing services, which no one can validate and which might not be scientifically solid: Thats pure evil!
The hot dog report is nothing to worry about. What I do worry about is the baad scientific practise, secrecy and marketing strategy Clear Labs is building on. You can not claim things without sharing evidence for it! Clear Food needs to stand up and share their data and protocols, additionally publish the results and raw data in a peer reviewed journal. Other wise the will not gain credibility and have an unsustainable future. However, so far they respectfully declined:
“Our technology and our results are sound. We do not, however, plan to provide the NHDSC or the public the full, unfettered results of our testing.” – Clear Labs’ response to the National Hot Dog and Sausage Council – 29 October
What are you scared from, Clear Labs? Pleas try to be more scientific and open, like you claim on your website. If you really care about the consumer, start to act like scientists would and not like a company who want’s to make a fortune.
PS: @Clear Labs: If you need help with your laboratory protocols I am happy to help, I think you have room for improvements of your methods. If journalists are interested in an interview or if I should proofread your newspaper article for scientific accuracy please let me know. I know this stuf can be confusing to non-scientists, but I think its important to report on this honestly and accurately.
Barnes & Turner (2015). The ecology of environmental DNA and implications for conservation genetics. Conservation Genetics.
Carlsen, Aas, Lindner, Vrålstad, Schumacher & Kauserud (2012). Don’t make a mista(g)ke: is tag switching an overlooked source of error in amplicon pyrosequencing studies?. Fungal Ecology
Elbrecht & Leese (2015). Can DNA-Based Ecosystem Assessments Quantify Species Abundance? Testing Primer Bias and Biomass—Sequence Relationships with an Innovative Metabarcoding Protocol. PloS One
Walker, Michael, Malcolm Burns & Burns (2013). Horse meat in beef products—species substitution 2013. Journal of the Association of Public Analysts.