Science Events

The real problem of irreproducible data

               

You have to know what is real before you know what is relevant. In science we trust that published means “real”. We may debate the impacts of different conditions and ooze skepticism over significance measurements and choose, sometimes less than tactfully, to object to an author’s conclusions, but overall we trust the data are real for the experimental conditions and that repetition of the experiment under the same conditions would yield the same outcome.

However, in a Catch 22 the data on reproducibility disagree.

In a collection of articles in the open source Nature special issue on data reproducibility, the most optimistic studies show 25 percent reproducibility of academic research. One article from Glenn Begley and Lee Ellis led to the development of Begley’s Six Rule for Reproducibility, which include blinded studies, full disclosure of all results, and reagent validation. Based on these rules, the optimistic 25 percent of reproducible studies are probably coming from far less than 25 percent of total academic labs. I know very few researchers that can honestly read all six rules without a little shame. 

On July 9th, Oxbridge Biotech Roundtable (OBR) hosted a panel discussion on this issue of data reproducibility. The panel included Dr Liz Silva, the MIND Program Manager at UCSF and the former Senior Editor at PLoS ONE; Dr Tim Gardner, founder of Riffyn; Dr William Gunn, Head of Academic Outreach at Mendeley; and Dr Corey Goodman, a partner with venBio.

Goodman opened the conversation with the first question he asks of founders pitching their science to him: “who has reproduced this?” The rest of the discussion focused on the problems, like an increasing rate of article retraction, and potential causes, such as intense pressure to publish, funding shortages, and increased oversight. Somewhat frustratingly, very few solutions were offered.  Gardner made a compelling argument for an incentivized approach to solving the problem. He suggested that two factors hold back scientists from solving the reproducibility issue: our culture and our lack of tools. We operate in “a culture of noise” in biology and accept that irreproducibility is unavoidable. However, if we employed adequate documentation tools similar to other industries it could greatly diminish the issue. Through incentives, like lower costs and timesavings by eliminating unnecessary, faulty repetitions, we can develop and adopt new tools and change the culture. (Gardner discusses this more in the Podcast referenced later.) When asked about automation as another tool in this solution, the panel felt it might be in the future, but currently there are other more pressing issues to address. (This could also be a reflection of the fear of the outdated bench scientist, but that is a topic for another day.)

Another potential, but more dogmatic approach, is a mandated NIH requirement for reproducibility standards. However, this reeks of additional strain on an already stretched system and more delays in publication, which will only compound the stresses that contribute to irreproducibility in the first place.

More adaptable and agile solutions are being developed through the Reproducibility Initiative. This partnership between Science Exchange, Mendeley, PLoS, and Figshare was briefly touched upon, but has the potential to revolutionize the way research is conducted as we incorporate tools, such as those being developed by Riffyn, and commercial replication services become cheaper and more convenient.

Overall this is a discussion that needs to continue and it requires total engagement from the entire scientific community. There is no clear solution but isn’t this the kind of problem we live for?

(To solicit more thoughts on solutions and the topic in general from two of the panelists, Silva and Gardner, I was asked to conduct an impromptu interview for a Podcast. By impromptu, I mean this interview was conducted on about five minutes notice.  I even forget Liz’s last name!  Anyway, it is clear I am not ready for the evening news.  As soon as the Podcast is up I will add the link here, so stay tuned!)

Science writing tips from former Nature editors and OBR

               image

          Minard’s graphic of Napolean’s Russian Campaign. Purported to be near perfect. 

Why is a blog post on a scientific writing workshop so difficult to write? Perhaps it’s the vision of a reader slowly nodding with a slight smirk thinking, “I love the author’s ironic inclusion of mistakes in a post of pointers,” when in truth, I’m not that witty? Publicly posting writing of any kind, scientific or personal, makes the author vulnerable. Writing on writing tips only exposes the author more.

With that introduction I have already ignored several of the tips presented by Richard Gallagher and Natalie Dewitt to 50 graduate students and postdocs eager to improve their science writing skills Monday February 24th at the University of California, San Francisco. The event was hosted by Oxbridge Biotech Roundtable and moderated by Ben Cohn. Below are some of the tips from Gallagher and Dewitt, who are the founders of Accendo Editing and former editors at Nature.

               image

Know your audience

Employ the following advice always keeping your audience in mind.

Titles

Avoid clichés, colons, incorporation of acronyms into words, and inform.

In my first conference abstract of graduate school I broke two of the rules above with: “Forging a Genetic Tool: The Quest to Increase Homologous Recombination Efficiency in Toxoplasma gondii”. My graduate lab loved catchy titles. Fortunately, my “forging a genetic tool” isn’t overly cliché, returning few relevant Google results. On the other hand, Dewitt’s example of “mending broken hearts stem cells” returns over 90,000 results!

               image

Colons are a technical issue as they interfere with some search engines and importing titles with colons can result in weird changes to the title structure.

The incorporation of acronyms into words like “Grb-ing receptor activation by the tail” experiences a love/hate reaction from readers. Is a catchy title worth decreasing readership over or increasing criticism from irritated readers? 

Acronym incorporation can also be at the expense of informing your audience.  Titles like “Death by Numbers” and “An Open and Shut Case” provide no indication of an article’s content and may be overlooked.

Clarity

Use short words, short sentences, and short paragraphs. Graphics are important. Tell a story and don’t show off!

Anglo-Saxon language is the language of the people and the language of short words. It is concise and sharp. Anglo-Saxon dialect selects “use” over “utilize”, “by” over “via”, and “help” over “facilitate” as pointed out by Dewitt. Carl Zimmer’s “The Index of Banned Words (The Continually Updated Edition)” is helpful for eliminating superfluous language as well. Short sentences and short paragraphs will follow suit as you are mindful of clarity in your writing.

Well-crafted graphics can say more in a small space than words can in an equivalent space.

The Inverted Pyramid of News Writing is an example of such a graphic and it lays out a good method for telling a story in science writing. If your writing doesn’t flow and tell a clear story you will lose or confuse your readers.

                     image

If you stick to short words, sentences, and paragraphs you will avoid the showy vocabulary or sentence structure better suited for Victorian fiction. Another trap scientists writing about science fall into is the need to prove their expertise through excessive detail. Stick to only what is necessary for the audience. If this pains you, try adding links for additional reading to the end of an article.

Minimize bias

Include alternative viewpoints and quotes from external experts and minimize hype.

Often I lazily overlook the inclusion of alternative viewpoints and quotes from people not involved in the subject of an article. For instance, in my blog on contract research organizations (CROs) I did not include the drawbacks of CROs. In my article on Mundipharma I neglected to include quotes from people outside of the patent acquisition deal. Such inclusions would strengthen the articles.

Odds are you are not working for Fox News so avoid the impulse to tell us how kittens are unwittingly killing us. Verify the hype surrounding novel discoveries through quotes from outside sources before reporting it. Excessive hype will only decrease clarity and perceived competency.

Engagement

Gotta have soul.

Gallagher defines the objective of science writing as, “To communicate knowledge that has been gained through scientific investigation in an accurate, engaging, and instructive fashion.” Accurate and instructive are easy enough for a diligent, detailed, and informed scientist but engaging is an art and something with which I struggle. The advice from Gallagher on adding soul into your science writing and keeping it vibrant is to include the human aspect of science. Did a novel discovery come about because of a researcher’s personal experience?  Who will personally be affected by a new regulation and can you include a quote from them?

Finally, a “hook” to bring readers in, perhaps by drawing a parallel to something universal or emphasizing the broad impact of your subject, is helpful.  Be careful with metaphors, though. See Philip Ball’s Nature article “A metaphor too far”.

Social media

Blogs, Twitter, and social media matter.

If you aspire to be a science writer you need to show an active interest and have an accessible example of your work. A blog meets both of those needs.

An active Twitter feed is equally valuable for showing your interests to potential employers in any field. Carol Stephen’s post “Twitter by the Numbers” provides some good guidance on how to manage your following to followed ratio. I use Twitter as a professional tool and manage my following numbers by not following my Facebook friends unless they maintain an active scientific/professional presence on Twitter. I also keep my tweets inline with my resumé and focus on professional development or interests listed on LinkedIn. As an engineer, I love congruence and clarity so keeping everything in my network reflective of and supportive of each branch is important.

And finally The Hook:

Why should busy scientists care about science writing and invest the time in improving? An article by Andrew Balas in the Journal of American Medical Informatics Association identifies the lag between significant discoveries in the lab and adoption into routine patient care as 17 years on average. Much of the burden to accelerate adoption falls on the discoverers and their colleagues to spread the information rapidly to a broad audience. If all scientists put forth the effort to improve their scientific communication and invest time in using instantaneous tools like social media, how much faster could people’s lives be impacted? This is why scientists should care. 

CRO Event: What contract research means to science and biotech

Images by Li Tai Fang

Contract research organizations (CROs) are becoming an integral part of the biotech research and development landscape, especially as big pharma continues to restructure its model for early stage discovery (January 16thpost), and everyone is taking this exploding market seriously.  However, the more we hear “CRO” the more confused some of us are as to what constitutes a CRO and how CROs fit into the research-sphere.  On Monday, January 27th, the Oxbridge Biotech Roundtable (OBR) Bay Area Chapter hosted “CROs: Partners and Disruptors in Drug Discovery” at the University of California, San Francisco to clarify some of this confusion.

An audience of nearly 200 registered attendees gathered to actively interact with a panel of CRO experts including Ken Meek, director of sales and marketing at Aragen Bioscience; Joe Francisco, Senior Client Services Scientist at Charles River Laboratories; Elizabeth Iorns, cofounder and CEO of Science Exchange; Jesse McGreivy, Chief Medical Officer at Pharmacyclics; and Kitty Yale, Senior Director at Gilead Sciences, Inc. The panel began by summarizing CROs as any entity willing to perform research for a second party for pay. The research includes everything from basic discovery to clinical trials. The panelists emphasized the importance of CROs to the biotech industry at all levels, but particularly startups and early stage ventures. David Rabuka, founder of Redwood Bioscience, did not attend the event but commented prior to his talk at the BioScience Forum event later in the week that CROs are essential to minimizing research costs and keeping startups viable. For the remainder of the organized discussion, panel members fielded questions from attendees including what are the drivers for the growth of CROs, of which increased regulatory challenges topped the list, and how is the perception of PhD scientists accepting jobs with CROs improving as CROs become more common.

As outsourcing increases, established CROs are growing to meet rising demand. Charles River, founded in 1947 on the shores of the Charles River in Boston as a provider of lab animals, has grown to locations in 14 countries, 25 locations in the US alone, offering services from toxicology to pre-clinical services. In addition, larger companies looking to expand their service portfolio are rapidly acquiring smaller providers. Aragen’s 50 or so employees are about to reach a larger market as Aragen’s acquisition by GVK BIO, an India-based CRO, was announced on January 29th. One attendee, perhaps with entrepreneurship on his mind, inquired what is missing in the enormous, but potentially crowded, CRO space? All the panelists agreed that providers of extremely specialized services are difficult to find. This is where the highly specialized research of academic labs becomes a commodity and Science Exchange is working to fill this need. Science Exchange puts consumers in touch with registered contract research providers. According to CEO Dr Iorns, several of the registered providers are academic labs looking to raise money through offering their niche services. Academic labs are realizing this is a perfect platform for fundraising to compensate for the deficits created by reductions in traditional funding. Furthermore, acting as a CRO avoids the complications surrounding crowdsourcing the biological sciences.

After the panel discussion, Sajith Wickramasekara, founder of Benchling, commented, “The increasing use of CROs by researchers highlights a greater trend in life science: an industry that’s embracing collaboration and specialization.” With greater collaboration comes the need for more formal data organization and sharing between team members. Benchling, which provides a cloud-based forum for analyzing and sharing DNA sequences and molecular biology tools is one of many startups capitalizing on these needs.

During the post-panel networking session, attendees, including a diverse mix of graduate students, postdoctoral fellows, and industry scientists, enjoyed food and drinks while making new connections. Benjamin Gaub, a PhD student at the University of California, Berkeley working to restore vision in blind animals through retinal reanimation as well as developing echolocation devices for the blind, appreciated the time as an opportunity to, “expand my network and get some exposure to industry.” It may be fair to assert that by brining people together for these events, OBR is providing a CRO-like service for networking. 

For more information on upcoming OBR events visit our site here