Peer review process

If it's on your mind and it has to do with multiple sclerosis in any way, post it here.

Peer review process

Postby BioDocFL » Sat Dec 08, 2007 3:35 pm

(This is somewhat off-topic from the original thread "EBV and MS" so I moved it out.)

Bob,

Regarding the peer review process: Arrr!!! Me thinks you have some misgivings about the pirates' code!

Some might think of it as a necessary evil, but overall it works well. What would be the alternatives? You have a boiling pot of good and bad science, big egos versus bigger egos, lots of time and careers invested with publication records being very important in the funding of future grants and a researcher's survival at stake.

The way the peer review process works typically is that the editors will ask 2-4 researchers in similar areas to read and critique the manuscript. Sometimes the reviewers are suggested by the authors, sometimes the authors will let the editors know of researchers who should not be included because of potential conflicts. Reviewers should let the editors know of any conflicts they might have in becoming involved. Usually the reviewers are kept anonymous from the authors' perspective. Some research areas are so small that it is often fairly obvious to the authors who reviewed their manuscript.

The reviewers typically have 2 weeks or so to read the manuscript, check reference articles listed and then submit their review. Their review should give a synopsis of the manuscript to show, in their own words, that they grasp the concepts. They should mention any points that need correcting or refinement. And the review will rate the manuscript on its importance and scientific merit, recommend whether or not the reviewer thinks it should be published, and whether there are major or minor revisions needed before the reviewer deems it worthy of publishing. It is rare for a paper to be accepted without reviewers wanting at least minor revisions. It is then up to the authors to make the requested changes, make a successful refutation as to why the changes are not needed, or withdraw their manuscript and try submitting it to a less important journal.

The process can be abused. A manuscript may be rejected and within the year a competitor, who may have been a reviewer of the first manuscript, gets something quite similar published. Did he/she steal the ideas, intentionally blackballing the original manuscript? That's possible. I've felt that happened to me once but life goes on. I doubt it occurs often.

There is the possibility that a reviewer does not agree with some underlying premise and harps on some reason to recommend not publishing the manuscript. You will probably see the authors writing a letter of refutation that is as long as or longer than their original manuscript because their ego kicks in when they see the reviewers' comments. That is why it is good to have multiple reviewers to avoid the spats. The editors of course are the final decision makers. The editors can also develop a positive or negative opinion of given authors over the years. In some tight-knit areas, editors (who are also authors and researchers) could potentially get too cozy with some authors.

For the most part, I think reviewers are fair and stick to logic to express their reservations about a manuscript. With the number of manuscripts that editors receive at prestigious journals, they are looking for reasons not to publish something so it is best for the authors to give their very best efforts in preparing the first submission.

I've been a reviewer (even on one or two of the articles mentioned in this forum). I have my own ideas about autoimmune diseases but I have to put that aside and review the manuscripts for the soundness of the work done and the conclusions that the authors make from their own work. I always ask for at least minor revisions and it is satisfying to see that they are incorporated. Reviewers usually get to see each others comments and the reviewers usually harp on the same points of concern.
Getting the reviews back can be a really maddening event for the authors or it can be a really big boost to the ego, depending on the ratings and comments.

One thing that is controversial is the authorship. Depending on the area (chemistry, biology, physics, etc.) the journal, and the priniciple investigator's ego, the order of authors and the listing of people as authors is very important. Usually, at least in biology, the person who did most of the work is listed first. He/she spent the long hours getting the experiments to work, perhaps input most of the technical ideas, and did most of the writing of drafts for the manuscript. They are perhaps the one working on a PhD or getting ready to apply for a grant. They are the one who needs the publication the most. The last author is usually the principle investigator in whose lab the work was done and who funded the work. Sometimes the big shot wants to be listed first since all future citations will be in the form 'Smith, et al.'. The other authors listed on the manuscript should have had some significant effort on the work. Fortunately many journals are now asking for submission of a form describing the contributions of each author to the manuscript. (Having an affair with one of the other authors or editors should not qualify!!!...say no more).

There is another process you alluded to: follow up discussion of published articles. This is usually the area of the 'Letters to the Editor'. After an article appears, you can write a short letter discussing a point with which you disagree or on which you want to elaborate. These letters also go through some review and the authors of the original article are usually asked to provide a response to the letter so the letter and response can appear in the same issue. Because of this, 'Letters to the Editor' may come three months to a year after the original article so they do not seem quite so timely. If it is a bigshot researcher and the area is hot, it could make it into the next issue.

Why do researchers agree to be reviewers? They can get some insight into what is going on in their area or related areas before the rest of the research community. They can also protect their own projects by rushing to publication something from their lab that is similar to a manuscript they are reviewing. In that case, they should notify the editor who may decide to publish papers from the two labs in the same issue. Many mentors will do reviewing to protect the projects of their graduate students, who really need a couple of original papers published to qualify for graduation. Getting scooped on something they spent 5-6 years on is really disheartening. Also reviewers are always trying to keep their sharpness at assessing good and bad science in order to improve their own writing skills, for manuscripts and grant applications. Maybe they think an editor will have some leniency towards publishing their articles if they help review others. That would certainly be getting into a gray area of ethics though. Personally I do it because I think it helps me in being thorough when I write my own manuscripts.

Peer review of grants is another animal, one with which I have little experience. There the reviewers get a lot of grants (maybe 20) they have to study in minute detail and compare. Then they have to meet and, as a committee, discuss and rate and prioritize the grants for the limited funding. Their recommendations then go to a funding review group that makes the final decision based on ratings of grant applications, available funds, and preferred areas of research according to guidelines from NIH higher-ups or whoever. Typically only about 10-15% of grants get funded. Manuscripts get accepted for publication at perhaps a 20-25% rate, depending on the journal. Some journals are tougher and will not even accept manuscripts without sponsership by a member of a prestigious science association.


Wesley
Last edited by BioDocFL on Sat Dec 08, 2007 10:46 pm, edited 2 times in total.
User avatar
BioDocFL
Family Elder
 
Posts: 251
Joined: Wed Aug 11, 2004 3:00 pm
Location: Florida

Advertisement

Postby Lyon » Sat Dec 08, 2007 5:33 pm

.
Last edited by Lyon on Thu Dec 01, 2011 6:23 pm, edited 1 time in total.
Lyon
Family Elder
 
Posts: 6063
Joined: Wed May 03, 2006 3:00 pm

Postby TwistedHelix » Sun Dec 09, 2007 7:06 am

Below is a link to one of my favourite web sites, " Bad Science", which is actually a column in the Guardian, (a British newspaper). I think it has relevance to this topic because it is a defence of the scientific method but be warned: fans of homoeopathy won't like it. Follow the links on the left hand side of the page to archived articles which deliver plenty of blows to the public misunderstanding of science and should, in my opinion, be compulsory reading in schools:

http://www.badscience.net/2007/11/a-kind-of-magic/
Dom
User avatar
TwistedHelix
Family Elder
 
Posts: 599
Joined: Fri Mar 25, 2005 4:00 pm
Location: Northamptonshire, England.

Re: Peer review process

Postby HarryZ » Sun Dec 09, 2007 8:17 am

Wesley,

Thank you for posting that excellent explanation of the peer review process. I wish I could write as well as you do!

I have a question that you may be able to shed some light on. In the past couple of years I have heard two neuros make the comment that Biogen/IDEC provides data to the medical community that is not exactly trusted. This, among other reasons, is why I am not exactly a fan of Biogen and how they handle their information process when it comes to MS medications.

In your experience within the peer review system have you ever heard similar comments about Biogen or other big pharma companies? Many thanks.

Harry
User avatar
HarryZ
Family Elder
 
Posts: 2441
Joined: Tue May 25, 2004 3:00 pm
Location: London, ON, Canada

Postby BioDocFL » Sun Dec 09, 2007 9:56 am

Harry,

Can't say I've heard or remember specific cases of fudged data from any big pharma companies. So I don't really have an opinion on Biogen.

I think it is the individual company's CEO and corporate culture, in pharma and other business areas, that can initially cause or sustain coverups of faulty products or hyping up products beyond what they can do. We're seeing problems in the financial markets now where companies are slow to own up to the bad debt (mortgages) on their books. Some companies make a full disclosure quickly to minimize the negative publicity, others don't, either through cowardice, lack of scruples, or ignorance.

For many decades the tobacco industry was able to get many articles published to dampen the negative image of tobacco use. There are researchers out there and journals who are willing to go with flimsy data and arguments to muddy up the waters of research. I don't know how they justify it within themselves but perhaps their motivation for entering medicine to help others was not so sincere. You meet all kinds in research. Fortunately, with the emphasis by the NIH and other organizations towards larger collaborative projects, there should be more informal and formal regulation of ethical practices within the collaborative teams, by their institutional internal review boards (who approve the projects, protocols, and standards), and by the journals. Hopefully there will be at least one person on a team who will question other team members on any ethical misteps, even if out of self-protection. Competition among journals to simply survive and even to become high-quality journals should lead editors to be ethical. Journals do develop reputations and an impact factor. Authors will want to publish in the more respected journals whenever possible. Sometimes you can see that a journal has a hidden agenda in what it publishes or is too closely tied to an industry, or is even a creation of an industry association.

One good thing has come out of the tobacco mess though. The state of Florida got a settlement from tobacco companies, I think it is about $200,000,000 to fund tobacco-related disease research. So every year or two around this time researchers apply for funding from this. I tried a couple of years ago but didn't get funded. Several of the researchers I work with did get funding.

Wesley
User avatar
BioDocFL
Family Elder
 
Posts: 251
Joined: Wed Aug 11, 2004 3:00 pm
Location: Florida

Postby HarryZ » Sun Dec 09, 2007 12:40 pm

Wesley,

I think it is the individual company's CEO and corporate culture, in pharma and other business areas, that can initially cause or sustain coverups of faulty products or hyping up products beyond what they can do.


Thanks for your input. I tend to share your ideas about how the CEO and other people high in the corporate organization direct their company. If you have your marketing/sales people hype a particular product and then it doesn't quite reach those predicted goals, I guess you can end up "skewing" the information to reduce the bad financial effect that can hurt your company in a huge way. Not that Biogen's top people would ever do anything like that :wink:

Harry
User avatar
HarryZ
Family Elder
 
Posts: 2441
Joined: Tue May 25, 2004 3:00 pm
Location: London, ON, Canada


Return to General Discussion

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users