View Full Version : Full disclosure from online, and even print, reviewers?
This subject has come up before, and again just recently, and it made me curious about getting feedback from the membership here.
When you're reading a review, does it matter to you if the reviewer paid for it out of his own pocket, got a discount, got the item on loan, or got the item for free outright? Do you think it should be made clear the nature of the procurement? How clear? and at what point in the review should they disclose this information and in what manner? Is there a difference between getting a $25 tourniquet for free and a $1,000 rifle? or a $3k custom 1911? If so what's the break point?
In case someone reading this doesn't know, I write gun & gear reviews for my website, and have also written for SWAT and Guns & Weapons for LE, and have several articles pending with both publishers. I'm asking these questions both to see if other consumers share my view on these things but also to better construct the reviews I do based on what other consumers want to see.
Personally, I dont really care how he got the item for review, but I do think reviewer should disclose how he got the item being reviewed. I am sure the amount of bias varies dependant upon method of procurement. In my search for the truth about firearms I have come to rely on someone elses reviews less and less (a select few members here being the exceptions).
I think the price point does matter.
A less expensive item is cost prohibitive to ship and handle, why bother dealing with it ? just keep it.
The only purpose behind a manufacturer "donating" an expensive item is to hope to obtain a favorable review, with minor things being overlooked or down played.
I dont know the paperwork involved with either donating a weapon as opposed to returning it, but considering how often its done for the gunrags, I cant imagine it being bad.
just my humble, amature opinion.
VooDoo6Actual
06-19-10, 09:59
I think full disclosure is always a good thing.
I see no down side.
I can filter the agendas, nuances at my interpretation & discretion from there.
Seen so many corrupt people through the years. The list is LONG, very LONG.
Belmont31R
06-19-10, 10:10
There can be bias in any review regardless of how the item up for review came into the reviewers hand. With that said it does not look good if the item was free, they got to keep the free item, item was sent directly from the manufacturer, etc. At any rate how the item was acquired, no matter how it was acquired, should be said from the start along with any affiliation with the company.
They should be purchased blindly, IE the company has no idea which actual product of theirs was going to get tested. It doesn't do any good to get items directly when the company knows this item is going to get tested because they are going to send the best not necessarily what everyone else is getting. They're going to give that individual product a little better attention before sending it out. Not in ALL cases but the majority would.
Im not talking about anyone specific but seems like some people in this industry are "professional" reviewers, and you rarely hear anything bad about the products they review. Why would you when you are getting thousands, probably into the tens of thousands of dollars in free shit? If you want that gravy train to keep flowing you don't write bad reviews or at least fully honest ones.
I also don't call these short round count firearm reviews as anything but commercials or giving "props" to a company. I appreciate the fact a few companies out there are sending their guns out to get tested where they are going to get 10k, 20k, maybe 30k plus rounds through them. I dont call a few hundred rounds anything but a familiarization shoot. Even the crappiest of manufacturers can probably cherry pick a gun, and get it to go 1-2k rounds. Thats not a tall order to fulfill...
For someone like me I read people's experiences with guns, and put them together like a puzzle. Put enough pieces together, and you get a picture you can look at.
Sad thing is I think it depends on who is reviewing it. I am fairly confident that, for example, if Rob_S reviews something, whether he paid for it, got it for free, or was paid to take it, would write an unbiased review to the best of his ability, and there are many other members here on M4C that I trust very well to write an unbiased review, no matter how they got it.
But with less known/trusted/reliable sources, I'd like to know how they got the gear and what their connection is to the manufacturer, if any. Not everybody knows who can be trusted to write good reviews, so I think it would be good practice across the board;full disclosure.
I think full disclosure is important, but just because a guy paid for something with his own money doesn't guarantee an objective review. That reviewer is likely to be subjective about the widget as they want to be able to justify their purchase.
Nevertheless, full disclosure is important for the reader to be able to properly discern whether the review is worth anything. I assume any review where it's not clear how the author got his widget that he obtained it for free from the manufacturer.
Army Chief
06-19-10, 11:02
Credibility is the currency of the reviewer's world, and while the terms associated with a particular T&E component may be relevant, they really don't shape my opinion one way or the other.
I've got a number of T&E projects underway right now in conjunction with the "Summer of 6.8 Project". Most of the components involved were purchased outright. Some were discounted as a courtesy. A few were provided as loan items, and one or two were outright write-offs. How I look at one versus another isn't really colored by my level of personal investment, but rather, by how well a particular component meets my needs.
This is easy for me to say, of course, because my purpose in conducting these reviews is simply to glean and share information with a group of like-minded enthusiasts, both here and at WeVo. If I have to make a hard call and label something substandard, there are no broader implications beyond my own personal disappointment. It isn't going to upset an editor. It isn't going to result in lost advertising revenues for any publisher. It isn't going to limit my opportunities as a compensated author in the future.
Where folks run into trouble, I think, is when they are writing -- and being directly compensated for -- publication. It is a much more difficult thing to walk the line of credibility when you are handed the new ACME piston AR, and asked to write a review for a periodical when you know that your publisher is friends with ACME's owner, and/or that ACME has already reserved a $3,600 full-page color ad that will run on your review's facing pages. In such a situation, it is nigh unto impossible to come back and say "the ACME piston AR is probably the most ill-conceived and amateurishly-executed pack-follower of the current lot of piston guns."
You get the idea. When we're putting a product through its paces here, for our own purposes, it makes comparatively little difference if it was begged, borrowed or bought, so long as the reviewer is known as a straight shooter. Add the element of "I'm being compensated for this in cash money," and that dynamic can quickly change, which is why I tend to agree with publications like SWAT that insist that authors make an investment in the equipment they are being put under contract to review.
Does a note to this effect (i.e. ownership) belong in your articles, Rob? I really don't know, and given what I have seen of your writing to this point, I really don't get the sense that it would make any difference. If you are reviewing gear that you made a personal investment in, that might be an appropriate thing to note, but if it is a loaner/write-off, I'm far less interested in the terms of how you came to acquire it than I am in what you have to say about it.
AC
I can filter the agendas, nuances at my interpretation & discretion from there.
The problem is I firmly believe most people cannot filter any type of agenda if their life depended on it. They are sheep and righteously submit to group-think.
There are two lines of thought on this (online or print, I believe is irrelevant):
1. The professional reviewer - In the interest of propriety, this person should not only always offer full disclosure, but do so in a manner that describes the way in which their reviewed products are procured AND the way in which they review similar objects require the utmost consistency and objectivity. Every BUIS should be subjected to the same considerations and tests, etc., etc. The professional reviewer should conduct his or her reviews as scientifically as possible, thus guaranteeing internal validity as much as possible. I absolutely understand that this is easier said than done. I mean, when was the last time you read a "bad" gun review in a magazine? I can think of none in recent memory. I admit I haven't read any of Rob's reviews.
2. The casual reviewer or hobbyist - he is obviously less submissive to objectivity, and reviews the item according to his own uses, not the public-at-large. This reviewer is "what you see, is what you get." Full disclosure is obviously optional, but the "why" of procurement should be explained, versus the "how."
The best example of objective reviews I have ever seen is found at http://www.chasingame.com/. The reviewers on this site review trail or scouting cameras used for wildlife scouting, viewing, hunting, and even security surveillance.
Army Chief
06-19-10, 11:23
Great post, kry226.
AC
One can rarely be accused of a covert bias if they have exercised full disclosure. If any degree of bias is involved or detected, total credibility is lost.
Translation: Just tell the full truth as it relates to your review and let others decide for themselves if you are biased or not.
That said, I'm not prepared to implicate that just because someone is given an item for review (which they may keep) they will do a biased review. However, if the item is a $3,000 1911 as opposed to a 12 round .40 magazine, the tendency to be swayed is likely to increase. It's a simple dynamic.
Great post, kry226.
AC
Thanks, Chief. You too. :)
this is kinda relevant :p
http://i260.photobucket.com/albums/ii27/bilylovec/chart/1244955571487.png
I want to know if the guy writing the review is a rep for S&W before I get his opinion on whether or not I should buy a S&W product.
If you've been given free shit or have any connection with the manufacturer of said product you should disclose such information.
I want to know if the guy writing the review is a rep for S&W before I get his opinion on whether or not I should buy a S&W product.
If you've been given free shit or have any connection with the manufacturer of said product you should disclose such information.
Just because you have received something for free to review by no means indicates you are in any way a designated rep.
Curious as to how you arrived at this conclusion.
Just because you have received something for free to review by no means indicates you are in any way a designated rep.
Not saying you are, just saying it should be disclosed. I treat anything written by a rep like an ad. Anything some one writes for a company that gives him or her freebies, I treat a little more cautious than a review by some one who payed hard earned money for it, they're more likely to point out the cons when they feel cheated.
Not saying you are, just saying it should be disclosed. I treat anything written by a rep like an ad. Anything some one writes for a company that gives him or her freebies, I treat a little more cautious than a review by some one who payed hard earned money for it, they're more likely to point out the cons when they feel cheated.
Completely agree, in fact your words above essentially mirror mine in this thread.
But then again, your prior wording does somewhat implicate a reviewer who obtains something for free as a rep. If you meant to separate the two, unfortunately you didn't do it effectively.
Certainly no big deal, though.
Nathan_Bell
06-19-10, 16:45
Full disclosure. Do it up front in the lead-in verbiage.
"We received this rifle from Blimpy's Blasters for review. We bought it through Jerry's Sports (a firearms distributor) for $1,200, in order to avoid any chance of a ringer"
Or
"We spoke to Mike at SHOT show and told him we would like to review his newest pistol. When we got back from Vegas we had a package at the FFL. No invoice or billing information, but after a call to Mike he explained he would like to have the rifle back by March 28th"
If it is a use item, socks, gloves, etc. It is assumed you will keep that type of stuff. TO avoid looking like you are accepting gear for good word, put a long term section for folks to see how the socks are doing a year after the initial review.
Being up front in that manner while not making folks turn the page is a challenge, but that type of forthrightness is what i like to see.
Another vote for full disclosure. Method of procurement and any consideration given. Financial or business interest the reviewer has in the product/vendor/manufacturer should also be disclosed.
Yes, full disclosure, and cost matters.
Incentives matter.
http://www.ftc.gov/opa/2009/10/endortest.shtm
http://www.ftc.gov/opa/2009/10/endortest.shtm
Most interesting part of that document
the post of a blogger who receives cash or in-kind payment to review a product is considered an endorsement.
RAM Engineer
06-20-10, 07:34
Just because someone spent their own money on something doesn't guarantee impartiality. Imagine someone who blew their cash on something only to find out it's a dud. They might either gloss over flaws or not write a review, which in it's own way skews the "database" of reviews for that item. That guy on here who wrote of his dissatisfaction with his Taurus is a good example of the ideal that all of us who aren't pro or semi-pro reviewers should follow.
Army Chief
06-20-10, 08:02
Very true.
Though I would like to think that professionally-minded reviewers could avoid this kind of bias, there is definitely a tendency for amateurs to blindly espouse the virtues of Product-X, partly because they are seeking validation, partly because they have never seen/used/owned a competing product, and partly because they don't want to look/feel like idiots for making a bad purchase decision.
AC
There is no 'perfect' solution. People who buy an item often have a problem about looking at that item dispassionately. We see it all the time on here with folks defending their Oly, DPMS, whatever rifles. Some items I buy, some are sent to me, there really isn't a set pattern to it. A few items i actually keep, but many I donate (either give the item to a deploying soldier or sell it and give them money to a mil-focused charity such as HATM, WWP, or AS). Knowing that an item isn't staying keeps me neutral on it.
Conversely, I'm still shocked at how far companies will go to suppress negative (but true) information about their products.
I think the tendency of the "amateur" to be enamored with a new purchase or hide flaws in a new purchase can be mitigated by another type of disclosure: prior experience. A guy that already owns 3 Colts & two BCMs is much less likely to buy an Oly to begin with, but if he does he is also much less likely to cover up any flaws as he really has no reason to. Conversely, a guy that buys an Oly as his first AR with zero prior experience is FAR more likely to not only lack the experience to determine what he should be reporting but if also more likely to have ego attached to the purchase and be unwilling to expose his bad choice.
Very true.
Though I would like to think that professionally-minded reviewers could avoid this kind of bias, there is definitely a tendency for amateurs to blindly espouse the virtues of Product-X, partly because they are seeking validation, partly because they have never seen/used/owned a competing product, and partly because they don't want to look/feel like idiots for making a bad purchase decision.
AC
The cold, hard fact in all of this is that there is no such thing as pure objectivity, or non-bias. It is a myth. We all go into a situation as a product of our own upbringing, and socialization and experience. How that experience has shaped us, over time, determines how we define our own perspective about whatever it is we are attempting to study or label. Which makes full disclosure all the more important.
full discloure on all details
if I decide I like something I usually seek out more info but want honest of why someone got something ?
variablebinary
06-20-10, 15:33
I dont care. I just look at the pics with occasional technical detail.
There are plenty of hacks and shills on internet forums. Even more than formal reviewers.
What can you do? Cry over it? "ZOMG, he got a free gun!". Meh. I got better things to waste my brain power on. This line of thinking goes deep. Consider all the larger forums are funded by gun manufacturer dollars. Does that make site owners guilty of impartiality?
99% of the time people have their minds made up about product X anyway. Very few people "grow", even when presented with credible information. They buy it, and it becomes the greatest thing ever. End of story.
On the upside, truth has a way of always finding its way to the surface. A product that is shit will never be able to hide this fact forever. The greatest reviews are collective, not the product of any one person's words
I dont care. I just look at the pics with occasional technical detail.
There are plenty of hacks and shills on internet forums. Even more than formal reviewers.
What can you do? Cry over it? "ZOMG, he got a free gun!". Meh. I got better things to waste my brain power on. This line of thinking goes deep. Consider all the larger forums are funded by gun manufacturer dollars. Does that make site owners guilty of impartiality?
99% of the time people have their minds made up about product X anyway. Very few people "grow", even when presented with credible information. They buy it, and it becomes the greatest thing ever. End of story.
On the upside, truth has a way of always finding its way to the surface. A product that is shit will never be able to hide this fact forever. The greatest reviews are collective, not the product of any one person's words
I don't want you to take this the wrong way, but...
I find your point of view interesting coming from you. I have never been clear on your relationship with Robinson Armament, what you paid for vs. got for free, etc. and now I can't help but wonder if the above position isn't influenced by that. It's almost a tail-wags-dog situation, depending on the particulars of that relationship.
variablebinary
06-20-10, 16:16
I find your point of view interesting coming from you
What point of view is that? That the best reviews are collective, and no one person's opinion should ever be gospel?
Just so I understand, you take issue with that? I want to be clear before I bother with a more substantive retort...
No, the fact that you said you don't put any value on knowing how an item for review (bought, loaned, given) came into a person's possession.
unless I misunderstood when you said "I dont care."?
Hence, having always wondered about, and never been clear on, the nature of your relationship with Robarm, I can't help but wonder if that relationship isn't what's dictating your opinion of whether or not acquisition method matters.
In other words, if you have been getting free stuff from them all along, I would expect no other response from you other than "it doesn't matter how the reviewer gets stuff", but I'd also want to know the particulars of that relationship in order to qualify your opinion and decide if it has any merit for me.
The part about the collective being the best way to evaluate a potential purchase I agree with, FWIW. But that's not what I'm talking about in this thread.
variablebinary
06-20-10, 17:00
No, the fact that you said you don't put any value on knowing how an item for review (bought, loaned, given) came into a person's possession.
unless I misunderstood when you said "I dont care."?
Hence, having always wondered about, and never been clear on, the nature of your relationship with Robarm, I can't help but wonder if that relationship isn't what's dictating your opinion of whether or not acquisition method matters.
In other words, if you have been getting free stuff from them all along, I would expect no other response from you other than "it doesn't matter how the reviewer gets stuff", but I'd also want to know the particulars of that relationship in order to qualify your opinion and decide if it has any merit for me.
The part about the collective being the best way to evaluate a potential purchase I agree with, FWIW. But that's not what I'm talking about in this thread.
I don't care. If I did I'd have to start wondering how the paycheck from Group One Enterprises or editors of play a role in the content of your writings. Maybe it extends to "Tacticalyellowvisor".
Maybe something. Maybe nothing. Thankfully your solitary opinion, paid for or otherwise, is largely irrelevant without being counterbalanced and affirmed by a much larger body of users.
And I don't get any perks, are we clear. Bookmark this page for future reference and don't question my integrity again
And I don't get any perks, are we clear. Bookmark this page for future reference and don't question my integrity again
Everyone with these integrity concerns these days. Makes one wonder why they aren't more up front with their situations in order to keep those questions from arising...
All I did was wonder what amount of the Robarm stuff you trot out and endorse you got for free vs. actually buying. If you say you buy all of it then that's it. No "questioning your integrity". To be clear, that IS what you're saying, right? That you pay for all of it and there are no freebies?
Don't be so sensitive.
To attempt to state this more clearly...
My initial response to you was intended to suss out more information in order to decide whether I valued your opinion or not.
Much like I would ask someone "how did you receive this item: purchase, loan, or gift?" in order to further weigh their opinion. I wanted to know if, in fact, you were receiving items for free so that I could in turn weigh your opinion about reviewers disclosing their procurement method.
If, in fact, you had been receiving all of these Robarm rifles and such for free, then I would place little to no value on your opinion of the practice as you would obviously be biased. It would be like asking a pedophile his opinion on statutory rape laws.
You say you get no freebies and you pay for all your gear (at least I think that's what you said) and I take you at your word (something I wouldn't do if I thought you had no "integrity") and hopefully that's that.
Bookmark this page for future reference and don't question my integrity again
http://www.theospot.net/pictures/misc/francis.jpg
:rolleyes:
Since you rah-rah RA a good bit and have never, until now, clarified your involvement with them I wondered myself.
variablebinary
06-20-10, 18:34
Since you rah-rah RA a good bit and have never, until now, clarified your involvement with them I wondered myself.
My involvement is clarified by default for anyone capable of passing a 4th grade literacy exam
5) Industry Disclosure – If you are an industry manufacturer/dealer or employed by one you must disclose this relationship through your screen name or signature line. Anonymous industry trolling will not be allowed.
My involvement is clarified by default for anyone capable of passing a 4th grade literacy exam
:rolleyes:
My involvement is clarified by default for anyone capable of passing a 4th grade literacy exam
You seem to have a problem w/answering simple questions.
Do you get free shit or not?
I don't care. If I did I'd have to start wondering how the paycheck from Group One Enterprises or editors of play a role in the content of your writings. Maybe it extends to "Tacticalyellowvisor".
I've gotten checks for articles published in SWAT which reported among other things:
FS2000--pointed out function problems that can arrise from putting a mag in with the bolt locked back when the bolt sometimes dropped forward but did not go all the way and any attempts to clear the gun using the bolt resulted in a doublefeed.
American AUG article: both the TPD variant and the MSAR variant had to be returned to the factory--the TPD to be checked because of intermitant problems; the MSAR would not fire more than 20 rounds without a malfunction.
US made Steyr AUG a3--charging handle very stiff to operate with little improvement from wear.
HK45--groves on inside of trigger guard wacked the trigger finger during recoil of more than a third of test shooters.
Don't assume that getting a check means biased reviews or free guns.
However, given some glowing gun reviews in other publications of less than glowing guns, I can certainly understand how someone might think this could be the case.
My involvement is clarified by default for anyone capable of passing a 4th grade literacy exam
Yet again you dance around the question asked of you without answering it. Your refusal to actually answer the question says a lot about your integrity.
variablebinary
06-20-10, 21:19
Yet again you dance around the question asked of you without answering it. Your refusal to actually answer the question says a lot about your integrity.
Interesting
I don't get any perks
http://cruddy.files.wordpress.com/2009/07/derp-derp-derp-dog-500x3751.jpg
Maybe we can try this thread again. If you don't want to answer questions about your involvement with one entity, or another, maybe it's best to remain outside the new thread.
Ok?
Good.
Powered by vBulletin® Version 4.2.5 Copyright © 2024 vBulletin Solutions, Inc. All rights reserved.