Friday, May 1, 2009

The New York Review of Each Others' Books

Over a decade ago, a colleague gave me an anthology of academic animal behavior papers dealing, allegedly, with issues of interest to companion animal trainers and owners.

Two editors. Two dozen or so contributors, including nine writing solely on cats, primates, or horses. The two editors are authors of twelve of the thirty papers.

And then there are the citations. Wilhelm cites Bertha who cites Inga who cites Wilhelm who cites Inga who cites Bertha. A regular Polish firing squad of citations.

Very cozy little club there.

See, here's how it goes in the darker corners of some universities. (And in private practice in certain fields, as we'll see.)

You submit your third-rate research to a third-rate, low-circulation journal in your tiny little area of specialization.

Make sure you copiously cite the work of the editor of the journal, as well as the work of every possible reviewer. Since the field is so small and unpopulated, and the whimsy of the editors is pretty easy for an insider to discern, it's not hard to guess who the reviewers will be.

The reviewers and editors have a vested interest in goosing their own cite stats. The editor needs to fill four issues a year of journal that may have fewer readers than contributors. That paper is gonna get published. Sure, a megalomaniac editor or reviewer may effectively re-write the thing, but it gets published with your name on it. And cited in the next round. The journal becomes a little credentialing vehicle in the battle for tenure, or just another semester's appointment as part-time visiting adjutant lecturer of thus'n'such.
(1)


And don't forget to cite yourself -- everything you've ever published. And one or two of the ponderous "classic" texts in the larger field that technically encompasses. The citations can be as gratuitous as you like. No one will check.

And what do you get in the end? A couple of dozen (at most) sub-sub-field insiders sitting around vigorously agreeing with one another, without having to perform any really original or difficult work, apply the scientific method with rigor, or meet broad peer review standards that are, well, standard in the scholarly world.

I can't tell you how many current and former academics, in fields ranging from the hardest sciences to the squidgiest humanities, have read what passes for a "paper" in the field of "companion animal behavior" and bitterly proclaimed themselves to have chosen the wrong discipline. Why did we sign up for all that hard work?

Case in point: the single paper in the anthology mentioned above that I found wildly interesting, provocative, rigorously designed, and potentially useful in a practical way is also the one whose author doesn't cite the editors or any of the other usual suspects in the rest of the volume, and is not cited in any of them. It seems to be well-embedded in a larger and more results-oriented field. (Though the author does rather copiously cite his own previous publications.)

Which brings me to the present-day, and what passes for "research" in the politicized world of "companion dog behavior modification."

This used to be called "dog training." Some hominids (2) still cling to this quaint terminology. (1542 -- "subject to discipline and instruction for the purpose of forming the character and developing the powers of, or making proficient in some occupation." Some hominids own the compact OED.) Its practitioners were judged by their success in "forming the character" and "making proficient," sometimes in formal tests and contests.

Such outcomes are now proclaimed irrelevant. Dog training has become politicized in the last decade to an extent that is unbelievable to an outside observer -- most particularly, the interested outside observer, the dog owner who just wants some advice on getting the damned dog to come when he’s called/pee outdoors/eschew leg-of-postal-worker. One large faction of the self-proclaimed revolutionists (let’s call them the Judean People’s Front) has taken up the mantle of “science.” Some of them even have university affiliations and postgraduate degrees in fields such as veterinary medicine and the sexual behavior of lizards.(3)

Why is this important? Because real dogs, and real owners, are looking for help. Because pseudo-science -- ideology cloaked in the vestments of science, but without its substance -- is being aggressively promoted as the only way for them to get that help, while the grunting empiricist hominid trainers are discounted by both pseudo-scientists and the media. Because the outcome of not getting effective help ranges from the tragedy of a dog whose full potential for achievement and dignity is squandered, to a dog who is killed because of training omissions, to a dog who seriously hurts a human because of training omissions.

The conclusions made by the researchers are not hiding away in those dark corners of universities and moldering in the pages of the seldom-read journals, as they would be if they dealt with, let’s say, philology. Because they purport to be directly relevant to human relationships with their pets, huge numbers of ordinary people are keenly interested in what they “prove.” An army of journalists, veterinarians, and non-academic “trainers” filter the conclusions in progressively simplistic ways before proclaiming Truth to their pet-owning clients. Most never read the original paper; those who do are frequently complicit in concealing the essential inadequacies of the “data” that allegedly supports the widely-reported conclusions.

Studies show ...

Rather than justifying their academic appointments or private practice fees by rigorously measuring the results of their interventions and objectively demonstrating the efficacy of their methods, (y'all can do that, right?) the Clan of Three Letters (PhD, DVM, JPF) now goes a huntin' for "data" that discredits the grunting, OED-reading, leather-leash-wielding, suspected Mexican hominids.

But the habit of intellectual sloth fostered by the ongoing sub-field circle-jerk means that the published "studies" don't hold up to even casual scrutiny from outside the circle. Not that alleged journalists, including "science" journalists, bother with even that in those few instances where the propaganda comes to outsider notice. Ralphing up a press release as "news coverage" is so much easier than reading the paper, much less applying basic logical analysis to the study design(4), methods, data analysis, and conclusions.

Hey, the journal already did all that, right? Dis here is peer reviewed.

See above.

In the next few months, Raised by Wolves will scrutinize a variety of papers and pronouncements from researchers and professional bodies that claim to provide solid data about dog behavior and how to change it through training. We’ll read and analyze the papers themselves, not the press releases or the gormless media buzz. We’ll hold the authors to the standards of objectivity, rigor, and evidence that any scientist should expect and welcome.

And we’ll always remember that This was in a journal, it must be credible is not a valid argument.



(1) Not that I am not positively oozing sympathy for the plight of any young academic seeking his or her sinecure, or just a paycheck for the next semester. You do what you gotta do, and geniuses must dance the dance with the hacks. The publish or perish two-step mentioned here is not one-one-hundredth as humiliating as the ass-kiss limbo. That said, it is just as much work to produce and publish crap as it is good stuff.

(2) Some hominids also cling to the word “hominid,” which we are now told is more properly “hominim.” Which is a homonym for homonym. Which is just too much for our tiny stone-chipping brains. And my spell-checker still rejects it, so there.

(3) No, I am not mocking the achievement of earning a degree in veterinary medicine, nor am scoffing at basic research into animal behavior. I am not a Republican, I totally get it about the volcano monitoring and fruit flies. But neither credential speaks to the problem of “How do get my dog to stop doing the things that drive me crazy, and consistently do the (reasonable) things that I expect of him?”

(4) I was forced kicking and screaming to take a class in methods (aka sadistics), despite the fact that my own post-graduate field of study was dangerously close to “humanities” rather than the “social science” that would be named in the degree -- which I did not complete. The professor for the class was mediocre, but his assistant was fantastic, great teacher. That methods class is the single most useful positive lesson I took away from six years of hazing.

15 comments:

  1. Excellent post. I look forward to the future fisking of the fascist forces of behavioral philosophy.

    (and re: note 1 - the "ass kiss limbo" is why I left grad school without three letters after my name. I decided that I preferred to kiss less ass for more money, and went into consuluting - where my income immediately exceeded that of my asshole advisor.)

    ReplyDelete
  2. This looks like fun.

    It's funny, I spend so much time trying to convince people they SHOULD listen more to scientists instead of their hairdresser. I sometimes have a harder time voicing criticism about publications because some take that to mean you can't trust ANYONE in the Ivory Tower League.

    ReplyDelete
  3. I'm looking forward to posts on this topic.

    I still read these sorts of articles and studies and books, but mostly to stay up on the truly stunning questions that random dog-owning friends come up and ask me after *they* read these things.

    Oy.

    Oh, and I'm in the People's Front of Judea. Judean People's Front! Splitters!!

    ReplyDelete
  4. I take it it has been a long winter? Has the snow began to melt in PA, yet? ;-)

    This looks promising, from the front, er, beginning. I, also, look forward to learning how to glean information from amongst the chaff.

    eli

    ReplyDelete
  5. CyborgSuzy, I feel your pain.

    "Oh these scientists, they say one thing, then another one says another -- I don't think they know anything ..."

    Contrasted with:

    "Herr Professor Famous applied physicist says that black folks is inferior. He's a scientist, so it must be true."

    Aaargghh!

    I attribute it to the truly dismal "science education" in this country -- which has been dismal since forever, and doesn't show any sign of improving.

    Because facility with technology is NOT the same thing as a scientific education.

    ReplyDelete
  6. quoting Heather:

    "I attribute it to the truly dismal "science education" in this country -- which has been dismal since forever, and doesn't show any sign of improving.

    Because facility with technology is NOT the same thing as a scientific education."

    SOOOOOO true.

    ReplyDelete
  7. Ha ha ha! Absolutely dead on. There are a lot of 'research' papers about dogs that are written by has-beens or wannabes alright. And the media guys? Perfectly captured. They wouldn't know how to critically appraise a paper, even a very basic one, if their lives depended on it.

    Can't wait to read more.

    ReplyDelete
  8. So, THAT"S why they hate Senor-Who-Must-Not-Be-Named so much. He really is a threat to their little emperor's clothing store.

    I ain't gonna miss a post on this series.

    ReplyDelete
  9. Salivating in anticipation of the next installment.

    And no, no one rang a dinner bell...

    ReplyDelete
  10. Being one of the great unwashed 'dawg-trayners' who does not prescribe to the lab-rat, lemon-brain philosophy of the white lab coats and jewel encrusted degree programs, I look forward to this with the greatest of interest!

    ReplyDelete
  11. Can we request articles to be reviewed? I went through the AABS journal a while back and have a few that I'd like a seasoned REAL dog pro to look at. Since I'm going through a psych course where we have to read and cite multiple studies in our own papers for our own studies, I'm getting a bit nitpicky on some of the behavioral stuff I'm reading.

    I'll definitely be waiting in the wings to read what's coming up! (And this is pretty much why I'm skipping grad school and going straight into professional trainer school!)

    ReplyDelete
  12. I got meself one o' dem dare three letter configurations behind my name, and a stack of rejection letters from first-, second- and third-tier journals uniformly affirming that I *would have been published* if ONLY I'd cited the work of the (anonymous to me) reviewer.

    Meanwhile, it's been the folks with six letters behind their names (DVM, PHD) who've told crowds of rapt behavior-wannabes that 'dog trainers should go to prison for what they do to dogs', and that's a direct quote.

    Looking forward to the literature review.

    ReplyDelete
  13. My own experience in graduate school is very similar to what you describe. And I'm not sure what your discipline was, but in the social sciences, it helps to be a number cruncher and not have a clue about what is actually happening.

    Academia where lots of people who never grew up go to hang out and earn a living. Not all academics are middle school students in their mentality, but far too many are.

    The behavioral revolution in the social sciences is one thing I'd like to see disappear. It's literally a "God that failed."

    Now, you and I have different philosophies on dog training, but I'd be a fool to tell you that dog training is a true science. If Karl Popper saw half of the things posited by the experts in animal behavior, I believe he'd have at least cardiac event. Falsifiablity--that's your main scientific principle, not butt-kissing!

    ReplyDelete
  14. LOL Nice job. I remember when experience out ranked degrees--in fact a "special major" was what was recommended to me.

    I ended up taking my experience into a private course (hands-on with academics) and then to the only established animal training program around.

    It has been amusing to see papers presented at conferences where the researcher has re-labeled terms so that they could present something "new."

    One of my colleagues is an academic, and although he can deal with a variety of issues successfully--he sucks at people skills.

    Ultimately, I think the pendulum swing is going to level out somewhere in the middle where experience and academics (or education--since you can also get that outside of college) will both be valued and we will get away from the "us and them" mentality and get to the "and both" frame of mind.

    ReplyDelete
  15. will this result in a book written by HH? I think it should.

    eli

    ReplyDelete

I've enabled the comments for all users; if you are posting as "anonymous" you MUST sign your comment. Anonymous unsigned comments will be deleted. Trolls, spammers, and litigants will be shot.