Better Call Paul | How Do We Choose and Read Research Papers?

In this episode, the third of five special Better Call Paul episodes, Kris and Paul discuss the process they use to choose and read research papers, both for Breaking Beta and for their own interests as climbers and coaches.

They talk about the differences between Primary Research and Review articles, and which type makes more sense when it comes to climbing-specific studies. They note the importance of choosing papers that come from reputable sources, and provide some resources to help determine whether or not a source is reputable. They break down how they actually go about reading through a paper, and discuss if and how this varies for them when they are looking at a paper for the podcast or just for their own interests.

*Additional studies/resources mentioned in this episode:

Beall’s List of Potential Predatory Journals and Publishers

How to (seriously) read a scientific paper by Elisabeth Pain; published on Science; March 21, 2016.

Ten simple rules for reading a scientific paper by Maureen A. Carey, Kevin L. Steiner, and William A. Petri Jr.; published on PLOS COMPUTATIONAL BIOLOGY; July 30, 2020.

How to read and understand a scientific paper: a guide for non-scientists by Jennifer Raff; published on Violent Metaphors; August 25, 2013.

How to read scientific papers quickly (and effectively organize them for a literature review) published on Genius Lab Gear.

---

New episodes of Better Call Paul drop on Wednesdays. Make sure you’re subscribed, leave us a review, and share! And please tell all of your friends who are confused and overwhelmed by the amount of jumbled and conflicting training info out there, that you have the perfect podcast for them.

Got a question? Comments? Want to suggest a paper to be discussed? Get in touch and let us know!

Better Call Paul | Breaking Beta is brought to you by Power Company Climbing and Crux Conditioning, and is a proud member of the Plug Tone Audio Collective. Find full episode transcripts, citations, and more at our website.

FULL EPISODE TRANSCRIPT:

Breaking Bad Audio Clip  00:05

Yeah. Tell him I'm in a conference.

 Can't you see I'm in conference? 

I'm looking to find a client of yours. I was hoping you could help me out with an address.

Which client?

Jesse Pinkman.

Jesse Pinkman, in the phonebook, Jesse Pinkman? Mmm, how would one track him down, I wonder? 

I need your full attention, Saul. Jesse Pinkman current whereabouts. The sooner you tell me, the sooner you can get back to whatever the hell this is exactly.

What am I, eighth grade hall monitor? Current whereabouts? Let me tell you something, Mike.

Whoa. You are good right there. Now let's both get comfortable.

Kris Hampton  00:54

Finding the answers that you want isn't as simple as just looking it up in the obvious place. It requires your full attention, some critical thinking skills and in this case, definitely getting comfortable because it might take a while to find the answers you want. This episode is going to be all about our process and and the process some others use as well, of choosing and reading papers. This is one of the common questions that I heard from folks, like "How did you choose which papers?", "How do you you know, you've you've made it known that you don't really like reading research, so how do you go about it?". And, and we do this both for the show, and for our own general knowledge as coaches. I think both of us have, have spent some time reading research. You probably a lot more than me, over the years.

Paul Corsaro  01:47

Quite a few, that's for sure. 

Kris Hampton  01:48

Do you enjoy reading research?

Paul Corsaro  01:52

I enjoy what comes out of it. But I don't think it's something that like, you know, I'm excited to sit down like with a cup of coffee and like skim through real quick and just, you know, knock out three or four papers at a time. Like it's a very active process. 

Kris Hampton  02:04

Yeah. 

Paul Corsaro  02:05

And if you're not going into it with that intention, and maybe trying to struggle and struggle to focus a little bit and tease out some of this information, you're not going to get a lot out of it that you need.

Kris Hampton  02:15

 Yeah, I agree. For me, it's like, I enjoy the the brain downtime that it sort of requires. And I don't mean that in like you're not using your brain, but I'm just using it in a different way than I do most of my work and my climbing throughout my general day. It's like I'm focused on this one thing and I'm trying to ask intelligent questions. But it's a mental shift for me and I have to be fully absorbed by that thinking and reading about this one thing. I certainly wouldn't choose it as like my leisure time activity, but I do enjoy that part of it.

Paul Corsaro  02:58

Oh, for sure. And like it's almost a good point of focus. Like I'll put my phone away like away from me, so I can't be distracted. So it is nice to have that time to really just think about one thing and really lock in on something.

Kris Hampton  03:10

Yep, totally. I agree. All right, you're ready to get this thing started.

Paul Corsaro  03:15

Let's do it. 

Breaking Bad Audio Clip  03:16

You clearly don't know who you're talking to, so let me clue you in.

Paul Corsaro  03:21

 I'm Paul Corsaro.

Kris Hampton  03:22

I'm Kris Hampton.

Breaking Bad Audio Clip  03:23

Lucky two guys, but just guys, okay?

Paul Corsaro  03:27

And you're listening to Breaking Beta,

Kris Hampton  03:30

Where we explore and explain the science of climbing

Breaking Bad Audio Clip  03:33

With our skills, you'll earn more than you ever would on your own.

Breaking Bad Audio Clip  03:38

We've got work to do. Are you ready?

Paul Corsaro  03:42

I'm ready. Are you?

Kris Hampton  03:44

Abso-fucking-lutely. Doing these episodes, Breaking Beta or these Better Call Pauls, this is the third Better Call Paul, is just a really great way for me, same as reading research, it's it's a really great way for me to focus in on furthering my education about coaching and climbing training in general. So I fucking love it. 

Paul Corsaro  04:09

It's great. 

Kris Hampton  04:12

This could be because we're going to be talking about you know, how we choose papers, how we read papers, it could be a an episode where we end up all over the place, so we're going to try to structure it in a way that makes sense for folks, especially if you're somebody who doesn't spend a lot of time reading research, but you would like to.

Paul Corsaro  04:31

And it can be pretty daunting. There's a lot going on when you're like, "Oh, I'm gonna go look at the research on rock climbing". Like where do you start? What should you actually listen to and use? How do you find all this? How do you do you have to pay for these? Like it's there's a lot going on with this.

Kris Hampton  04:47

Right and it's drastically different than like sitting down and reading a training book or something. That's going to be more of a sit down and read it as opposed to reading research, where it's flipping through and lots of critical thinking and, you know requires you to kind of be all over the place

Paul Corsaro  05:05

Very much so

Kris Hampton  05:08

There are two basic types of papers, primary research and reviews. And to decide which you want to look at, you first have to sort of think about how much literature there is on the subject matter. For instance, when we were doing Season One, we talked about creatine and stretching. And those two topics are huge, so looking at an individual paper, which is it's an original study done by, done by the authors. That's, that's what primary research is. And then a review is taking a bunch of primary research papers and sort of collecting them all together, trying to pull the relevant information out. When there's a massive amount of research, looking at one paper only gives you a tiny little sliver of what's out there. So in those cases, like we did with stretching and creatine, makes sense to look at a review.

Paul Corsaro  06:06

And you know, those reviews are great ways to get a general idea on what the consensus on how to go about a certain concept or training method. And it can also point you in the direction of different pieces of primary research because those reviews assemble everything. 

Kris Hampton  06:21

Yeah

Paul Corsaro  06:21

And you can either A, look for, you know, when they cite the certain study, in a certain part of this review, look for the things that you agree with, and go, "Oh, yeah, that makes sense. I'm going to follow this deeper down the rabbit hole". It's also important, if you've find a couple things that surprise you that you don't agree with, you can go look up those papers too and either A, see something that maybe you don't agree with and how things were done or B, it makes you think a little bit harder about how you think about things, and maybe it'll challenge some biases and make you better in the long run.

Kris Hampton  06:48

Yeah, totally. If if there is a review on the topic that you're looking at, I think that's a great source for the primary research in general. They've done the work of taking all this research and saying these are relevant to this topic, so we're going to include these in this larger review. And you can go look at all those citations and and pull if you want to see the primary research, you can do it that way, you know, find a good list through a review.

Paul Corsaro  07:17

Or you could just read the review, try and come up with some training method that's different than everything else is out there, jump immediately to implementing it without seeing if it's actually applicable, or works with training someone and then call it the new direction of climbing and then there you go,

Kris Hampton  07:31

Hahaha. This, this approach was born for Instagram. You should, you should put it on there. However, when you're looking at like climbing, which is the thing that we're focused on, there's not a ton of research. So you, you sort of have to go down the path of looking at primary ,looking at individual papers. And, you know, we might be doing a sort of review on our own, looking at four or five papers that are on one topic, but there really isn't going to be enough to do a comprehensive review of any topic in the climbing world.

Paul Corsaro  08:11

I mean, I haven't found any and I've looked for some. If you're listening and you know of one, shoot that our way. But yeah, I haven't found any at least systematic reviews or meta analysis or anything like that. I haven't really come across any of that directly pertaining to climbing yet.

Kris Hampton  08:26

Do you go in and look at the full paper? Do you generally read every word in a paper? I'm curious.

Paul Corsaro  08:33

I try to go through every section, piece by piece. Starting, you know, you always got to start with the abstract, At least scan it to see if this is something you want to continue reading. Sometimes you'll see an abstract and be like "Ah, this isn't really helpful to me, like I could spend my time doing something different". But once you get past the abstract, I do try and work through section by section. My eyes glaze, may glaze over sometimes and they get into like the statistical processes. It's you know, that's why we brought Dale on last last episode, right? Because that's not my wheelhouse. We all look at, trying to read every word. You know, I'm highlighting things. I'm writing questions, so I can come back to at the end of the paper. But I try and work through section by section.

Kris Hampton  09:14

Do you do the same with reviews?

Paul Corsaro  09:16

I do skim a little more with the review. I think the tables are the really helpful part of the review, where they get they give you the basic idea of the study, really just rough idea of the methods, the results for each study. You can kind of get an idea of what they're getting at. I think the discussion is really helpful for the review.

Kris Hampton  09:35

Same

Paul Corsaro  09:35

Because they go a bit more into detail and kind of put it into a more cogent train of thought that you can follow a little easier than just jumping all over the place from study to study, if that makes sense.

Kris Hampton  09:44

Yeah, I agree. That's kind of what I do with reviews as well. I was curious about that within your process, because, you know, there aren't really methods that they're talking about in a review because they're looking at so many different papers with different methods. So I tend to just go straight to the discussion. Like, I'll read the the introduction and, you know, see what it is the authors are actually trying to find out, you know, what, what questions are they trying to answer here? And then I'll go straight to the discussion and see, you know, where those where those questions answered, what are they seeing in the research? For me, I can scan reviews a little more, particularly when I'm using it from my own general knowledge. You know, if we're doing it for Breaking Beta, I might read it a little more closely. But from my own general coaching knowledge, I'll I'll scan. , With primary research, that's kind of the only time I look at the abstract is to decide if I really want to read the paper in full. Is it asking an interesting question? And I'm talking general coaching knowledge, training knowledge here. Is it relatable to real life, real world climbing or does it get way way in the weeds of something that really isn't going to have an impact on how we train or climb? And then I'll, you know, if that's the case, I just, I just put the paper away. I'm like, I don't, I don't need to know this right now. Maybe down the road, you know, somebody will ask me a really interesting question based on this. But right now, if it's not going to impact my coaching or training, I just don't bother.

Kris Hampton  09:44

Yeah, and I think that's why the abstract is there. Personally, like if you're just looking at the abstracts, you're not getting the information you need, but it's step one to filter the, the, the papers you're going to dig deeper on.

Kris Hampton  11:40

And I think that's a, you know, a really important tactic, using the abstract to decide "Is this a thing I'm going to look deeper into?". Sometimes with Breaking Beta, we will purposely go deeper into papers that we know don't really translate very well to real world useful information. They might be great for scientific purposes, don't really translate to our coaching. But in in real life, I would probably not go down that rabbit hole at all. I wouldn't look at that paper.

Paul Corsaro  12:16

And you know, the good news is abstracts are all free. You can find the abstracts for every paper out there. You know, PubMed is a great resource. You can go to the journal's site, website and subscribe to an email feed that'll give you a notification if new studies get published. You can tag that to certain keywords. Like every week, I'll get emails with a couple things, "bouldering", "sport climbing", or just other training topics I'm interested in. I'll change those filters, so I get like a weekly digest, if you will. And you can just pull up the title and look at the abstract and you know, either save this paper or just move on

Kris Hampton  12:49

is that the main way that you find papers?

Paul Corsaro  12:52

Uh,  PubMed is a big one with those alerts. I'm also subscribed to the the NSCH Journal is a good one, Journal of Strength and Conditioning Research. And that also has a Strength and Conditioning Journal as well, which is the Strength Conditioning Journal is a little more I would say coachable applicable things and the Journal of Strength and Conditioning Research is more of the nitty gritty science based, more scientific topics, just looking at maybe some more in depth physiological things or genetic things, stuff along that nature. But yeah, it's usually email alerts, or sometimes I'll just set some time aside and just scan through and search for certain keywords, if I'm thinking about something training wise, or want to just get up to speed on certain topics.

Kris Hampton  13:41

Yeah, I think that's a that's a great way to see what research is out there. There's also Google Scholar, which is really easy to use for folks. 

Paul Corsaro  13:52

Absolutely. 

Kris Hampton  13:52

But I think if you're, if you're someone who's pressed for time, doesn't want to spend a lot of time searching for keywords, I think the using that filter system through PubMed is really the best, most effective use of the sites.

Paul Corsaro  14:09

There's a couple there's a couple sites. One is called Mass. It's more like a powerlifting type of focus, but they will actually combine a bunch of training studies and you can pay for a subscription. It's not terribly expensive, and they'll combine the studies. You'll get access to the studies. They'll break down their takings from each study and that's another option for people as well, these services that'll group things and do a little bit of the work for you to get you started

Kris Hampton  14:39

Someone made a comment on one of our Instagram posts from Season One that if a paper is not in PubMed, then it's not a reputable paper. And I've looked into that a little bit just to see and PubMed takes a very specific type of paper. They don't take every bit of sports research that there is. It just doesn't fit into their parameters, but that doesn't mean that it's not reputable. There are however, lots of journals out there that are not reputable, that are predatory. You know, they're all based on you paying them to be published, or and they're just not...they're not nearly as stringent as the reputable journals are. There's a site that you can go to if that's a thing you're concerned about, called Beall's List. It's at beallslist.net. I have the link in the show notes and on the blog post. You can just type in the journal name, and it will tell you if there are red flags on that journal.

Kris Hampton  14:53

And that's a great resource, because there's a lot of that that's behind the scenes that you won't necessarily know. 

Kris Hampton  15:54

Yeah. 

Paul Corsaro  15:55

Until you look that up. So that's, that's a good thing to have in there. I think. 

Kris Hampton  15:57

Yeah, that's a tough thing to know. How would you?

Paul Corsaro  16:00

 Yeah, it's intentionally hard to know. You know, they hide that shit. So

Kris Hampton  16:04

Especially with a sport like climbing, which is so small, so niche, you know, it's going to be a tiny little audience, so it's going to be harder to get published in a big journal. And just because it's a small journal doesn't mean it's a bad study. It can still be really rock solid work, it just might not cater to the larger scientific community.

Paul Corsaro  16:25

And that's why there's these frameworks for putting out these articles that you can really look at and see if people did do the work and follow a scientific process to find the answers. So it doesn't matter really the size of the journal as long as it's reputable.

Kris Hampton  16:38

Yeah, totally. Anything else you've thought of, Paul, about choosing papers?

Paul Corsaro  16:45

I think personally, for me, and you mentioned this as well, I'm looking for things I can apply to, to working with individuals, either in person or online. But sometimes if I'm working on a certain topic, or I can use these with people, then maybe I'll look at the more abstract, science-based research that's done about it. But I'm always about how can I use this? How is this applicable to my coaching practice?

Kris Hampton  17:12

Yeah, totally. One of the only times I go into papers that I would normally discount as "I don't think I can pull anything useful from this, in my real world coaching", is, if I'm looking at an article that cites one of those studies, then I might go look at it to try to understand a little more how it ties into this thing that I am finding useful. So I do think I will often find myself down that citation rabbit hole, where I'm looking at papers that normally I would have completely skipped over,

Paul Corsaro  17:51

For sure. Getting stuck in that the footnote or the reference  wormhole, deeper, deeper and deeper. Now I'm gonna get pulled to this paper, now I'm gonna get pulled to this paper. So it can happen, that's for sure.

Kris Hampton  18:01

It's really easy for it to happen. 

Paul Corsaro  18:03

Haha

Kris Hampton  18:04

All right, the nuts and bolts of looking at it. Now that you've kind of chosen a paper, you know it's from somewhere reputable, how do you actually go about looking at it? I have a really specific process that I use for Breaking Beta, which is different than how I normally read a paper. Normally, I do a little more skimming than I do for this show. If I find it's something that's, that could potentially be really useful in my training or coaching, then I might use the process I do for Breaking Beta to look at it more in depth. Usually, I'm more of a skimmer. How about you? Is your is your Breaking Beta process the same as your normal process?

Paul Corsaro  18:49

If I'm sitting down to read the paper, it's pretty similar. I try and be pretty granular and you know, go line by line best I can. When I get to the methods of something, I'll try and I'll pull out a separate piece of paper or just write on the study the characteristics of the population they're doing the study on. I'll try and write out their methods in my own words, just so I can A, talk about it and B, I think if you write something out in your own words, that helps you kind of grasp things a little better. 

Kris Hampton  19:14

Totally

Paul Corsaro  19:14

So I'll try and do that a lot as I work through things. I'll write things out, whether it's like I just talked about, the subjects or the methods or maybe thoughts or concepts that they're trying to make a point on. I'll try and write those out as I work through things. So I'm reading and writing at the same time, pretty much.

Kris Hampton  19:31

 Yeah, I do the same thing. Do you, do you use a hard copy? Like do you print it out and look at it or do you most often just look at it on a computer?

Paul Corsaro  19:41

I have an iPad, so I'll just pull the PDF onto an iPad and I've got some basic just PDF editing software that lets me highlight and right. And that way what's cool too is I think I've shared a couple with you or I've done that and it saves those and it can just send it to someone. They can already see what I've highlighted or pulled out or I can go back and see that quickly. And my office/work room is usually a goddamn disaster of papers and things, so the smallest amount of paper right now I can use the better. 

Kris Hampton  20:09

Yeah, I hear you. I've been thinking along the same lines of like, I have to stop using hard copies. Because I'm definitely a highlighter person and I've I've done it a little bit using a PDF editor, but I'm not nearly as good at it and I'm not as comfortable with it. So I'm currently still using hard copies, but the environment would be happy if I didn't do that.

Paul Corsaro  20:34

I mean, I'm looking up right now. I've got like, nine of those Moleskine notebooks that I used up before that. There's papers everywhere. Like, it's just a little more simple for me these days and I like it. 

Kris Hampton  20:44

Yeah, for me, number one, I get comfortable. I am going to be sitting in a place where I know I can sit for a while, you know, might have coffee, something to drink. I know, I'm gonna be there. I have a hard copy and I have highlighters. I really like using highlighters. And the funny thing is, I went back to our Season One papers, and I had color coded my highlighting, but I'd never made a key. So now I don't remember what my color coding meant haha

Paul Corsaro  21:13

I was about to ask what the key was, but I guess we're just lost there huh? Haha

Kris Hampton  21:16

Haha yeah. It was all in my head for Season One and now I can't remember at all

Paul Corsaro  21:20

I had, so I had two different highlighting colors. One was something I liked or what I thought was useful and then I had a red highlight, which is something I just wasn't quite sure about or wanted to go back and like think about so you can know

Kris Hampton  21:32

That's smart

Paul Corsaro  21:33

You know where to take things and with these ideas. 

Kris Hampton  21:35

Yeah, I like that. I spend a lot of time writing in the margins as well. Or I'll just put big question marks next to something that I want to come back to. But lots of scribbling for me. Questions, comments. 

Paul Corsaro  21:51

Well, and I think that's reflective of an active process of reading, though, right? You're not just staring at it, just trying to passively absorb. With this process, you need to be pretty active with it and engage with it in a 100% there way.

Kris Hampton  22:05

Yeah, I think that's a key, that's definitely a strong key to reading research, is you have to be really active, or you're going to zone out. You know, unless you're a very special breed of person that can read through this stuff, you will zone out. And for me, having the highlighters and the questions in the margins are a great way to really find the relevant information quickly. Because when I'm reading a paper, I tend to jump around a lot and flip back and forth and, you know, "What did they say over here?" and "Let me look at that table again" and you know, this just helps me do it a little more efficiently.

Paul Corsaro  22:44

 I agree. 

Kris Hampton  22:46

You said and correct me if I'm wrong here, but you kind of go linearly through the paper. You start on page one and just keep reading.

Paul Corsaro  22:48

Yep, all the way through. And then at the end, I'll go back and reread any of the questions I may have written down when I was doing it. And sometimes, you know, you write that question down and it gets answered later in the paper. So I'll either be able to address that question or be like, "Hey, this is still a question. I need to dig deeper or recheck some things as I go back through the paper a second or third time". But the first time through? Yeah, I'm just following along line by line, actively reading.

Kris Hampton  23:20

That's exactly what I used to do. And I, one of the things that sort of developed out of creating this podcast is something that I really like and that I'm going to continue to do when I'm just reading for my own sake and that's that, you know, I'm looking for, "What is the big question that's being asked?", "What's the aim of this research?". So I'll go first to the introduction and oftentimes, the final paragraph of the introduction starts with "The aim of this research is to" or "The aim of the study is to", 

Paul Corsaro  23:52

I would say, most of the time, it's there. Like it's pretty explicitly stated, which is great.

Kris Hampton  23:56

Yeah. So I highlight that, you know. I know I need that for these episodes. And then I know I need results for these episodes, so I'll go and go straight to, you know, from there to the conclusion discussion area, and look for what the results are like. Did you answer your question? Is that what actually happened here and what was that answer? And then I start to form my own questions, writing these in the margins. highlighting them. Because I think it's, it's easy for researchers to get into the weeds. They're excited about what they're doing. You know, they can get into the weeds and not actually be answering the questions they set out to answer, but shooting off on tangents and going down other rabbit holes. And those things can be really interesting and valuable, but, but I want to know "Did you answer the actual question that you set out to answer?". Sometimes they don't and then I tried to decide why they didn't, like looking at the methods and trying to decipher those.

Paul Corsaro  25:03

 I like that. That's an interesting way of going about it. I might have to try that for a paper or two this next season, just see if that changes the process or if I pull anything different out of it.

Kris Hampton  25:11

 Yeah, I also, like I put on this really critical lens. It's kind of just what I do with everything, you know. But I definitely come into research looking for the things that these researchers are saying that will cause it to not really apply in the real world. You know, they're stripping away variables, it's a, it's a necessary part of the science. And a lot of the things that they are saying or answering or that their study shows, I can't actually apply and sometimes I'm way too critical with that. So I'm trying to learn to tone that back. But sometimes it also shows you some really interesting things. You know, for instance, in Episode Two of Season One, we looked at the, you know, the fingertip pulp study, and that, that study completely strips away the real world variables of repeated attempts that are going to shred your skin, particularly if it isn't thick and tough and the fact that if you're using a 2.8 millimeter edge to begin with, it's, you know, it's likely that that climber will succeed because they can get their weight on their feet, not because they can pull on that edge harder than everybody else. So when they make statements like toughening your skin might be counterproductive for climbing on tiny edges, I have to be able to look at that and say, okay, they're missing a lot of the real world variables here, so I have to take all of this with a grain of salt.

Paul Corsaro  26:53

Yeah, or just go into it with the mindset that, okay, that might apply in this extremely specific situation where you're pulling on a sandstone block drilled to a wood plate pulling straight down. 

Kris Hampton  27:06

Yeah

Paul Corsaro  27:06

Taking away everything else that makes up climbing. But it's useful in that situation, but like you said, maybe not as reflective in the real world.

Kris Hampton  27:15

Yeah, so I think it's important to have that critical lens on and to go into it with that thought, you know

Paul Corsaro  27:22

Agreed.

Kris Hampton  27:22

The, the job of the scientists is to hold up to those critiques, right, so?

Paul Corsaro  27:28

Absolutely. 

Kris Hampton  27:29

So it should be welcomed in these papers for people to come at them with a critical lens.

Paul Corsaro  27:36

I agree

Kris Hampton  27:37

By having my critical lens on, I can look at the methods and say that the methods do or don't really apply either to the original question or to the real world. Lke maybe they're stripping out a variable that's a really necessary part of using this skill, or strength or whatever, in the real world. So I think that's probably the place in the paper where I spend the most of my time is, I'll sort of skim through the introduction, through the conclusion and the discussion and then I go into the methods, and that's where I have to be really comfortable. And I spend a lot of time scratching my head and with a calculator. And, you know, like you said, I think it's a great idea to try to write those methods out, in your own way. Because researchers are people, and they're going to fall victim to the fact that they're very familiar with the thing they're doing, so it's harder to explain it. And there are going to be parts missing sometimes that you don't quite understand or, 

Paul Corsaro  28:45

Or it could be wildly complicated 

Kris Hampton  28:47

Yeah

Paul Corsaro  28:47

And it's hard to grasp what they're actually doing, because there is just so much information in the methods.

Kris Hampton  28:52

Yeah, for me, the methods are the most complicated, time consuming part for sure.

Paul Corsaro  28:58

And that should be because A, they need to write that out so another group can either replicate these results or build on the results based on a similar study, or by creating a similar study. And just they need to, they need to show their work. They need to explain what exactly they did as clear as possible.

Kris Hampton  29:18

Yep. Have there been times when you're reading papers that when you first read the methods, you're like, "Why on earth cid they do it this way? This doesn't make any sense whatsoever." and then after you've spent a while, like marinating in, in their methods, you're going, "Oh, now I get it. It actually does make sense"?.

Paul Corsaro  29:42

I can't think of any specific examples, but I know that's definitely happened. Maybe not all the time when I'm confused or I think methods don't make sense, but definitely a few times looking back on it.

Kris Hampton  29:52

I think it's happened to me a fair number of times largely because I go in with that skeptical lens. And I'm immediately looking for the problems, you know. I'm like, "I'm gonna tear this shit apart." and I'm like, "Why did you do this? Shouldn't have done this. Why didn't you do it this way?" and then after I'm in there for a few minutes, I'm like, "Oh, okay, wait a minute, that kind of makes sense". 

Paul Corsaro  30:15

Hahaha

Kris Hampton  30:17

I came across a paper actually called "Ten Simple Rules for Reading a Scientific Paper." It's from Carey, Steiner and Petri. Very applicable name here for a scientist. One of the things that they had in there, they had ten rules, but one of them was "Six Important Questions To Ask." and I really liked these six questions, and they are something I've tried to do, but I haven't put them into, you know, this list nearly as well as they did. Number one is, what do the authors want to know? What's their motivation? You know, I think that's, that's where all all of these papers come from, is these people are passionate about this thing and they have a question they want to answer and they're motivated to find that and it takes a lot of fucking work and time and money to get to where they are. So they have to be really motivated.

Paul Corsaro  31:17

Some of the other ones are what did they do to approach...what do they do? So you know, the approach the methods, which I think we just talked about was, you know, how do they do it? How can someone do it exactly like they did? Were there any holes in how they did these things? And that's a big question asked, to kind of find those answers.

Kris Hampton  31:36

And then why was it done that way? And they make the point of, you know, looking at the context within the field, which I think is really important. If you're a, you know, if you're a baseball scientist, and you're coming in to study something about climbing, it's entirely possible, you don't quite understand climbing nearly as well. So your your study is going to be set up in a way that reflects your knowledge of climbing, which is very little. So if me, as a reader of this research, who understands climbing really well is going in and reading something about rugby, and then trying to take those results and translate it into climbing. I first need to understand rugby a little bit, you know, before I even have a chance of making that transition.

Paul Corsaro  32:30

And you're gonna usually find a good starting point for this in the introduction to the paper. So pass the abstract when they get into that first section, because usually, there's a good amount of work done there that highlights that context and makes it a little bit more appreciable as you go into the paper. So I think that'd be a good mix of both the introduction and the methods sections of a paper. Could give you a lot of good answers to the context within the field and why things were done that way. 

Kris Hampton  32:54

Yep, I agree. 

Paul Corsaro  32:56

The fourth question is, what do the results show? So you know, that's where you're gonna find the figures and the data and the actual measurable outcome of whatever experiment or test was run, you know. This is where they have the information that's backing up the claims they make in the end.

Kris Hampton  33:10

Yeah, I found a few articles, you know, asking, like students and researchers, how they read papers, that I'll also link in the show notes here for folks. And a lot of those people would jump straight from the abstract or introduction into the figures. For me, that does not work. I have to, I have to form a more complete picture of the question that's being asked and what the answers they think they found are before I can look at the figures. But if you're a person who is inclined to, you know, to graphs and charts, that might be a great way to go.

Paul Corsaro  33:49

For sure. Yeah, everyone's gonna go, you know, the route that they're most comfortable with. So some people like the more concrete, empirical things coming from the table, and numbers and statistics, and all those methods that go into these papers.

Kris Hampton  34:03

Yep. And I think number five works with number four really well, for me anyways. How did the authors interpret the results? And that's where I find the tables and the figures really important, because the authors might interpret the results a little differently than I would.

Paul Corsaro  34:21

Very much so. 

Kris Hampton  34:22

And that's not saying one is necessarily more valid than the other. It's just saying, I have a different experience, I have a different perspective on the sport than they do, on how I interact with clients and athletes than they do. So my interpretation of those results is going to reflect my experience and theirs is going to reflect their experience. So I think it's important to know both of those things.

Paul Corsaro  34:50

And at the very least, you know, the claims they make in that discussion should be backed up by the data. However, whatever context they give it or whatever direction they take it like it shouldn't just be some wild claim where you look back and you can't understand why they said this. So that's, that's just shows how you know, you're jumping back and forth all over these papers as you're going through it 1, 2, 3 times. And then, you know, the last the end of the paper, what should be done next? That's the big question to ask. And, you know, sometimes papers don't do a fantastic job of this, I think. But I think you know, that's up to us, too, as we're reading it, and using this in our practice. We can think about that, and where can we take this next or how can I use this concept and better it? And maybe I'm not gonna, I'm not gonna read a study about it. But you can test it out in practice with the people you work with, or with yourself along those lines. 

Kris Hampton  35:37

Yeah, that's actually a area that I wish was fleshed out a little bit in that how many practitioners are reading studies and saying, "I wish they had done this. I wish I knew the answer to this. This is the next logical question, I hope someone is asking it."? But then those thoughts just float off into the ether and, and the researchers never hear those questions. I wish there was a more direct line of communication that I know that I knew of, to the researchers, for asking those sorts of questions from the practitioners. Because for me, that's the really important part of all of this is, "How can we build on this study to better answer this question, or to answer the next logical question?"

Paul Corsaro  36:31

And I know, from the few people I know who have published research, that usually there's at least one email address people can reach out to and for the most part, you know, I'm not trying to put words in anybody's mouth, but most of the times the people do this research, are happy to communicate at least a little bit, or would like to receive feedback on where to take things next. I'm sure some people don't like that. But that could be a next step if you have a question, or you want to encourage a direction of a line of thought to go, that could be someone to reach out to.

Kris Hampton  37:06

Yeah, totally. And we're putting our questions out into the, into the world. 

Paul Corsaro  37:11

Oh, yeah

Kris Hampton  37:12

So hopefully, some researchers are actually listening to this, to get some of that, not just to hear us bashing them on occasion.

Paul Corsaro  37:18

Haha

Kris Hampton  37:20

 I'm going to run through those one more time, just to have them all in one spot for folks. Six important questions to ask when you're reading research. Number one, what did the authors want to know? What's their motivation? Number two, what did they do? The approach and the methods. Why was it done that way? Context within the field. And number four, what did the results show? You can see that in the figures and the data. And number five, how did the authors interpret the results? And number six, what should be done next? And I was happy to come across this because I feel like this is something we did pretty well in Season One, going through these papers is making sure we're asking all of these questions.

Paul Corsaro  38:07

It's always nice to find something backing up the way you do things in retrospect, right?

Kris Hampton  38:11

Yeah, exactly.

Paul Corsaro  38:12

Haha. Yeah. 

Kris Hampton  38:12

Do we want to go into Sci Hub at all?

Paul Corsaro  38:15

Um, it is a shame how easy they make it to find papers. It's definitely illegal and you should never do it. Never. 

Paul Corsaro  38:23

But if you you should never absolutely ever use Sci Hub, even on accident, because it's really easy to get papers that way

Kris Hampton  38:23

Hahahaha

Kris Hampton  38:31

Hahaha

Paul Corsaro  38:31

But don't do it.

Kris Hampton  38:31

Yeah, I was I was sort of thinking maybe we'll just not say it. 

Paul Corsaro  38:37

I will say going back to the communication with a lot of these researchers, a lot of times if you reach out to them, they may be able to find you a copy of that full text.

Kris Hampton  38:45

Yeah, that's a good, good beta. A lot of these, when you're looking at them, they say, you know, email the author for for the full text. And I think it's probably...and this is totally me guessing, but it's probably fairly rare that people actually do that. It seems like making a phone call to someone, you know, like, you go to someone's website, and they're like, just call this number and nobody ever fucking calls. 

Paul Corsaro  39:11

Yeah. Hahaha

Kris Hampton  39:12

Even though you could reach somebody that way. That's what I feel like it is, but that's entirely me guessing.

Paul Corsaro  39:19

Yeah, that could be a good way to get a full text if you don't want to deal with some of the hoops that publishing companies put out there. 

Kris Hampton  39:26

Yeah, exactly. It's it would be, you know, this thing I'm lamenting that, as far as I know, does not exist, this open line of communication between the researchers and the practitioners, it's almost impossible for that to exist when all of these papers are behind various paywalls. And, you know, if there were an easy place to collect all of these things, and for practitioners to look at it, and then communicate that would be fantastic. 

Paul Corsaro  39:53

I'd love it. That'd be awesome. 

Kris Hampton  39:55

Yeah. 

Paul Corsaro  39:55

But alas, it doesn't exist yet. I think.

Kris Hampton  39:59

Yeah. And you know hopefully, we're going to have, you know, within this season and maybe a few episodes after the season, we'll talk to some of these researchers and you know, get some of their takes on these things as well how, how publishing works, how communication with practitioners works and what what they see that works well and what doesn't. I think that's something really valuable that we could dive into.

Paul Corsaro  40:28

I'm excited for when we go down that route.

Kris Hampton  40:30

All right, we've put links to several articles showing how other researchers, PhD students, et cetera, read research for themselves. If you're looking at these, keep in mind the context. Students are going to read it differently than a researcher who's hoping to continue that line of questioning and and that's going to be different than how coaches are reading the research, and how podcasters are reading the research. You know, we're all looking at it through our own lens from our own perspective, so keep that in context. You don't have to necessarily mimic the way someone else reads research to get the benefits of it. You can find both Paul and I all over the internets by following the links right there in your show notes. You can find Paul at his gym, Crux Conditioning, in Chattanooga. We are both instructors at the Power Company Climbing Academy, along with several other coaches, and you can get signed up for updates on new courses for climbers and coaches by following the link in your show notes. We've got a big course coming this summer that you're going to want to be the first to know about. Don't forget to subscribe to the show. Leave us a review. And please tell all of your friends who think they fully understand the research because they read the abstract or worse, they read an Instagram summary, even if it's from us, that you have the perfect podcast for them. And we'll see you next week when we discuss the research standards decided upon by the International Rock Climbing Research Association. That is a mouthful. And whether or not they make sense when looking at real world rock climbing. Thanks, y'all. We'll see you next time.

Breaking Bad Audio Clip  42:08

It's done.

Breaking Bad Audio Clip  42:09

You keep saying that and it's bullshit every time, always. You know what? I'm done

Breaking Bad Audio Clip  42:16

 Okay, you and I were done.

Kris Hampton  42:21

Breaking Beta is brought to you by Power Company Climbing and Crux Conditioning and as a proud member of the Plug Tone Audio Collective. For transcripts, citations and more visit powercompanyclimbing.com/breakingbeta.

Breaking Bad Audio Clip  42:34

Let's not get lost in the who what and whens. The point is we did our due diligence.

Kris Hampton  42:40

Our music, including our theme song Tumbleweed, is from legendary South Dakota band Riff Lord

Breaking Bad Audio Clip  42:46

This is it. This is how it ends.