how to analyze evidence in a research paper

What this handout is about

This handout will provide a broad overview of gathering and using evidence. It will help you decide what counts as evidence, put evidence to work in your writing, and determine whether you have enough evidence. It will also offer links to additional resources.

Introduction

Many papers that you write in college will require you to make an argument ; this means that you must take a position on the subject you are discussing and support that position with evidence. It’s important that you use the right kind of evidence, that you use it effectively, and that you have an appropriate amount of it. If, for example, your philosophy professor didn’t like it that you used a survey of public opinion as your primary evidence in your ethics paper, you need to find out more about what philosophers count as good evidence. If your instructor has told you that you need more analysis, suggested that you’re “just listing” points or giving a “laundry list,” or asked you how certain points are related to your argument, it may mean that you can do more to fully incorporate your evidence into your argument. Comments like “for example?,” “proof?,” “go deeper,” or “expand” in the margins of your graded paper suggest that you may need more evidence. Let’s take a look at each of these issues—understanding what counts as evidence, using evidence in your argument, and deciding whether you need more evidence.

What counts as evidence?

Before you begin gathering information for possible use as evidence in your argument, you need to be sure that you understand the purpose of your assignment. If you are working on a project for a class, look carefully at the assignment prompt. It may give you clues about what sorts of evidence you will need. Does the instructor mention any particular books you should use in writing your paper or the names of any authors who have written about your topic? How long should your paper be (longer works may require more, or more varied, evidence)? What themes or topics come up in the text of the prompt? Our handout on understanding writing assignments can help you interpret your assignment. It’s also a good idea to think over what has been said about the assignment in class and to talk with your instructor if you need clarification or guidance.

What matters to instructors?

Instructors in different academic fields expect different kinds of arguments and evidence—your chemistry paper might include graphs, charts, statistics, and other quantitative data as evidence, whereas your English paper might include passages from a novel, examples of recurring symbols, or discussions of characterization in the novel. Consider what kinds of sources and evidence you have seen in course readings and lectures. You may wish to see whether the Writing Center has a handout regarding the specific academic field you’re working in—for example, literature , sociology , or history .

What are primary and secondary sources?

A note on terminology: many researchers distinguish between primary and secondary sources of evidence (in this case, “primary” means “first” or “original,” not “most important”). Primary sources include original documents, photographs, interviews, and so forth. Secondary sources present information that has already been processed or interpreted by someone else. For example, if you are writing a paper about the movie “The Matrix,” the movie itself, an interview with the director, and production photos could serve as primary sources of evidence. A movie review from a magazine or a collection of essays about the film would be secondary sources. Depending on the context, the same item could be either a primary or a secondary source: if I am writing about people’s relationships with animals, a collection of stories about animals might be a secondary source; if I am writing about how editors gather diverse stories into collections, the same book might now function as a primary source.

Where can I find evidence?

Here are some examples of sources of information and tips about how to use them in gathering evidence. Ask your instructor if you aren’t sure whether a certain source would be appropriate for your paper.

Print and electronic sources

Books, journals, websites, newspapers, magazines, and documentary films are some of the most common sources of evidence for academic writing. Our handout on evaluating print sources will help you choose your print sources wisely, and the library has a tutorial on evaluating both print sources and websites. A librarian can help you find sources that are appropriate for the type of assignment you are completing. Just visit the reference desk at Davis or the Undergraduate Library or chat with a librarian online (the library’s IM screen name is undergradref).

Observation

Sometimes you can directly observe the thing you are interested in, by watching, listening to, touching, tasting, or smelling it. For example, if you were asked to write about Mozart’s music, you could listen to it; if your topic was how businesses attract traffic, you might go and look at window displays at the mall.

An interview is a good way to collect information that you can’t find through any other type of research. An interview can provide an expert’s opinion, biographical or first-hand experiences, and suggestions for further research.

Surveys allow you to find out some of what a group of people thinks about a topic. Designing an effective survey and interpreting the data you get can be challenging, so it’s a good idea to check with your instructor before creating or administering a survey.

Experiments

Experimental data serve as the primary form of scientific evidence. For scientific experiments, you should follow the specific guidelines of the discipline you are studying. For writing in other fields, more informal experiments might be acceptable as evidence. For example, if you want to prove that food choices in a cafeteria are affected by gender norms, you might ask classmates to undermine those norms on purpose and observe how others react. What would happen if a football player were eating dinner with his teammates and he brought a small salad and diet drink to the table, all the while murmuring about his waistline and wondering how many fat grams the salad dressing contained?

Personal experience

Using your own experiences can be a powerful way to appeal to your readers. You should, however, use personal experience only when it is appropriate to your topic, your writing goals, and your audience. Personal experience should not be your only form of evidence in most papers, and some disciplines frown on using personal experience at all. For example, a story about the microscope you received as a Christmas gift when you were nine years old is probably not applicable to your biology lab report.

Using evidence in an argument

Does evidence speak for itself.

Absolutely not. After you introduce evidence into your writing, you must say why and how this evidence supports your argument. In other words, you have to explain the significance of the evidence and its function in your paper. What turns a fact or piece of information into evidence is the connection it has with a larger claim or argument: evidence is always evidence for or against something, and you have to make that link clear.

As writers, we sometimes assume that our readers already know what we are talking about; we may be wary of elaborating too much because we think the point is obvious. But readers can’t read our minds: although they may be familiar with many of the ideas we are discussing, they don’t know what we are trying to do with those ideas unless we indicate it through explanations, organization, transitions, and so forth. Try to spell out the connections that you were making in your mind when you chose your evidence, decided where to place it in your paper, and drew conclusions based on it. Remember, you can always cut prose from your paper later if you decide that you are stating the obvious.

Here are some questions you can ask yourself about a particular bit of evidence:

• OK, I’ve just stated this point, but so what? Why is it interesting? Why should anyone care?
• What does this information imply?
• What are the consequences of thinking this way or looking at a problem this way?
• I’ve just described what something is like or how I see it, but why is it like that?
• I’ve just said that something happens—so how does it happen? How does it come to be the way it is?
• Why is this information important? Why does it matter?
• How is this idea related to my thesis? What connections exist between them? Does it support my thesis? If so, how does it do that?
• Can I give an example to illustrate this point?

Answering these questions may help you explain how your evidence is related to your overall argument.

How can I incorporate evidence into my paper?

There are many ways to present your evidence. Often, your evidence will be included as text in the body of your paper, as a quotation, paraphrase, or summary. Sometimes you might include graphs, charts, or tables; excerpts from an interview; or photographs or illustrations with accompanying captions.

When you quote, you are reproducing another writer’s words exactly as they appear on the page. Here are some tips to help you decide when to use quotations:

• Quote if you can’t say it any better and the author’s words are particularly brilliant, witty, edgy, distinctive, a good illustration of a point you’re making, or otherwise interesting.
• Quote if you are using a particularly authoritative source and you need the author’s expertise to back up your point.
• Quote if you are analyzing diction, tone, or a writer’s use of a specific word or phrase.
• Quote if you are taking a position that relies on the reader’s understanding exactly what another writer says about the topic.

Be sure to introduce each quotation you use, and always cite your sources. See our handout on quotations for more details on when to quote and how to format quotations.

Like all pieces of evidence, a quotation can’t speak for itself. If you end a paragraph with a quotation, that may be a sign that you have neglected to discuss the importance of the quotation in terms of your argument. It’s important to avoid “plop quotations,” that is, quotations that are just dropped into your paper without any introduction, discussion, or follow-up.

Paraphrasing

When you paraphrase, you take a specific section of a text and put it into your own words. Putting it into your own words doesn’t mean just changing or rearranging a few of the author’s words: to paraphrase well and avoid plagiarism, try setting your source aside and restating the sentence or paragraph you have just read, as though you were describing it to another person. Paraphrasing is different than summary because a paraphrase focuses on a particular, fairly short bit of text (like a phrase, sentence, or paragraph). You’ll need to indicate when you are paraphrasing someone else’s text by citing your source correctly, just as you would with a quotation.

When might you want to paraphrase?

• Paraphrase when you want to introduce a writer’s position, but their original words aren’t special enough to quote.
• Paraphrase when you are supporting a particular point and need to draw on a certain place in a text that supports your point—for example, when one paragraph in a source is especially relevant.
• Paraphrase when you want to present a writer’s view on a topic that differs from your position or that of another writer; you can then refute writer’s specific points in your own words after you paraphrase.
• Paraphrase when you want to comment on a particular example that another writer uses.
• Paraphrase when you need to present information that’s unlikely to be questioned.

When you summarize, you are offering an overview of an entire text, or at least a lengthy section of a text. Summary is useful when you are providing background information, grounding your own argument, or mentioning a source as a counter-argument. A summary is less nuanced than paraphrased material. It can be the most effective way to incorporate a large number of sources when you don’t have a lot of space. When you are summarizing someone else’s argument or ideas, be sure this is clear to the reader and cite your source appropriately.

Statistics, data, charts, graphs, photographs, illustrations

Sometimes the best evidence for your argument is a hard fact or visual representation of a fact. This type of evidence can be a solid backbone for your argument, but you still need to create context for your reader and draw the connections you want them to make. Remember that statistics, data, charts, graph, photographs, and illustrations are all open to interpretation. Guide the reader through the interpretation process. Again, always, cite the origin of your evidence if you didn’t produce the material you are using yourself.

Do I need more evidence?

Let’s say that you’ve identified some appropriate sources, found some evidence, explained to the reader how it fits into your overall argument, incorporated it into your draft effectively, and cited your sources. How do you tell whether you’ve got enough evidence and whether it’s working well in the service of a strong argument or analysis? Here are some techniques you can use to review your draft and assess your use of evidence.

Make a reverse outline

A reverse outline is a great technique for helping you see how each paragraph contributes to proving your thesis. When you make a reverse outline, you record the main ideas in each paragraph in a shorter (outline-like) form so that you can see at a glance what is in your paper. The reverse outline is helpful in at least three ways. First, it lets you see where you have dealt with too many topics in one paragraph (in general, you should have one main idea per paragraph). Second, the reverse outline can help you see where you need more evidence to prove your point or more analysis of that evidence. Third, the reverse outline can help you write your topic sentences: once you have decided what you want each paragraph to be about, you can write topic sentences that explain the topics of the paragraphs and state the relationship of each topic to the overall thesis of the paper.

For tips on making a reverse outline, see our handout on organization .

Color code your paper

You will need three highlighters or colored pencils for this exercise. Use one color to highlight general assertions. These will typically be the topic sentences in your paper. Next, use another color to highlight the specific evidence you provide for each assertion (including quotations, paraphrased or summarized material, statistics, examples, and your own ideas). Lastly, use another color to highlight analysis of your evidence. Which assertions are key to your overall argument? Which ones are especially contestable? How much evidence do you have for each assertion? How much analysis? In general, you should have at least as much analysis as you do evidence, or your paper runs the risk of being more summary than argument. The more controversial an assertion is, the more evidence you may need to provide in order to persuade your reader.

Play devil’s advocate, act like a child, or doubt everything

This technique may be easiest to use with a partner. Ask your friend to take on one of the roles above, then read your paper aloud to them. After each section, pause and let your friend interrogate you. If your friend is playing devil’s advocate, they will always take the opposing viewpoint and force you to keep defending yourself. If your friend is acting like a child, they will question every sentence, even seemingly self-explanatory ones. If your friend is a doubter, they won’t believe anything you say. Justifying your position verbally or explaining yourself will force you to strengthen the evidence in your paper. If you already have enough evidence but haven’t connected it clearly enough to your main argument, explaining to your friend how the evidence is relevant or what it proves may help you to do so.

Common questions and additional resources

• I have a general topic in mind; how can I develop it so I’ll know what evidence I need? And how can I get ideas for more evidence? See our handout on brainstorming .
• Who can help me find evidence on my topic? Check out UNC Libraries .
• I’m writing for a specific purpose; how can I tell what kind of evidence my audience wants? See our handouts on audience , writing for specific disciplines , and particular writing assignments .
• How should I read materials to gather evidence? See our handout on reading to write .
• How can I make a good argument? Check out our handouts on argument and thesis statements .
• How do I tell if my paragraphs and my paper are well-organized? Review our handouts on paragraph development , transitions , and reorganizing drafts .
• How do I quote my sources and incorporate those quotes into my text? Our handouts on quotations and avoiding plagiarism offer useful tips.
• How do I cite my evidence? See the UNC Libraries citation tutorial .
• I think that I’m giving evidence, but my instructor says I’m using too much summary. How can I tell? Check out our handout on using summary wisely.
• I want to use personal experience as evidence, but can I say “I”? We have a handout on when to use “I.”

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Lunsford, Andrea A., and John J. Ruszkiewicz. 2016. Everything’s an Argument , 7th ed. Boston: Bedford/St Martin’s.

Miller, Richard E., and Kurt Spellmeyer. 2016. The New Humanities Reader , 5th ed. Boston: Cengage.

University of Maryland. 2019. “Research Using Primary Sources.” Research Guides. Last updated October 28, 2019. https://lib.guides.umd.edu/researchusingprimarysources .

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Write Clearly: Using Evidence Effectively

How do i select evidence, how do i analyze evidence.

• Incorporating Evidence

Ask Us: Chat, email, visit or call

Get assistance

The library offers a range of helpful services.  All of our appointments are free of charge and confidential.

• Book an appointment

Much of how to use evidence is about finding a clear and logical relation between the evidence you use and your claim. For example, if you are asked to write a paper on the effects of pollution on watersheds, you would not use a story your grandfather told you about the river he used to swim in that is now polluted. You would look for peer-reviewed journal articles by experts on the subject.

Once you have found the appropriate type of evidence, it is important to select the evidence that supports your specific claim. For example, if you are writing a psychology paper on the role of emotions in decision-making, you would look for psychology journal articles that connect these two elements.

For example:

By referencing the study in the first example and supplying textual evidence in the second, the initial statement in the paragraph moves from opinion to supported argument; however, you must still analyze your evidence.

Once you have selected your evidence it is important to tell you reader why the evidence supports your claim. Evidence does not speak for itself: some readers may draw different conclusions from your evidence, or may not understand the relation between your evidence and your claim. It is up to you to walk your reader through the significance of the evidence to your claim and your larger argument. In short, you need a reason why the evidence supports the claim – you need to analyze the evidence.

Some questions you could consider are:

• Why is this evidence interesting or effective?
• What are the consequences or implications of this evidence?
• Why is this information important?
• How has it been important to my paper or to the field I am studying?
• How is this idea related to my thesis?
• This evidence points to a result of an experiment or study, can I explain why these results are important or what caused them?
• Can I give an example to illustrate this point?
• What are the strengths and weaknesses of the evidence presented?

If we look to our first examples, they may look like this once we add analysis to our evidence:

• Emotions play a larger role in rational decision-making than most us think ( claim ). Subjects deciding to wear a seatbelt demonstrated an activity in the ventromedial frontal lobe, the part of the brain that governs emotion (Shibata 2001) ( evidence ). This suggests that people making rational decisions, even when performing naturalized tasks such as putting on a seatbelt, rely on their emotions ( analysis ).

Or, when we look at the example of Dr. Jekyll and Mr. Hyde:

• The physical descriptions of the laboratory and the main house, in Dr Jekyll and Mr. Hyde, metaphorically point to the gothic elements in the novel ( claim ). The main house had "a great air of wealth and comfort" (13), while the laboratory door "which was equipped with neither bell nor knocker, was blistered and distained" (3) ( evidence ). The comforting and welcoming look of the main house is in sharp contrast to the door of the laboratory, which does not even have a bell to invite people in. The laboratory door is eerie and gothic highlighting the abnormal and mystical events that take place behind it ( analysis ).
• << Previous: Start Here
• Next: Incorporating Evidence >>
• Last Updated: Jul 12, 2022 1:00 PM
• URL: https://guides.lib.uoguelph.ca/UseEvidenceEffectively

Suggest an edit to this guide

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

• Joyner Library
• Laupus Health Sciences Library
• Music Library
• Digital Collections
• Special Collections
• North Carolina Collection
• Teaching Resources
• The ScholarShip Institutional Repository
• Country Doctor Museum

Evidence-Based Practice for Nursing: Evaluating the Evidence

• What is Evidence-Based Practice?
• Asking the Clinical Question
• Finding Evidence
• Evaluating the Evidence
• Articles, Books & Web Resources on EBN

Evaluating Evidence: Questions to Ask When Reading a Research Article or Report

For guidance on the process of reading a research book or an article, look at Paul N. Edward's paper,  How to Read a Book  (2014) . When reading an article, report, or other summary of a research study, there are two principle questions to keep in mind:

1. Is this relevant to my patient or the problem?

• Once you begin reading an article, you may find that the study population isn't representative of the patient or problem you are treating or addressing. Research abstracts alone do not always make this apparent.
• You may also find that while a study population or problem matches that of your patient, the study did not focus on an aspect of the problem you are interested in. E.g. You may find that a study looks at oral administration of an antibiotic before a surgical procedure, but doesn't address the timing of the administration of the antibiotic.
• The question of relevance is primary when assessing an article--if the article or report is not relevant, then the validity of the article won't matter (Slawson & Shaughnessy, 1997).

2. Is the evidence in this study valid?

• Validity is the extent to which the methods and conclusions of a study accurately reflect or represent the truth. Validity in a research article or report has two parts: 1) Internal validity--i.e. do the results of the study mean what they are presented as meaning? e.g. were bias and/or confounding factors present? ; and 2) External validity--i.e. are the study results generalizable? e.g. can the results be applied outside of the study setting and population(s) ?
• Determining validity can be a complex and nuanced task, but there are a few criteria and questions that can be used to assist in determining research validity. The set of questions, as well as an overview of levels of evidence, are below.

For a checklist that can help you evaluate a research article or report, use our checklist for Critically Evaluating a Research Article

• How to Critically Evaluate a Research Article

How to Read a Paper--Assessing the Value of Medical Research

Evaluating the evidence from medical studies can be a complex process, involving an understanding of study methodologies, reliability and validity, as well as how these apply to specific study types. While this can seem daunting, in a series of articles by Trisha Greenhalgh from BMJ, the author introduces the methods of evaluating the evidence from medical studies, in language that is understandable even for non-experts. Although these articles date from 1997, the methods the author describes remain relevant. Use the links below to access the articles.

• How to read a paper: Getting your bearings (deciding what the paper is about) Not all published research is worth considering. This provides an outline of how to decide whether or not you should consider a research paper. more... less... Greenhalgh, T. (1997b). How to read a paper. Getting your bearings (deciding what the paper is about). BMJ (Clinical Research Ed.), 315(7102), 243–246.
• Assessing the methodological quality of published papers This article discusses how to assess the methodological validity of recent research, using five questions that should be addressed before applying recent research findings to your practice. more... less... Greenhalgh, T. (1997a). Assessing the methodological quality of published papers. BMJ (Clinical Research Ed.), 315(7103), 305–308.
• How to read a paper. Statistics for the non-statistician. I: Different types of data need different statistical tests This article and the next present the basics for assessing the statistical validity of medical research. The two articles are intended for readers who struggle with statistics more... less... Greenhalgh, T. (1997f). How to read a paper. Statistics for the non-statistician. I: Different types of data need different statistical tests. BMJ (Clinical Research Ed.), 315(7104), 364–366.
• How to read a paper: Statistics for the non-statistician II: "Significant" relations and their pitfalls The second article on evaluating the statistical validity of a research article. more... less... Greenhalgh, T. (1997). Education and debate. how to read a paper: Statistics for the non-statistician. II: "significant" relations and their pitfalls. BMJ: British Medical Journal (International Edition), 315(7105), 422-425. doi: 10.1136/bmj.315.7105.422
• How to read a paper. Papers that report drug trials more... less... Greenhalgh, T. (1997d). How to read a paper. Papers that report drug trials. BMJ (Clinical Research Ed.), 315(7106), 480–483.
• How to read a paper. Papers that report diagnostic or screening tests more... less... Greenhalgh, T. (1997c). How to read a paper. Papers that report diagnostic or screening tests. BMJ (Clinical Research Ed.), 315(7107), 540–543.
• How to read a paper. Papers that tell you what things cost (economic analyses) more... less... Greenhalgh, T. (1997e). How to read a paper. Papers that tell you what things cost (economic analyses). BMJ (Clinical Research Ed.), 315(7108), 596–599.
• Papers that summarise other papers (systematic reviews and meta-analyses) more... less... Greenhalgh, T. (1997i). Papers that summarise other papers (systematic reviews and meta-analyses). BMJ (Clinical Research Ed.), 315(7109), 672–675.
• How to read a paper: Papers that go beyond numbers (qualitative research) A set of questions that could be used to analyze the validity of qualitative research more... less... Greenhalgh, T., & Taylor, R. (1997). Papers that go beyond numbers (qualitative research). BMJ (Clinical Research Ed.), 315(7110), 740–743.

Levels of Evidence

In some journals, you will see a 'level of evidence' assigned to a research article. Levels of evidence are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care. The combination of these attributes gives the level of evidence for a study.  Many systems for assigning levels of evidence exist.  A frequently used system in medicine is from the  Oxford Center for Evidence-Based Medicine .  In nursing, the system for assigning levels of evidence is often from Melnyk & Fineout-Overholt's 2011 book,  Evidence-based Practice in Nursing and Healthcare: A Guide to Best Practice .  The Levels of Evidence below are adapted from Melnyk & Fineout-Overholt's (2011) model.  

Uses of Levels of Evidence : Levels of evidence from one or more studies provide the "grade (or strength) of recommendation" for a particular treatment, test, or practice. Levels of evidence are reported for studies published in some medical and nursing journals. Levels of Evidence are most visible in Practice Guidelines, where the level of evidence is used to indicate how strong a recommendation for a particular practice is. This allows health care professionals to quickly ascertain the weight or importance of the recommendation in any given guideline. In some cases, levels of evidence in guidelines are accompanied by a Strength of Recommendation.

About Levels of Evidence and the Hierarchy of Evidence : While Levels of Evidence correlate roughly with the hierarchy of evidence (discussed elsewhere on this page), levels of evidence don't always match the categories from the Hierarchy of Evidence, reflecting the fact that study design alone doesn't guarantee good evidence. For example, the systematic review or meta-analysis of randomized controlled trials (RCTs) are at the top of the evidence pyramid and are typically assigned the highest level of evidence, due to the fact that the study design reduces the probability of bias  ( Melnyk , 2011),  whereas the weakest level of evidence is the  opinion from authorities and/or reports of expert committees.  However, a systematic review may report very weak evidence for a particular practice and therefore the level of evidence behind a recommendation may be lower than the position of the study type on the Pyramid/Hierarchy of Evidence.

About Levels of Evidence and Strength of Recommendation : The fact that a study is located lower on the Hierarchy of Evidence does not necessarily mean that the strength of recommendation made from that and other studies is low--if evidence is consistent across studies on a topic and/or very compelling, strong recommendations can be made from evidence found in studies with lower levels of evidence, and study types located at the bottom of the Hierarchy of Evidence. In other words, strong recommendations can be made from lower levels of evidence.

For example: a case series observed in 1961 in which two physicians who noted a high incidence (approximately 20%) of children born with birth defects to mothers taking thalidomide resulted in very strong recommendations against the prescription and eventually, manufacture and marketing of thalidomide. In other words, as a result of the case series, a strong recommendation was made from a study that was in one of the lowest positions on the hierarchy of evidence.

Hierarchy of Evidence for Quantitative Questions

The pyramid below represents the hierarchy of evidence, which illustrates the strength of study types; the higher the study type on the pyramid, the more likely it is that the research is valid. The pyramid is meant to assist researchers in prioritizing studies they have located to answer a clinical or practice question. 

For clinical questions, you should try to find articles with the highest quality of evidence. Systematic Reviews and Meta-Analyses are considered the highest quality of evidence for clinical decision-making and should be used above other study types, whenever available, provided the Systematic Review or Meta-Analysis is fairly recent. 

As you move up the pyramid, fewer studies are available, because the study designs become increasingly more expensive for researchers to perform. It is important to recognize that high levels of evidence may not exist for your clinical question, due to both costs of the research and the type of question you have.  If the highest levels of study design from the evidence pyramid are unavailable for your question, you'll need to move down the pyramid.

While the pyramid of evidence can be helpful, individual studies--no matter the study type--must be assessed to determine the validity.

Hierarchy of Evidence for Qualitative Studies

Qualitative studies are not included in the Hierarchy of Evidence above. Since qualitative studies provide valuable evidence about patients' experiences and values, qualitative studies are important--even critically necessary--for Evidence-Based Nursing. Just like quantitative studies, qualitative studies are not all created equal. The pyramid below  shows a hierarchy of evidence for qualitative studies.

Adapted from Daly et al. (2007)

Help with Research Terms & Study Types: Cut through the Jargon!

• CEBM Glossary
• Centre for Evidence-Based Medicine|Toronto
• Cochrane Collaboration Glossary
• Qualitative Research Terms (NHS Trust)
• << Previous: Finding Evidence
• Next: Articles, Books & Web Resources on EBN >>
• Last Updated: Jan 12, 2024 10:03 AM
• URL: https://libguides.ecu.edu/ebn

• Utility Menu

GA4 Tracking Code

Gen ed writes, writing across the disciplines at harvard college.

• Evidence and Analysis

Why It Matters

An assignment prompt’s guidance on evidence and analysis sets parameters for the content and form of a writing assignment: What kinds of sources should you be working with? Where should you find those sources? How should you be working with them?

More on "Evidence and Analysis"

The evidence and analysis you're asked to use (or not use) for a writing assignment often reflect the genre and size of the assignment at hand. With any writing assignment prompt, it’s important to step back and make sure you’re clear about the scope of evidence and analysis you’ll be working with.  For example: 

In terms of evidence,

• what kinds of evidence should be used (peer-reviewed articles versus op-ed pieces),
• which evidence in particular and how much (3–5 readings from class versus independent research), and
• why (because op-ed pieces capture a kind of public discourse better than peer-reviewed articles, or because 3–5 readings from class is manageable for a 4-page essay and also reinforces the readings assigned for the course, etc.).

In terms of analysis,

• is the assignment asking you to make an argument? If so, what kind of argument? (e.g., a rhetorical analysis weighing the pros and cons of a think piece, or a policy memo making normative claims about recommended courses of action, or a test a theory essay assessing the applicability of a framework to real-world cases?)
• if not, what is it asking you to do with evidence? (e.g., summarize a source’s argument, or draft a research question based on an annotated bibliography or data set)
• why? (because it’s important to establish other thinkers’ positions accurately before taking your own position, or because asking questions before moving on to a thesis or conclusion will make the research process more compelling).

What It Looks Like

• Science & Technology in Society
• Ethics & Civics
• Histories, Societies, Individuals
• Aesthetics & Culture

STEP 1: PROPOSAL WITH ANNOTATED BIBLIOGRAPHY

Length: 250–500 words, not including annotated bibliography. The annotated bibliography must have at least 5 different references from outside the course and 5 different references from the syllabus.

Source requirements:

• Minimum 5 different references from outside the course (at least 3 must be peer-reviewed scholarly sources)   [1]
• Minimum 5 different references from Gen Ed 1093 reading assignments listed on the syllabus; lectures do not count toward the reference requirement, and Reimagining Global Health will only count as one reference   [2]
• Citation format either AAA or APA   [3] , consistent throughout the paper
• Careful attention to academic integrity and appropriate citation practices
• The annotated bibliography does not count toward your word count, but in-text citations do.   [4]

__________ [1]  Explicit guidance about what kinds of sources and how many sources to include [2]  Clarification about what does / doesn't count toward the required number of sources [3]  Clear guidance about citation format  [4]  Clarification about what does / doesn't count toward the required word count  

Adapted from Gen Ed 1093 : Who Lives, Who Dies, Who Cares? Reimagining Global Health | Fall 2020

On p 13 of Why not Socialism?, G. A. Cohen states that the principle of “socialist equality of opportunity” is a principle of justice. What is the principle of “socialist equality of opportunity,” why does Cohen think it is a principle of justice, why does he think it is a desirable principle, and why does he think it is feasible? Which part of his argument do you think is most vulnerable to objections? Formulate some objections and explore how Cohen could respond. Do you think the objections succeed, or is Cohen’s view correct?

Proceed as follows:   [1]  State what socialist equality of opportunity is, by way of contrast with the two other kinds of equality of opportunity identified by Cohen. Explain why Cohen thinks, as a matter of justice, socialist equality of opportunity is preferable to the other two, and explain why an additional principle of community is needed to supplement that principle of justice. Then assess whether Cohen offers additional reasons (beyond the superiority of his principle over the alternatives) as to why equality and community are desirable, both for the camping trip and society at large. In a next step briefly summarize what he says about the feasibility of the principle. Devote about two thirds of your discussion to the tasks sketched so far, and then devote the remaining third to your exploration of the objections to parts of Cohen’s argument and an exploration of their success.

General Guidance

In section, your TF will discuss general guidelines to writing a philosophy paper.   [2] Please also consult the “Advice on Written Assignments” posted on Canvas before writing the paper. Recall that you will write three papers in this course. The assignments get progressively more demanding. In the first paper, the emphasis is on reconstructing arguments, allowing you to develop the skill of logical reconstruction rather than narrative summary of a text. …The second paper goes beyond reconstruction, putting more emphasis on the critically evaluating arguments. The third paper gives you an opportunity to develop a well-reasoned defense in support of your own view regarding one of the central issues of the class.   [3]  

__________ [1]   Students are given clear advice about how to use evidence differently at different points in their assignment. [2]  Students are assured that they will learn guidelines for working with evidence and analysis in a more disciplinary kind of writing (with which many of them will likely be unfamiliar). [3]  The move from “reconstruction” to “critically evaluating” to “well-reasoned defense” signals a scaffolded development of ways to work with evidence, along with reasons why students are being are being asked to work with evidence in a certain way for this first essay, viz., “ to develop the skill of logical reconstruction."

Adapted from Gen Ed 1121 : Economic Justice | Spring 2020 Professor Mathias Risse

Research Requirements

All projects, regardless of which modality you adopt, will need to include   [1]

• an annotated bibliography that includes at least 5 scholarly sources. These sources can include scholarly articles, books, or websites. For a website, please check with the TFs to confirm the viability of it as a source.   [2] There are legitimately scholarly websites, but many content-related sites are not scholarly.
• a 1-page artist statement.

See “How tos_Annotated Bibliography_your Artist Statement” for specific instructions for both the annotated bibliography and the artist statement.   [3]

__________ [1]  Explicit guidance about what kinds of sources and how many to include [2]  Advice on how to get help evaluating whether a source counts as viable evidence [3]  Additional resources (tied to guidelines and process) that help explain the roles of evidence and analysis in the assignment

Adapted from Gen Ed 1099 : Pyramid Schemes: What Can Ancient Egyptian Civilization Teach Us? Professor Peter der Manuelian

Introduce yourself to another student in the class by making a virtual mixtape for them. ⋮ Your tape should contain the following (in any order):   [1]

• The greeting on the Golden Record that best describes you (or record your own)
• One piece of music included on the Golden Record
• Your personal summer hit of 2020
• A “found sound” (recorded in your environment that seems characteristic or interesting)
• A piece of music that best describes you
• Your favorite piece/song by a musician outside the US/Canada

Use these guidelines as a starting point for your mixtape. Feel free to get creative. The mixtape should say something important about YOU. (There will be no written text accompanying your file. The sounds have to say it all.)   [2]

__________ [1]  Students are given a clear checklist of what to include in their assignment. [2]  In this assignment, the evidence makes its argument through curation, rather than additional written analysis. Making sure students understand that particular relationship of evidence to analysis ahead of time frames the assignment’s purpose and genre.

Adapted from Gen Ed 1006 : Music from Earth | Fall 2020 Professor Alex Rehding

• DIY Guides for Analytical Writing Assignments

• Types of Assignments
• Style and Conventions
• Specific Guidelines
• Advice on Process
• Receiving Feedback

Assignment Decoder

• What is the best evidence and how to find it

Why is research evidence better than expert opinion alone?

In a broad sense, research evidence can be any systematic observation in order to establish facts and reach conclusions. Anything not fulfilling this definition is typically classified as “expert opinion”, the basis of which includes experience with patients, an understanding of biology, knowledge of pre-clinical research, as well as of the results of studies. Using expert opinion as the only basis to make decisions has proved problematic because in practice doctors often introduce new treatments too quickly before they have been shown to work, or they are too slow to introduce proven treatments.

However, clinical experience is key to interpret and apply research evidence into practice, and to formulate recommendations, for instance in the context of clinical guidelines. In other words, research evidence is necessary but not sufficient to make good health decisions.

Which studies are more reliable?

Not all evidence is equally reliable.

Any study design, qualitative or quantitative, where data is collected from individuals or groups of people is usually called a primary study. There are many types of primary study designs, but for each type of health question there is one that provides more reliable information.

For treatment decisions, there is consensus that the most reliable primary study is the randomised controlled trial (RCT). In this type of study, patients are randomly assigned to have either the treatment being tested or a comparison treatment (sometimes called the control treatment). Random really means random. The decision to put someone into one group or another is made like tossing a coin: heads they go into one group, tails they go into the other.

The control treatment might be a different type of treatment or a dummy treatment that shouldn't have any effect (a placebo). Researchers then compare the effects of the different treatments.

Large randomised trials are expensive and take time. In addition sometimes it may be unethical to undertake a study in which some people were randomly assigned not to have a treatment. For example, it wouldn't be right to give oxygen to some children having an asthma attack and not give it to others. In cases like this, other primary study designs may be the best choice.

Laboratory studies are another type of study. Newspapers often have stories of studies showing how a drug cured cancer in mice. But just because a treatment works for animals in laboratory experiments, this doesn't mean it will work for humans. In fact, most drugs that have been shown to cure cancer in mice do not work for people.

Very rarely we cannot base our health decisions on the results of studies. Sometimes the research hasn't been done because doctors are used to treating a condition in a way that seems to work. This is often true of treatments for broken bones and operations. But just because there's no research for a treatment doesn't mean it doesn't work. It just means that no one can say for sure.

Why we shouldn’t read studies

An enormous amount of effort is required to be able to identify and summarise everything we know with regard to any given health intervention. The amount of data has soared dramatically. A conservative estimation is there are more than 35,000 medical journals and almost 20 million research articles published every year. On the other hand, up to half of existing data might be unpublished.

How can anyone keep up with all this? And how can you tell if the research is good or not? Each primary study is only one piece of a jigsaw that may take years to finish. Rarely does any one piece of research answer either a doctor's, or a patient's questions.

Even though reading large numbers of studies is impractical, high-quality primary studies, especially RCTs, constitute the foundations of what we know, and they are the best way of advancing the knowledge. Any effort to support or promote the conduct of sound, transparent, and independent trials that are fully and clearly published is worth endorsing. A prominent project in this regard is the All trials initiative.

Why we should read systematic reviews

Most of the time a single study doesn't tell us enough. The best answers are found by combining the results of many studies.

A systematic review is a type of research that looks at the results from all of the good-quality studies. It puts together the results of these individual studies into one summary. This gives an estimate of a treatment's risks and benefits. Sometimes these reviews include a statistical analysis, called a meta-analysis , which combines the results of several studies to give a treatment effect.

Systematic reviews are increasingly being used for decision making because they reduce the probability of being misled by looking at one piece of the jigsaw. By being systematic they are also more transparent, and have become the gold standard approach to synthesise the ever-expanding and conflicting biomedical literature.

Systematic reviews are not fool proof. Their findings are only as good as the studies that they include and the methods they employ. But the best reviews clearly state whether the studies they include are good quality or not.

Three reasons why we shouldn’t read (most) systematic reviews

Firstly, systematic reviews have proliferated over time. From 11 per day in 2010, they skyrocketed up to 40 per day or more in 2015.[1][2] Some have described this production as having reached epidemic proportions where the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted.[3][4] So, finding more than one systematic review for a question is the rule more than the exception, and it is not unusual to find several dozen for the hottest questions.

Second, most systematic reviews address a narrow question. It is difficult to put them in the context of all of the available alternatives for an individual case. Reading multiple reviews to assess all of the alternatives is impractical, even more if we consider they are typically difficult to read for the average clinician, who will need to solve several questions each day.[5]

Third, systematic reviews do not tell you what to do, or what is advisable for a given patient or situation. Indeed, good systematic reviews explicitly avoid making recommendations.

So, even though systematic reviews play a key role in any evidence-based decision-making process, most of them are low-quality or outdated, and they rarely provide all the information needed to make decisions in the real world.

How to find the best available evidence?

Considering the massive amount of information available, we can quickly discard periodically reviewing our favourite journals as a means of sourcing the best available evidence.

The traditional approach to search for evidence has been using major databases, such as PubMed  or EMBASE . These constitute comprehensive sources including millions of relevant, but also irrelevant articles. Even though in the past they were the preferred approach to searching for evidence, information overload has made them impractical, and most clinicians would fail to find the best available evidence in this way, however hard they tried.

Another popular approach is simply searching in Google. Unfortunately, because of its lack of transparency, Google is not a reliable way to filter current best evidence from unsubstantiated or non-scientifically supervised sources.[6]

Three alternatives to access the best evidence

Alternative 1 - Pick the best systematic review Mastering the art of identifying, appraising, and applying high-quality systematic reviews into practice can be very rewarding. It is not easy, but once mastered it gives a view of the bigger picture: of what is known, and what is not known.

The best single source of highest-quality systematic reviews is produced by an international organisation called the Cochrane Collaboration, named after a well-known researcher.[4] They can be accessed at The Cochrane Library .

Unfortunately, Cochrane reviews do not cover all of the existing questions and they are not always up to date. Also, there might be non-Cochrane reviews out-performing Cochrane reviews.

There are many resources that facilitate access to systematic reviews (and other resources), such as Trip database , PubMed Health , ACCESSSS , or Epistemonikos (the Cochrane Collaboration maintains a comprehensive list of these resources).

Epistemonikos database is innovative both in simultaneously searching multiple resources and in indexing and interlinking relevant evidence. For example, Epistemonikos connects systematic reviews and their included studies, and thus allows clustering of systematic reviews based on the primary studies they have in common. Epistemonikos is also unique in offering an appreciable multilingual user interface, multilingual search, and translation of abstracts in more than nine languages.[6] This database includes several tools to compare systematic reviews, including the matrix of evidence, a dynamic table showing all of the systematic reviews, and the primary studies included in those reviews.

Additionally, Epistemonikos partnered with Cochrane, and during 2017 a combined search in both the Cochrane Library and Epistemonikos was released.

Alternative 2 - Read trustworthy guidelines Although systematic reviews can provide a synthesis of the benefits and harms of the interventions, they do not integrate these factors with patients’ values and preferences or resource considerations to provide a suggested course of action. Also, to fully address the questions, clinicians would need to integrate the information of several systematic reviews covering all the relevant alternatives and outcomes. Most clinicians will likely prefer guidance rather than interpreting systematic reviews themselves.

Trustworthy guidelines, especially if developed with high standards, such as the Grading of Recommendations, Assessment, Development, and Evaluation ( GRADE ) approach, offer systematic and transparent guidance in moving from evidence to recommendations.[7]

Many online guideline websites promote themselves as “evidence based”, but few have explicit links to research findings.[8] If they don’t have in-line references to relevant research findings, dismiss them. If they have, you can judge the strength of the commitment to evidence to support inference, checking whether statements are based on high-quality versus low-quality evidence using alternative 1 explained above.

Unfortunately, most guidelines have serious limitations or are outdated.[9][10] The exercise of locating and appraising the best guideline is time consuming. This is particularly challenging for generalists addressing questions from different conditions or diseases.

Alternative 3 - Use point-of-care tools Point-of-care tools, such as BMJ Best Practice, have been developed as a response to the genuine need to summarise the ever-expanding biomedical literature on an ever-increasing number of alternatives in order to make evidence-based decisions. In this competitive market, the more successful products have been those delivering innovative, user-friendly interfaces that improve the retrieval, synthesis, organisation, and application of evidence-based content in many different areas of clinical practice.

However, the same impossibility in catching up with new evidence without compromising quality that affects guidelines also affects point-of-care tools. Clinicians should become familiar with the point-of-care information resource they want or can access, and examine the in-line references to relevant research findings. Clinicians can easily judge the strength of the commitment to evidence checking whether statements are based on high-quality versus low-quality evidence using alternative 1 explained above. Comprehensiveness, use of GRADE approach, and independence are other characteristics to bear in mind when selecting among point-of-care information summaries.

A comprehensive list of these resources can be found in a study by Kwag et al .

Finding the best available evidence is more challenging than it was in the dawn of the evidence-based movement, and the main cause is the exponential growth of evidence-based information, in any of the flavours described above.

However, with a little bit of patience and practice, the busy clinician will discover evidence-based practice is far easier than it was 5 or 10 years ago. We are entering a stage where information is flowing between the different systems, technology is being harnessed for good, and the different players are starting to generate alliances.

The early adopters will surely enjoy the first experiments of living systematic reviews (high-quality, up-to-date online summaries of health research that are updated as new research becomes available), living guidelines, and rapid reviews tied to rapid recommendations, just to mention a few. [13][14][15]

It is unlikely that the picture of countless low-quality studies and reviews will change in the foreseeable future. However, it would not be a surprise if, in 3 to 5 years, separating the wheat from the chaff becomes trivial. Maybe the promise of evidence-based medicine of more effective, safer medical intervention resulting in better health outcomes for patients could be fulfilled.

Author: Gabriel Rada

Competing interests: Gabriel Rada is the co-founder and chairman of Epistemonikos database, part of the team that founded and maintains PDQ-Evidence, and an editor of the Cochrane Collaboration.

 Related Blogs

  Living Systematic Reviews: towards real-time evidence for health-care decision making

• Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010 Sep 21;7(9):e1000326. doi: 10.1371/journal.pmed.1000326
• Epistemonikos database [filter= systematic review; year=2015]. A Free, Relational, Collaborative, Multilingual Database of Health Evidence. https://www.epistemonikos.org/en/search?&q=*&classification=systematic-review&year_start=2015&year_end=2015&fl=14542 Accessed 5 Jan 2017.
• Ioannidis JP. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q. 2016 Sep;94(3):485-514. doi: 10.1111/1468-0009.12210.
• Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.
• Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014 May;174(5):710-8. doi: 10.1001/jamainternmed.2014.368.
• Agoritsas T, Vandvik P, Neumann I, Rochwerg B, Jaeschke R, Hayward R, et al. Chapter 5: finding current best evidence. In: Users' guides to the medical literature: a manual for evidence-based clinical practice. Chicago: MacGraw-Hill, 2014.
• Guyatt GH, Oxman AD, Vist GE, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi: 10.1136/bmj.39489.470347
• Neumann I, Santesso N, Akl EA, Rind DM, Vandvik PO, Alonso-Coello P, Agoritsas T, Mustafa RA, Alexander PE, Schünemann H, Guyatt GH. A guide for health professionals to interpret and use recommendations in guidelines developed with the GRADE approach. J Clin Epidemiol. 2016 Apr;72:45-55. doi: 10.1016/j.jclinepi.2015.11.017
• Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J, Schunemann H. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010 Dec;19(6):e58. doi: 10.1136/qshc.2010.042077
• Martínez García L, Sanabria AJ, García Alvarez E, Trujillo-Martín MM, Etxeandia-Ikobaltzeta I, Kotzeva A, Rigau D, Louro-González A, Barajas-Nava L, Díaz Del Campo P, Estrada MD, Solà I, Gracia J, Salcedo-Fernandez F, Lawson J, Haynes RB, Alonso-Coello P; Updating Guidelines Working Group. The validity of recommendations from clinical guidelines: a survival analysis. CMAJ. 2014 Nov 4;186(16):1211-9. doi: 10.1503/cmaj.140547
• Kwag KH, González-Lorenzo M, Banzi R, Bonovas S, Moja L. Providing Doctors With High-Quality Information: An Updated Evaluation of Web-Based Point-of-Care Information Summaries. J Med Internet Res. 2016 Jan 19;18(1):e15. doi: 10.2196/jmir.5234
• Banzi R, Cinquini M, Liberati A, Moschetti I, Pecoraro V, Tagliabue L, Moja L. Speed of updating online evidence based point of care summaries: prospective cohort analysis. BMJ. 2011 Sep 23;343:d5856. doi: 10.1136/bmj.d5856
• Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, Gruen RL. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 2014 Feb 18;11(2):e1001603. doi: 10.1371/journal.pmed.1001603
• Vandvik PO, Brandt L, Alonso-Coello P, Treweek S, Akl EA, Kristiansen A, Fog-Heen A, Agoritsas T, Montori VM, Guyatt G. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest. 2013 Aug;144(2):381-9. doi: 10.1378/chest.13-0746
• Vandvik PO, Otto CM, Siemieniuk RA, Bagur R, Guyatt GH, Lytvyn L, Whitlock R, Vartdal T, Brieger D, Aertgeerts B, Price S, Foroutan F, Shapiro M, Mertz R, Spencer FA. Transcatheter or surgical aortic valve replacement for patients with severe, symptomatic, aortic stenosis at low to intermediate surgical risk: a clinical practice guideline. BMJ. 2016 Sep 28;354:i5085. doi: 10.1136/bmj.i5085

Discuss EBM

• What does evidence-based actually mean?
• Simply making evidence simple
• Six proposals for EBMs future
• Promoting informed healthcare choices by helping people assess treatment claims
• The blind leading the blind in the land of risk communication
• Transforming the communication of evidence for better health
• Clinical search, big data, and the hunt for meaning
• Living systematic reviews: towards real-time evidence for health-care decision making
• The rise of rapid reviews
• Evidence for the Brave New World on multimorbidity
• Genetics and personalised medicine: where’s the revolution?
• Policy, practice, and politics
• The straw men of integrative health and alternative medicine
• Where’s the evidence for teaching evidence-based medicine?

EBM Toolkit home

Learn, Practise, Discuss, Tools

Developing Deeper Analysis & Insights

Analysis is a central writing skill in academic writing. Essentially, analysis is what writers do with evidence to make meaning of it. While there are specific disciplinary types of analysis (e.g., rhetorical, discourse, close reading, etc.), most analysis involves zooming into evidence to understand how the specific parts work and how their specific function might relate to a larger whole. That is, we usually need to zoom into the details and then reflect on the larger picture. In this writing guide, we cover analysis basics briefly and then offer some strategies for deepening your analysis. Deepening your analysis means pushing your thinking further, developing a more insightful and interesting answer to the “so what?” question, and elevating your writing.

Analysis Basics

Questions to Ask of the Text:

• Is the evidence fully explained and contextualized? Where in the text/story does this evidence come from (briefly)? What do you think the literal meaning of the quote/evidence is and why? Why did you select this particular evidence?
• Are you selecting a long enough quote to work with and analyze? While over-quoting can be a problem, so too can under-quoting.
• Do you connect each piece of evidence explicitly to the claim or focus of the paper?

Strategies & Explanation

• Sometimes turning the focus of the paper into a question can really help someone to figure out how to work with evidence. All evidence should answer the question--the work of analysis is explaining how it answers the question.
• The goal of evidence in analytical writing is not just to prove that X exists or is true, but rather to show something interesting about it--to push ideas forward, to offer insights about a quote. To do this, sometimes having a full sentence for a quote helps--if a writer is only using single-word quotes, for example, they may struggle to make meaning out of it.

Deepening Analysis

Not all of these strategies work every time, but usually employing one of them is enough to really help elevate the ideas and intellectual work of a paper:

• Bring the very best point in each paragraph into the topic sentence. Often these sentences are at the very end of a paragraph in a solid draft. When you bring it to the front of the paragraph, you then need to read the paragraph with the new topic sentence and reflect on: what else can we say about this evidence? What else can it show us about your claim?
• Complicate the point by adding contrasting information, a different perspective, or by naming something that doesn’t fit. Often we’re taught that evidence needs to prove our thesis. But, richer ideas emerge from conflict, from difference, from complications. In a compare and contrast essay, this point is very easy to see--we get somewhere further when we consider how two things are different. In an analysis of a single text, we might look at a single piece of evidence and consider: how could this choice the writer made here be different? What other choices could the writer have made and why didn’t they? Sometimes naming what isn’t in the text can help emphasize the importance of a particular choice.
• Shift the focus question of the essay and ask the new question of each piece of evidence. For example, a student is looking at examples of language discrimination (their evidence) in order to make an argument that answers the question: what is language discrimination? Questions that are definitional (what is X? How does Y work? What is the problem here?) can make deeper analysis challenging. It’s tempting to simply say the equivalent of “Here is another example of language discrimination.” However, a strategy to help with this is to shift the question a little bit. So perhaps the paragraphs start by naming different instances of language discrimination, but the analysis then tackles questions like: what are the effects of language discrimination? Why is language discrimination so problematic in these cases? Who perpetuates language discrimination and how? In a paper like this, it’s unlikely you can answer all of those questions--but, selecting ONE shifted version of a question that each paragraph can answer, too, helps deepen the analysis and keeps the essay focused.
• Examine perspective--both the writer’s and those of others involved with the issue. You might reflect on your own perspectives as a unique audience/reader. For example, what is illuminated when you read this essay as an engineer? As a person of color? As a first-generation student at Cornell? As an economically privileged person? As a deeply religious Christian? In order to add perspective into the analysis, the writer has to name these perspectives with phrases like: As a religious undergraduate student, I understand X to mean… And then, try to explain how the specificity of your perspective illuminates a different reading or understanding of a term, point, or evidence. You can do this same move by reflecting on who the intended audience of a text is versus who else might be reading it--how does it affect different audiences differently? Might that be relevant to the analysis?
• Qualify claims and/or acknowledge limitations. Before college level writing and often in the media, there is a belief that qualifications and/or acknowledging the limitations of a point adds weakness to an argument. However, this actually adds depth, honesty, and nuance to ideas. It allows you to develop more thoughtful and more accurate ideas. The questions to ask to help foster this include: Is this always true? When is it not true? What else might complicate what you’ve said? Can we add nuance to this idea to make it more accurate? Qualifications involve words like: sometimes, may effect, often, in some cases, etc. These terms are not weak or to be avoided, they actually add accuracy and nuance.

A Link to a  PDF Handout of this Writing Guide 

Critically Analyzing Information Sources: Critical Appraisal and Analysis

• Critical Appraisal and Analysis

Initial Appraisal : Reviewing the source

• What are the author's credentials--institutional affiliation (where he or she works), educational background, past writings, or experience? Is the book or article written on a topic in the author's area of expertise? You can use the various Who's Who publications for the U.S. and other countries and for specific subjects and the biographical information located in the publication itself to help determine the author's affiliation and credentials.
• Has your instructor mentioned this author? Have you seen the author's name cited in other sources or bibliographies? Respected authors are cited frequently by other scholars. For this reason, always note those names that appear in many different sources.
• Is the author associated with a reputable institution or organization? What are the basic values or goals of the organization or institution?

B. Date of Publication

• When was the source published? This date is often located on the face of the title page below the name of the publisher. If it is not there, look for the copyright date on the reverse of the title page. On Web pages, the date of the last revision is usually at the bottom of the home page, sometimes every page.
• Is the source current or out-of-date for your topic? Topic areas of continuing and rapid development, such as the sciences, demand more current information. On the other hand, topics in the humanities often require material that was written many years ago. At the other extreme, some news sources on the Web now note the hour and minute that articles are posted on their site.

C. Edition or Revision

Is this a first edition of this publication or not? Further editions indicate a source has been revised and updated to reflect changes in knowledge, include omissions, and harmonize with its intended reader's needs. Also, many printings or editions may indicate that the work has become a standard source in the area and is reliable. If you are using a Web source, do the pages indicate revision dates?

D. Publisher

Note the publisher. If the source is published by a university press, it is likely to be scholarly. Although the fact that the publisher is reputable does not necessarily guarantee quality, it does show that the publisher may have high regard for the source being published.

E. Title of Journal

Is this a scholarly or a popular journal? This distinction is important because it indicates different levels of complexity in conveying ideas. If you need help in determining the type of journal, see Distinguishing Scholarly from Non-Scholarly Periodicals . Or you may wish to check your journal title in the latest edition of Katz's Magazines for Libraries (Olin Reference Z 6941 .K21, shelved at the reference desk) for a brief evaluative description.

Critical Analysis of the Content

Having made an initial appraisal, you should now examine the body of the source. Read the preface to determine the author's intentions for the book. Scan the table of contents and the index to get a broad overview of the material it covers. Note whether bibliographies are included. Read the chapters that specifically address your topic. Reading the article abstract and scanning the table of contents of a journal or magazine issue is also useful. As with books, the presence and quality of a bibliography at the end of the article may reflect the care with which the authors have prepared their work.

A. Intended Audience

What type of audience is the author addressing? Is the publication aimed at a specialized or a general audience? Is this source too elementary, too technical, too advanced, or just right for your needs?

B. Objective Reasoning

• Is the information covered fact, opinion, or propaganda? It is not always easy to separate fact from opinion. Facts can usually be verified; opinions, though they may be based on factual information, evolve from the interpretation of facts. Skilled writers can make you think their interpretations are facts.
• Does the information appear to be valid and well-researched, or is it questionable and unsupported by evidence? Assumptions should be reasonable. Note errors or omissions.
• Are the ideas and arguments advanced more or less in line with other works you have read on the same topic? The more radically an author departs from the views of others in the same field, the more carefully and critically you should scrutinize his or her ideas.
• Is the author's point of view objective and impartial? Is the language free of emotion-arousing words and bias?

C. Coverage

• Does the work update other sources, substantiate other materials you have read, or add new information? Does it extensively or marginally cover your topic? You should explore enough sources to obtain a variety of viewpoints.
• Is the material primary or secondary in nature? Primary sources are the raw material of the research process. Secondary sources are based on primary sources. For example, if you were researching Konrad Adenauer's role in rebuilding West Germany after World War II, Adenauer's own writings would be one of many primary sources available on this topic. Others might include relevant government documents and contemporary German newspaper articles. Scholars use this primary material to help generate historical interpretations--a secondary source. Books, encyclopedia articles, and scholarly journal articles about Adenauer's role are considered secondary sources. In the sciences, journal articles and conference proceedings written by experimenters reporting the results of their research are primary documents. Choose both primary and secondary sources when you have the opportunity.

D. Writing Style

Is the publication organized logically? Are the main points clearly presented? Do you find the text easy to read, or is it stilted or choppy? Is the author's argument repetitive?

E. Evaluative Reviews

• Locate critical reviews of books in a reviewing source , such as the Articles & Full Text , Book Review Index , Book Review Digest, and ProQuest Research Library . Is the review positive? Is the book under review considered a valuable contribution to the field? Does the reviewer mention other books that might be better? If so, locate these sources for more information on your topic.
• Do the various reviewers agree on the value or attributes of the book or has it aroused controversy among the critics?
• For Web sites, consider consulting this evaluation source from UC Berkeley .

Permissions Information

If you wish to use or adapt any or all of the content of this Guide go to Cornell Library's Research Guides Use Conditions to review our use permissions and our Creative Commons license.

• Next: Tips >>
• Last Updated: Apr 18, 2022 1:43 PM
• URL: https://guides.library.cornell.edu/critically_analyzing

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

Planning and Writing a Research Paper

Collect Evidence

The evidence you collect will shape your research paper tremendously. You will have to decide what evidence is appropriate for your audience, purpose, and thesis. To help you make this decision, consider what kind of appeal you are making to your audience—logical, emotional, or ethical. Click on the tabs below for more information.

• LOGOS OR LOGICAL APPEAL
• PATHOS OR EMOTIONAL APPEAL
• ETHOS OR ETHICAL APPEAL

You appeal to the reader’s intellect through factual or objective evidence. 

You appeal to the reader’s feelings and their heart. 

You appeal to the reader’s sense of justice, fair play or trust.  The writer is seen to have authority regarding what they are writing about.  

EXAMPLE OF A RESEARCH QUESTION AND ARGUMENT

Here is an example that describes research evidence to support an observable trend; this collection of evidence appeals to the readers’ logic and intelligence.

Research Question:  What trends in research led to the computer industry segmentation that has occurred since the 1960s?

Argument Appealing to Logic and Intelligence:   You might learn from early research that the initial phase of the US space program generated much interest in robotics and in programmable machines. That interest led to government funding for research in these areas during the 1960s. This evidence might suggest to you that the government’s role in financing research was instrumental in nurturing the fledgling computer industry of the 1960s. At the same time, you might learn that the government came into conflict with proponents of the growing industry in the 1970s by attempting to curtail domination by a single manufacturer through enforcement of the Sherman Antitrust Act. This evidence, in turn, will help you write about and explain the industry segmentation that has occurred since the 1960s, with its attendant competitive emphasis on constant improvement and innovation, realized in paradigm shifts such as those that occur in object-oriented programming.  

Key Takeaways

• The evidence you collect will shape your research paper tremendously.
• To help you make this decision, consider what kind of appeal you are making to your audience—logical, emotional, or ethical. 

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

• Library databases
• Library website

Evidence-Based Research: Evidence Types

Introduction.

Not all evidence is the same, and appraising the quality of the evidence is part of evidence-based practice research. The hierarchy of evidence is typically represented as a pyramid shape, with the smaller, weaker and more abundant research studies near the base of the pyramid, and systematic reviews and meta-analyses at the top with higher validity but a more limited range of topics.

Several versions of the evidence pyramid have evolved with different interpretations, but they are all comprised of the types of evidence discussed on this page. Walden's Nursing 6052 Essentials of Evidence-Based Practice class currently uses a simplified adaptation of the Johns Hopkins model .

Evidence Levels:

Level I:  Experimental, randomized controlled trial (RCT), systematic review RTCs with or without meta-analysis

Level II:  Quasi-experimental studies, systematic review of a combination of RCTs and quasi-experimental studies, or quasi-experimental studies only, with or without meta-analysis

Level III:  Nonexperimental, systematic review of RCTs, quasi-experimental with/without meta-analysis, qualitative, qualitative systematic review with/without meta-synthesis  (see Daly 2007 for a sample qualitative hierarchy) 

Level IV : Respected authorities’ opinions, nationally recognized expert committee or consensus panel reports based on scientific evidence

Level V:  Literature reviews, quality improvement, program evaluation, financial evaluation, case reports, nationally recognized expert(s) opinion based on experiential evidence

Systematic review

What is a systematic review.

A systematic review is a type of publication that addresses a clinical question by analyzing research that fits certain explicitly-specified criteria. The criteria for inclusion is usually based on research from clinical trials and observational studies. Assessments are done based on stringent guidelines, and the reviews are regularly updated. These are usually considered one of the highest levels of evidence and usually address diagnosis and treatment questions.

Benefits of Systematic Reviews

Systematic reviews refine and reduce large amounts of data and information into one document, effectively summarizing the evidence to support clinical decisions. Since they are typically undertaken by a entire team of experts, they can take months or even years to complete, and must be regularly updated. The teams are usually comprised of content experts, an experienced searcher, a bio-statistician, and a methodologist. The team develops a rigorous protocol to thoroughly locate, identify, extract, and analyze all of the evidence available that addresses their specific clinical question.

As systematic reviews become more frequently published, concern over quality led to the PRISMA Statement to establish a minimum set of items for reporting in systematic reviews and meta-analyses.

Many systematic reviews also contain a meta-analysis.

What is a Meta-Analysis?

Meta-analysis is a particular type of systematic review that focuses on selecting and reviewing quantitative research. Researchers conducting a meta-analysis combine the results of several independent studies and reviews to produce a synthesis where possible. These publications aim to assist in making decisions about a particular therapy.

Benefits of Meta-Analysis

A meta-analysis synthesizes large amounts of data using a statistical examination. This type of analysis provides for some control between studies and generalized application to the population.

To learn how to find systematic reviews in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Systematic Reviews

Further reading

• Cochrane Handbook for Systematic Reviews of Interventions *updated 2022

Guidelines & summaries

Practice guidelines.

A practice guideline is a systematically-developed statement addressing common patient health care decisions in specific clinical settings and circumstances.  They should be valid, reliable, reproducible, clinically applicable, clear and flexible. Documentation must be included and referenced. Practice guidelines may come from organizations, associations, government entities, and hospitals/health systems.

ECRI Guidelines Trust

Best Evidence Topics

Best evidence topics are sometimes referred to as Best BETs. These topics are developed and supported for situations or setting when the high levels of evidence don't fit or are unavailable. They originated from emergency medicine providers' need to conduct rapid evidence-based clinical decisions.

Critically-Appraised Topics

Critically-appraised topics are a standardized one- to two-page summary of the evidence supporting a clinical question. They include a critique of the literature and statement of relevant results. They can be found online in many repositories.

To learn how to find critically-appraised topics in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Critically-Appraised Topics

Critically-Appraised Articles

Critically-appraised articles are individual articles by authors that evaluate and synopsize individual research studies. ACP Journal Club is the most well known grouping of titles that include critically appraised articles.

To learn how to find critically-appraised articles in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Critically-Appraised Articles

Randomized controlled trial

A randomized controlled trial (RCT) is a clinical trial in which participants are randomly assigned to either the treatment group or control group. This random allocation of participants helps to reduce any possible selection bias and makes the RCT a high level of evidence. Having a control group, which receives no treatment or a placebo treatment, to compare the treatment group against allows researchers to observe the potential efficacy of the treatment when other factors remain the same. Randomized controlled trials are quantitative studies and are often the only studies included in systematic reviews.

To learn how to find randomize controlled trials, please see our CINAHL & MEDLINE help pages:

• CINAHL Search Help: Randomized Controlled Trials
• MEDLINE Search Help: Randomized Controlled Trials

Cohort study

A cohort study is an observational longitudinal study that analyzes risk factors and outcomes by following a group (cohort) that share a common characteristic or experience over a period of time.

Cohort studies can be retrospective, looking back over time at data that has already been collected, or can be prospective, following a group forward into the future and collecting data along the way.

While cohort studies are considered a lower level of evidence than randomized controlled trials, they may be the only way to study certain factors ethically. For example, researchers may follow a cohort of people who are tobacco smokers and compare them to a cohort of non-smokers looking for outcomes. That would be an ethical study. It would be highly unethical, however, to design a randomized controlled trial in which one group of participants are forced to smoke in order to compare outcomes.

To learn how to find cohort studies, please see our CINAHL and MEDLINE help pages:

• CINAHL Search Help: Cohort Studies
• MEDLINE Search Help: Cohort Studies

Case-controlled studies

Case-controlled studies are a type of observational study that looks at patients who have the same disease or outcome. The cases are those who have the disease or outcome while the controls do not. This type of study evaluates the relationship between diseases and exposures by retrospectively looking back to investigate what could potentially cause the disease or outcome.

To learn how to find case-controlled studies, please see our CINAHL and MEDLINE help pages:

• CINAHL Search Help: Case Studies
• MEDLINE Search Help: Case Studies

Background information & expert opinion

Background information and expert opinion can be found in textbooks or medical books that provide basic information on a topic. They can be helpful to make sure you understand a topic and are familiar with terms associated with it.

To learn about accessing background information, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Background Information & Expert Opinion
• Previous Page: Levels of Evidence Pyramid
• Next Page: CINAHL Search Help
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Using Research and Evidence

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

What type of evidence should I use?

There are two types of evidence.

First hand research is research you have conducted yourself such as interviews, experiments, surveys, or personal experience and anecdotes.

Second hand research is research you are getting from various texts that has been supplied and compiled by others such as books, periodicals, and Web sites.

Regardless of what type of sources you use, they must be credible. In other words, your sources must be reliable, accurate, and trustworthy.

How do I know if a source is credible?

You can ask the following questions to determine if a source is credible.

Who is the author? Credible sources are written by authors respected in their fields of study. Responsible, credible authors will cite their sources so that you can check the accuracy of and support for what they've written. (This is also a good way to find more sources for your own research.)

How recent is the source? The choice to seek recent sources depends on your topic. While sources on the American Civil War may be decades old and still contain accurate information, sources on information technologies, or other areas that are experiencing rapid changes, need to be much more current.

What is the author's purpose? When deciding which sources to use, you should take the purpose or point of view of the author into consideration. Is the author presenting a neutral, objective view of a topic? Or is the author advocating one specific view of a topic? Who is funding the research or writing of this source? A source written from a particular point of view may be credible; however, you need to be careful that your sources don't limit your coverage of a topic to one side of a debate.

What type of sources does your audience value? If you are writing for a professional or academic audience, they may value peer-reviewed journals as the most credible sources of information. If you are writing for a group of residents in your hometown, they might be more comfortable with mainstream sources, such as Time or Newsweek . A younger audience may be more accepting of information found on the Internet than an older audience might be.

Be especially careful when evaluating Internet sources! Never use Web sites where an author cannot be determined, unless the site is associated with a reputable institution such as a respected university, a credible media outlet, government program or department, or well-known non-governmental organizations. Beware of using sites like Wikipedia , which are collaboratively developed by users. Because anyone can add or change content, the validity of information on such sites may not meet the standards for academic research.

• Departments and Units
• Majors and Minors
• LSA Course Guide
• LSA Gateway

Search: {{$root.lsaSearchQuery.q}}, Page {{$root.page}}

• Accessibility
• Undergraduates
• Instructors
• Alums & Friends

• ★ Writing Support
• Minor in Writing
• First-Year Writing Requirement
• Transfer Students
• Writing Guides
• Peer Writing Consultant Program
• Upper-Level Writing Requirement
• Writing Prizes
• International Students
• ★ The Writing Workshop
• Dissertation ECoach
• Fellows Seminar
• Dissertation Writing Groups
• Rackham / Sweetland Workshops
• Dissertation Writing Institute
• Guides to Teaching Writing
• Teaching Support and Services
• Support for FYWR Courses
• Support for ULWR Courses
• Writing Prize Nominating
• Alums Gallery
• Commencement Archive
• Giving Opportunities
• How Do I Effectively Integrate Textual Evidence?
• How Do I Make Sure I Understand an Assignment?
• How Do I Decide What I Should Argue?
• How Can I Create Stronger Analysis?
• How Do I Write a Great Title?
• What Exactly is an Abstract?
• How Do I Present Findings From My Experiment in a Report?
• What is a Run-on Sentence & How Do I Fix It?
• How Do I Check the Structure of My Argument?
• How Do I Write an Intro, Conclusion, & Body Paragraph?
• How Do I Incorporate Quotes?
• How Can I Create a More Successful Powerpoint?
• How Can I Create a Strong Thesis?
• How Can I Write More Descriptively?
• How Do I Incorporate a Counterargument?
• How Do I Check My Citations?

See the bottom of the main Writing Guides page for licensing information.

Academic writing often requires students to use evidence, and learning how to use evidence effectively is an important skill for college writers to master. Often, the evidence college writers are asked to use comes from their textbooks, course readings, or other written work by professional scholars. It is important to learn how to use these writings responsibly and accurately.

General Considerations

There are three methods of incorporating the writing of others into your paper as evidence:

• quotation , which is anything from a word to several sentences taken word-for-word from the original source and enclosed in quotation marks
• paraphrase , which is a rephrasing in your own voice and sentence structure of one portion of the original source and is about the same length as the original sentence or sentences you are paraphrasing
• summary , which is shorter than the original source and gives the text’s central idea in your own words

Some words to use in signal phrases are argues, asserts, contends, emphasizes, explains, observes, suggests, writes.

In what follows, you will learn some strategies for using these methods of incorporating evidence into your paper.

In Practice

Quoting When you use a q uotation as evidence, you should integrate it into your own writing using a “signal phrase.” Take, for example, this quotation, taken from page 418 of the essay “Prejudice and the Individual” by Gordon Allport: “Much prejudice is caught rather than directly taught.” Here are three ways to integrate Allport’s quotation into a sentence of your own with a signal phrase:

Allport claims that “prejudice is caught rather than directly taught” (418). “Much prejudice is caught rather than directly taught,” claims Allport (418). “Much prejudice,” Allport claims, “is caught rather than directly taught” (418).

You can adapt a quotation to fit your own paragraph and sentence structure by making small changes to words and indicating those changes with square brackets. Say, for example, you liked this quotation from Allport:

“It should be added that overgeneralized prejudgments of this sort are prejudices only if they are not reversible when exposed to new knowledge” (417).

However, you want to apply Allport’s words to a specific example of your own. You could adapt the quotation like this:

The young man in my example was not prejudiced, according to Allport’s definition; his opinion was “reversible when [he was] exposed to new knowledge” (417).

You can also use ellipses to indicate that you have left irrelevant words out of a quotation. Again, say you wanted to use this quotation from Allport:

“The best opinion today says that if we eliminate discrimination, then—as people become acquainted with one another on equal terms—attitudes are likely to change, perhaps more rapidly than through the continued preaching or teaching of tolerance” (417).

But the middle part is less important to your paper than what Allport says at the start and the end. You could modify the quotation like this:

“The best opinion today says that if we eliminate discrimination . . . attitudes are likely to change, perhaps more rapidly than through the continued preaching or teaching of tolerance” (417).

Longer quotations must be formatted in a special way; usually, they are indented from the left margin and/or single-spaced. Depending on what citation style you use, guidelines differ regarding what defines a long quotation and how a long quotation should be formatted. Typically, a quotation of four or five lines is considered long.

Paraphrasing To paraphrase a source for use as evidence, you should use as little of the original language as possible and put the passage in your own voice and sentence structure. Also, because paraphrasing involves wrapping your words around someone else’s idea, people often forget to give credit to the author. Even though a paraphrase is in your words, it is not your idea. Remember to cite your source when you paraphrase. Here is another quotation from Allport and an example of weak and strong paraphrase:

“Education combats easy overgeneralizations, and as the educational level rises we find a reduction in stereotyped thinking” (Allport 422).

WEAK PARAPHRASE: Learning fights against stereotypes, and as more people are more educated we notice a decrease in prejudice (422).

STRONG PARAPHRASE: Allport explains that the more we learn, the harder we will find it to make unfair assumptions about groups of people, which means as more people pursue more education, prejudice decreases (422).

In the weak example above, you can see the sentence structure in the paraphrase is very similar to the quotation—notice, for instance, the use in both the original sentence and the weak paraphrase of a comma plus the conjunction “and.” Also, the replacement of Allport’s words with synonyms makes the paraphrase too close to the original—Allport’s “education” is replaced with “learning” in the paraphrase; his “combats” is exchanged for “fights”; “overgeneralizations” becomes “stereotypes.” The strong example above does a better job of restating Allport’s idea in a new sentence structure and without simple word substitution. Also, notice the weak paraphrase does not give Allport credit by mentioning him, but the strong one does.

Summarizing When you summarize another writer’s idea to use as evidence in a paper of your own, you are taking the essence of the writer’s idea and stating it more briefly, with less detail and explanation, than in the original. You may summarize an article or a chapter, or even a book, in a sentence, a paragraph, a page, or more—the purpose of your summary should dictate how specific you are. Summaries should be mostly in your own words, but often summaries include quotations or paraphrases when it is necessary to highlight a certain key point. When you are writing a summary, you need to be very careful not to use the original writer’s words without putting those words in quotation marks. You also need to be sure that when you summarize, you are fairly representing the original writer’s main idea. Here is a paragraph from Allport and examples of weak and strong summary:

“While discrimination ultimately rests on prejudice, the two processes are not identical. Discrimination denies people their natural or legal rights because of their membership in some unfavored group. Many people discriminate automatically without being prejudiced; and others, the “gentle people of prejudice,” feel irrational aversion, but are careful not to show it in discriminatory behavior. Yet in general, discrimination reinforces prejudices, and prejudices provide rationalizations for discrimination. The two concepts are most distinct when it comes to seeking remedies. The corrections for discrimination are legal, or lie in a direct change of social practices; whereas the remedy for prejudice lies in education and the conversion of attitudes. The best opinion today says that if we eliminate discrimination, then—as people become acquainted with one another on equal terms—attitudes are likely to change, perhaps more rapidly than through the continued preaching or teaching of tolerance.” (Allport 417)

WEAK SUMMARY: Discrimination is when people are denied their rights because they belong to some unfavored group, and it is addressed with legal action or a change in social practices. Eliminating discrimination from society would have a drastic effect on social attitudes overall, according to Allport (417).

STRONG SUMMARY: Allport explains that discrimination occurs when an individual is refused rights because he or she belongs to a group which is the object of prejudice. In this way, discrimination reinforces prejudice, but if instances of discrimination are ruled illegal or seen as socially unacceptable, prejudice will likely decrease along with discrimination (417).

You will notice that the weak summary above uses exact words and phrases from the source (“unfavored group,” “social practices”) and also some words and phrases very close to the original (“when people are denied,” “eliminating discrimination”). It does not effectively restate the original in different language. It also does not fairly represent the complete idea of the source paragraph: it does not explain the relationship between discrimination and prejudice, an important part of what Allport says. The strong example does a better job using independent language and fairly conveying Allport’s point.

How to choose which method of incorporating evidence to use These methods of incorporating evidence into your paper are helpful in different ways. Think carefully about what you need each piece of evidence to do for you in your paper, then choose the method that most suits your needs.

You should use a quotation if

• you are relying on the reputation of the writer of the original source to give authority or credibility to your paper.
• the original wording is so remarkable that paraphrasing would diminish it.

A paraphrase is a good choice if

• you need to provide a supporting fact or detail but the original writer’s exact words are not important.
• you need to use just one specific idea from a source and the rest of the source is not as important.

Summary is useful when

• you need to give an overview of a source to orient your reader.
• you want to provide background that leads up to the point of your paper.

Last but certainly not least, remember that anytime you use another person’s ideas or language, you must give credit to that person. If you do not know the name of the person whose idea or language you are using, you must still give credit by referring to a title or any such available information. You should always check with your instructor to see what method of citing and documenting sources you should use. The examples on this handout are cited using MLA style.

The sample text in these exercises is Holly Devor’s “Gender Role Behaviors and Attitudes.”

1. Read the paragraph from Devor below, then identify which summary of it is weak and which is strong.

“Body postures and demeanors which communicate subordinate status and vulnerability to trespass through a message of "no threat" make people appear to be feminine. They demonstrate subordination through a minimizing of spatial use: people appear to be feminine when they keep their arms closer to their bodies, their legs closer together, and their torsos and heads less vertical than do masculine-looking individuals. People also look feminine when they point their toes inward and use their hands in small or childlike gestures.” (486)

A. Devor argues that body language suggests a great deal about gender and power in our society. People who minimize the body space they occupy and whose physical gestures are minimal and unobtrusive appear inferior and feminine (486).

B. Devor says that body postures and demeanors that imply weakness make people look feminine. Minimizing the space one takes up and using infantile gestures also makes one appear feminine (486).

2. Read the sentence from Devor below, then identify which paraphrase of it is weak and which is strong.

“They demonstrate subordination through a minimizing of spatial use: people appear to be feminine when they keep their arms closer to their bodies, their legs closer together, and their torsos and heads less vertical than do masculine-looking individuals.” (486)

A. Devor explains that people demonstrate a lesser position by using less space, keeping arms close, legs together, and head less upright (486).

B. According to Devor, taking up less space with one’s body—keeping arms and legs close and hunching to reduce height—makes one appear inferior and implies femininity (486).

3. The quotations of Devor below, taken from the paragraph in exercise 1, contain technical errors. Identify and correct them.

A. Devor argues that “[b]ody postures and demeanors which communicate subordinate status and vulnerability make people appear to be feminine” (486).

B. The actress looked particularly feminine because she “point their toes inward and use their hands in small or childlike gestures” (486).

C. Devor claims that “using their hands in small or childlike gestures” makes people look feminine (486).

Answers: 1. A. STRONG B. WEAK – This example uses too many exact words and phrases from the original.

2. A. WEAK – This example uses too many exact words and phrases from the source, and its sentence structure is also too close to the original. B. STRONG

3. A. Devor argues that “[b]ody postures and demeanors which communicate subordinate status and vulnerability . . . make people appear to be feminine.” B. The actress looked particularly feminine because she “point[s her] toes inward and use[s her] hands in small or childlike gestures.” C. Devor claims that “us[ing] their hands in small or childlike gestures” makes people look feminine.

Allport, Gordon, “Prejudice and the Individual,” in The Borozoi College Reader , 6th ed. Eds. Charles Muscatine and Marlene Griffith (New York: Alfred A Knopf, 1988): 416-22.

Devor, Holly, “Gender Role Behaviors and Attitudes,” in Signs of Life in the USA: Readings on Popular Culture for Writers , 4th ed. Eds. Sonia Maasik and Jack Solomon (New York: Bedford / St Martin's, 2003): 484-89.

• Information For
• Prospective Students
• Current Students
• Faculty and Staff
• Alumni and Friends
• More about LSA
• How Do I Apply?
• LSA Magazine
• Student Resources
• Academic Advising
• Global Studies
• LSA Opportunity Hub
• Social Media
• Update Contact Info
• Privacy Statement
• Report Feedback

12.1 Introducing Research and Research Evidence

Learning outcomes.

By the end of this section, you will be able to:

• Articulate how research evidence and sources are key rhetorical concepts in presenting a position or an argument.
• Locate and distinguish between primary and secondary research materials.
• Implement methods and technologies commonly used for research and communication within various fields.

The writing tasks for this chapter and the next two chapters are based on argumentative research. However, not all researched evidence (data) is presented in the same genre. You may need to gather evidence for a poster, a performance, a story, an art exhibit, or even an architectural design. Although the genre may vary, you usually will be required to present a perspective , or viewpoint, about a debatable issue and persuade readers to support the “validity of your viewpoint,” as discussed in Position Argument: Practicing the Art of Rhetoric . Remember, too, that a debatable issue is one that has more than a single perspective and is subject to disagreement.

The Research Process

Although individual research processes are rhetorically situated, they share some common aspects:

• Interest. The researcher has a genuine interest in the topic. It may be difficult to fake curiosity, but it is possible to develop it. Some academic assignments will allow you to pursue issues that are personally important to you; others will require you to dive into the research first and generate interest as you go.
• Questions. The researcher asks questions. At first, these questions are general. However, as researchers gain more knowledge, the questions become more sharply focused. No matter what your research assignment is, begin by articulating questions, find out where the answers lead, and then ask still more questions.
• Answers. The researcher seeks answers from people as well as from print and other media. Research projects profit when you ask knowledgeable people, such as librarians and other professionals, to help you answer questions or point you in directions to find answers. Information about research is covered more extensively in Research Process: Accessing and Recording Information and Annotated Bibliography: Gathering, Evaluating, and Documenting Sources .
• Field research. The researcher conducts field research. Field research allows researchers not only to ask questions of experts but also to observe and experience directly. It allows researchers to generate original data. No matter how much other people tell you, your knowledge increases through personal observations. In some subject areas, field research is as important as library or database research. This information is covered more extensively in Research Process: Accessing and Recording Information .
• Examination of texts. The researcher examines texts. Consulting a broad range of texts—such as magazines, brochures, newspapers, archives, blogs, videos, documentaries, or peer-reviewed journals—is crucial in academic research.
• Evaluation of sources. The researcher evaluates sources. As your research progresses, you will double-check information to find out whether it is confirmed by more than one source. In informal research, researchers evaluate sources to ensure that the final decision is satisfactory. Similarly, in academic research, researchers evaluate sources to ensure that the final product is accurate and convincing. Previewed here, this information is covered more extensively in Research Process: Accessing and Recording Information .
• Writing. The researcher writes. The writing during the research process can take a range of forms: from notes during library, database, or field work; to journal reflections on the research process; to drafts of the final product. In practical research, writing helps researchers find, remember, and explore information. In academic research, writing is even more important because the results must be reported accurately and thoroughly.
• Testing and Experimentation. The researcher tests and experiments. Because opinions vary on debatable topics and because few research topics have correct or incorrect answers, it is important to test and conduct experiments on possible hypotheses or solutions.
• Synthesis. The researcher synthesizes. By combining information from various sources, researchers support claims or arrive at new conclusions. When synthesizing, researchers connect evidence and ideas, both original and borrowed. Accumulating, sorting, and synthesizing information enables researchers to consider what evidence to use in support of a thesis and in what ways.
• Presentation. The researcher presents findings in an interesting, focused, and well-documented product.

Types of Research Evidence

Research evidence usually consists of data, which comes from borrowed information that you use to develop your thesis and support your organizational structure and reasoning. This evidence can take a range of forms, depending on the type of research conducted, the audience, and the genre for reporting the research.

Primary Research Sources

Although precise definitions vary somewhat by discipline, primary data sources are generally defined as firsthand accounts, such as texts or other materials produced by someone drawing from direct experience or observation. Primary source documents include, but are not limited to, personal narratives and diaries; eyewitness accounts; interviews; original documents such as treaties, official certificates, and government documents detailing laws or acts; speeches; newspaper coverage of events at the time they occurred; observations; and experiments. Primary source data is, in other words, original and in some way conducted or collected primarily by the researcher. The Research Process: Where to Look for Existing Sources and Compiling Sources for an Annotated Bibliography contain more information on both primary and secondary sources.

Secondary Research Sources

Secondary sources , on the other hand, are considered at least one step removed from the experience. That is, they rely on sources other than direct observation or firsthand experience. Secondary sources include, but are not limited to, most books, articles online or in databases, and textbooks (which are sometimes classified as tertiary sources because, like encyclopedias and other reference works, their primary purpose might be to summarize or otherwise condense information). Secondary sources regularly cite and build upon primary sources to provide perspective and analysis. Effective use of researched evidence usually includes both primary and secondary sources. Works of history, for example, draw on a large range of primary and secondary sources, citing, analyzing, and synthesizing information to present as many perspectives of a past event in as rich and nuanced a way as possible.

It is important to note that the distinction between primary and secondary sources depends in part on their use: that is, the same document can be both a primary source and a secondary source. For example, if Scholar X wrote a biography about Artist Y, the biography would be a secondary source about the artist and, at the same time, a primary source about the scholar.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/writing-guide/pages/1-unit-introduction

• Authors: Michelle Bachelor Robinson, Maria Jerskey, featuring Toby Fulwiler
• Publisher/website: OpenStax
• Book title: Writing Guide with Handbook
• Publication date: Dec 21, 2021
• Location: Houston, Texas
• Book URL: https://openstax.org/books/writing-guide/pages/1-unit-introduction
• Section URL: https://openstax.org/books/writing-guide/pages/12-1-introducing-research-and-research-evidence

© Dec 19, 2023 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

The “PP-ICONS” approach will help you separate the clinical wheat from the chaff in mere minutes .

ROBERT J. FLAHERTY, MD

Fam Pract Manag. 2004;11(5):47-52

Keeping up with the latest advances in diagnosis and treatment is a challenge we all face as phycians. We need information that is both valid (that is, accurate and correct) and relevant to our patients and practices. While we have many sources of clinical information, such as CME lectures, textbooks, pharmaceutical advertising, pharmaceutical representatives and colleagues, we often turn to journal articles for the most current clinical information.

Unfortunately, a great deal of research reported in journal articles is poorly done, poorly analyzed or both, and thus is not valid. A great deal of research is also irrelevant to our patients and practices. Separating the clinical wheat from the chaff can take skills that many of us never were taught.

Reading the abstract is often sufficient when evaluating an article using the PP-ICONS approach.

The most relevant studies will involve outcomes that matter to patients (e.g., morbidity, mortality and cost) versus outcomes that matter to physiologists (e.g., blood pressure, blood sugar or cholesterol levels).

Ignore the relative risk reduction, as it overstates research findings and will mislead you.

The article “Making Evidence-Based Medicine Doable in Everyday Practice” in the February 2004 issue of FPM describes several organizations that can help us. These organizations, such as the Cochrane Library, Bandolier and Clinical Evidence, develop clinical questions and then review one or more journal articles to identify the best available evidence that answers the question, with a focus on the quality of the study, the validity of the results and the relevance of the findings to everyday practice. These organizations provide a very valuable service, and the number of important clinical questions that they have studied has grown steadily over the past five years. (See “Four steps to an evidence-based answer.” )

FOUR STEPS TO AN EVIDENCE-BASED ANSWER

When faced with a clinical question, follow these steps to find an evidence-based answer:

Search the Web site of one of the evidence review organizations, such as Cochrane (http://www.cochrane.org/cochrane/revabstr/mainindex.htm), Bandolier ( http://www.jr2.ox.ac.uk/bandolier ) or Clinical Evidence ( http://www.clinicalevidence.com ), described in “Making Evidence-Based Medicine Doable in Everyday Practice,” FPM, February 2004, page 51 . You can also search the TRIP+ Web site ( http://www.tripdatabase.com ), which simultaneously searches the databases of many of the review organizations. If you find a systematic review or meta-analysis by one of these organizations, you can be confident that you’ve found the best evidence available.

If you don’t find the information you need through step 1, search for meta-analyses and systematic reviews using the PubMed Web site (see the tutorial at http://www.nlm.nih.gov/bsd/pubmed_tutorial/m1001.html ). Most of the recent abstracts found on PubMed provide enough information for you to determine the validity and relevance of the findings. If needed, you can get a copy of the full article through your hospital library or the journal’s Web site.

If you cannot find a systematic review or meta-analysis on PubMed, look for a randomized controlled trial (RCT). The RCT is the “gold standard” in medical research. Case reports, cohort studies and other research methods simply are not good enough to use for making patient care decisions.

Once you find the article you need, use the PP-ICONS approach to evaluate its usefulness to your patient.

If you find a systematic review or meta-analysis done by one of these organizations, you can feel confident that you have found the current best evidence. However, these organizations have not asked all of the common clinical questions yet, and you will frequently be faced with finding the pertinent articles and determining for yourself whether they are valuable. This is where the PP-ICONS approach can help.

What is PP-ICONS?

When you find a systematic review, meta-analysis or randomized controlled trial while reading your clinical journals or searching PubMed ( http://www.ncbi.nlm.nih.gov/entrez/query.fcgi ), you need to determine whether it is valid and relevant. There are many different ways to analyze an abstract or journal article, some more rigorous than others. 1 , 2 I have found a simple but effective way to identify a valid or relevant article within a couple of minutes, ensuring that I can use or discard the conclusions with confidence. This approach works well on articles regarding treatment and prevention, and can also be used with articles on diagnosis and screening.

The most important information to look for when reviewing an article can be summarized by the acronym “PP-ICONS,” which stands for the following:

Patient or population,

Intervention,

Comparison,

Number of subjects,

Statistics.

For example, imagine that you just saw a nine-year-old patient in the office with common warts on her hands, an ideal candidate for your usual cryotherapy. Her mother had heard about treating warts with duct tape and wondered if you would recommend this treatment. You promised to call Mom back after you had a chance to investigate this rather odd treatment.

When you get a free moment, you write down your clinical question: “Is duct tape an effective treatment for warts in children?” Writing down your clinical question is useful, as it can help you clarify exactly what you are looking for. Use the PPICO parts of the acronym to help you write your clinical question; this is actually how many researchers develop their research questions.

You search Cochrane and Bandolier without success, so now you search PubMed, which returns an abstract for the following article: “Focht DR 3rd, Spicer C, Fairchok MP. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med . 2002 Oct;156(10):971-974.”

You decide to apply PP-ICONS to this abstract (see "Abstract from PubMed" ) to determine if the information is both valid and relevant.

ABSTRACT FROM PUBMED

Using the PP-ICONS approach, physicians can evaluate the validity and relevance of clinical articles in minutes using only the abstract, such as this one, obtained free online from PubMed, http://www.ncbi. nlm.nih.gov/entrez/query.fcgi. The author uses this abstract to evaluate the use of duct tape to treat common warts.

Problem. The first P in PP-ICONS is for “problem,” which refers to the clinical condition that was studied. From the abstract, it is clear that the researchers studied the same problem you are interested in, which is important since flat warts or genital warts may have responded differently. Obviously, if the problem studied were not sufficiently similar to your clinical problem, the results would not be relevant.

Patient or population. Next, consider the patient or population. Is the study group similar to your patient or practice? Are they primary care patients, for example, or are they patients who have been referred to a tertiary care center? Are they of a similar age and gender? In this case, the researchers studied children and young adults in outpatient clinics, which is similar to your patient population. If the patients in the study are not similar to your patient, for example if they are sicker, older, a different gender or more clinically complicated, the results might not be relevant.

Intervention. The intervention could be a diagnostic test or a treatment. Make sure the intervention is the same as what you are looking for. The patient’s mother was asking about duct tape for warts, so this is a relevant study.

Comparison. The comparison is what the intervention is tested against. It could be a different diagnostic test or another therapy, such as cryotherapy in this wart study. It could even be placebo or no treatment. Make sure the comparison fits your question. You usually use cryotherapy for common warts, so this is a relevant comparison.

Outcome. The outcome is particularly important. Many outcomes are “disease-oriented outcomes,” which are based on “disease-oriented evidence” (DOEs). DOEs usually reflect changes in physiologic parameters, such as blood pressure, blood sugar, cholesterol, etc. We have long assumed that improving the physiologic parameters of a disease will result in a better disease outcome, but that is not necessarily true. For instance, finasteride can improve urinary flow rate in prostatic hypertrophy, but it does not significantly change symptom scores. 3

DOEs look at the kinds of outcomes that physiologists care about. More relevant are outcomes that patients care about, often called “patient-oriented outcomes.” These are based on “patient-oriented evidence that matters” (POEMs) and look at outcomes such as morbidity, mortality and cost. Thus, when looking at a journal article, DOEs are interesting but of questionable relevance, whereas POEMs are very interesting and very relevant. In the study on the previous page, the outcome is complete resolution of the wart, which is something your patient is interested in.

Number. The number of subjects is crucial to whether accurate statistics can be generated from the data. Too few patients in a research study may not be enough to show that a difference actually exists between the intervention and comparison groups (known as the “power” of a study). Many studies are published with less than 100 subjects, which is usually inadequate to provide reliable statistics. A good rule of thumb is 400 subjects. 4 Fifty-one patients completed the wart study, which is a pretty small number to generate good statistics.

Statistics. The statistics you are interested in are few in number and easy to understand. Since statistics are frequently misused in journal articles, it is worth a few minutes to learn which to believe and which to ignore.

Relative risk reduction. It is not unusual to find a summary statement in a journal article similar to this one from an article titled “Long-Term Effects of Mammography Screening: Updated Overview of the Swedish Randomised Trials”: 5

“There were 511 breast cancer deaths in 1,864,770 women-years in the invited groups and 584 breast cancer deaths in 1,688,440 women-years in the control groups, a significant 21 percent reduction in breast cancer mortality.”

This 21-percent statistic is the relative risk reduction (RRR), which is the percent reduction in the measured outcome between the experimental and control groups. (See “Some important statistics” for more information on calculating the RRR and other statistics.) The RRR is not a good way to compare outcomes. It amplifies small differences and makes insignificant findings appear significant, and it doesn’t reflect the baseline risk of the outcome event. Nevertheless, the RRR is very popular and will be reported in nearly every journal article, perhaps because it makes weak results look good. Think of the RRR as the “reputation reviving ratio” or the “reporter’s reason for ‘riting.” Ignore the RRR. It will mislead you. In our wart treatment example, the RRR would be (85 percent - 60 percent)/60 percent x 100 = 42 percent. The RRR could thus be interpreted as showing that duct tape is 42 percent more effective than cryotherapy in treating warts.

SOME IMPORTANT STATISTICS

Absolute risk reduction (ARR): The difference between the control group’s event rate (CER) and the experimental group’s event rate (EER).

Control event rate (CER): The proportion of patients responding to placebo or other control treatment. For example, if 25 patients are in a control group and the event being studied is observed in 15 of those patients, the control event rate would be 15/25 = 0.60.

Experimental event rate (EER): The proportion of patients responding to the experimental treatment or intervention. For example, if 26 patients are in an experimental group and the event being studied is observed in 22 of those patients, the experimental event rate would be 22/26 = 0.85.

Number needed to treat (NNT): The number of patients that must be treated to prevent one adverse outcome or for one patient to benefit. The NNT is the inverse of the ARR; NNT = 1/ARR.

Relative risk reduction (RRR): The percent reduction in events in the treated group compared to the control group event rate.

Absolute risk reduction. A better statistic is the absolute risk reduction (ARR), which is the difference in the outcome event rate between the control group and the experimental treated group. Thus, in our wart treatment example, the ARR is the outcome event rate (complete resolution of warts) for duct tape (85 percent) minus the outcome event rate for cryotherapy (60 percent) = 25 percent. Unlike the RRR, the ARR does not amplify small differences but shows the true difference between the experimental and control interventions. Using the ARR, it would be accurate to say that duct tape is 25-percent more effective than cryotherapy in treating warts.

Number needed to treat. The single most clinically useful statistic is the number needed to treat (NNT). The NNT is the number of patients who must be treated to prevent one adverse outcome. To think about it another way, the NNT is the number of patients who must be treated for one patient to benefit. (The rest who were treated obtained no benefit, although they still suffered the risks and costs of treatment.) In our wart therapy article, the NNT would tell us how many patients must be treated with the experimental treatment for one to benefit more than if he or she had been treated with the standard treatment.

Now this is a statistic that physicians and their patients can really appreciate! Furthermore, the NNT is easy to calculate, as it is simply the inverse of the ARR. For our wart treatment study, the NNT is 1/25 percent =1/0.25 = 4, meaning that 4 patients need to be treated with duct tape for one to benefit more than if treated by cryotherapy.

Wrapped up in this simple little statistic are some very important concepts. The NNT provides you with the likelihood that the test or treatment will benefit any individual patient, an impression of the baseline risk of the adverse event, and a sense of the cost to society. Thus, it gives perspective and hints at the “reasonableness” of a treatment. The value of this statistic has become appreciated in the last five years, and more journal articles are reporting it.

What is a reasonable NNT? In a perfect world, a treatment would have an NNT of 1, meaning that every patient would benefit from the treatment. Real life is not so kind (see “Examples of NNTs” ). Clearly, an NNT of 1 is great and an NNT of 1,000 is terrible. Although it is hard to come up with firm guidelines, for primary therapies I am satisfied with an NNT of 10 or less and very pleased with an NNT less than 5. Our duct tape NNT of 4 is good, particularly since the treatment is cheap, easy and painless.

EXAMPLES OF NNTS

The number needed to treat (NNT) is one of the most useful statistics for physicians and patients. It calculates the number of patients that must be treated to prevent one adverse event or for one patient to benefit. Note that NNTs for preventive interventions will usually be higher than NNTs for treatment interventions. The lower the NNT, the better.

The following examples of NNTs are borrowed from an excellent list available through the Bandolier Web site at http://www.jr2.ox.ac.uk/bandolier/band50/b50-8.html .

Note that NNTs for preventive interventions (e.g., the use of aspirin to prevent cardiac problems) will usually be higher than NNTs for treatment interventions (e.g., the use of duct tape to cure warts). Prevention groups contain both higher-risk and lower-risk individuals, so they produce bigger denominators, whereas treatment groups only contain diseased patients. Thus, an NNT for prevention of less than 20 might be particularly good.

When discussing a particular therapy, I explain the NNT to my patient. Since this statistical concept is easy to understand, it can help the patient be a more informed partner in therapeutic decisions.

You will soon start to see a similar statistic, the number needed to screen (NNS), which is the number of patients needed to screen for a particular disease for a given duration for one patient to benefit. 6 Although few NNSs have been calculated, they are likely to involve higher numbers, since the screening population consists of patients with and without the disease. For example, in the article on mammography screening mentioned above, the NNS was 961 for 16 years. In other words, you would need to screen 961 women for 16 years to prevent one breast cancer death.

The good news and the bad

Using PP-ICONS to assess the wart study, the problem, the patient/population, the intervention, the comparison and the outcome are all relevant to your patient. The number of subjects is on the small side, making you a little wary, but the intervention is cheap and low-risk. The statistics, particularly the NNT, are reasonable. On balance, this looks like a fair approach, so you call the patient’s mother and discuss it with her.

The PP-ICONS approach is an easy way to screen an article for validity and relevance, and the abstract often contains all of the information you need. Even the statistics can be done quickly in your head. You can apply PP-ICONS when searching for a particular article, when you come across an article in your reading, when data are presented at lectures, when a pharmaceutical representative hands you an article to support his or her pitch, and even when reading news stories describing medical breakthroughs.

Don’t be discouraged if you find that high-quality articles are rare, even in the most prestigious journals. This seems to be changing for the better, although many careers are still being built on questionable research. Nevertheless, screening articles will help you find the truth that is out there and will help you practice the best medicine. And as we become more discerning end-users of research, we might just stimulate improvements in clinical research in the process.

Miser WF. Critical appraisal of the literature. J Amer Board Fam Pract . 1999;12(4):315-333.

Guyatt GH, et al. Users’ guides to the medical literature. How to use an article about therapy or prevention. Are the results of the study valid?. JAMA . 1993;270(21):2598-2601.

Lepor H, et al. The efficacy of terazosin, finasteride or both in benign prostatic hyperplasia. Veterans Affairs Cooperative Studies Benign Prostatic Hyperplasia Study Group. N Engl J Med . 1996;335(8):533-539.

Krejcie RV, Morgan DW. Determining sample size for research activities. Educational and Psychological Measurement . 1970;30:607-610.

Nystrom L, et al. Long-term effects of mammography screening: updated overview of the Swedish randomised trials. Lancet . 2002;359(9310):909-919.

Rembold CM. Number needed to screen: development of a statistic for disease screening. BMJ . 1998;317:307-312.

Continue Reading

More in fpm, more in pubmed.

Copyright © 2004 by the American Academy of Family Physicians.

This content is owned by the AAFP. A person viewing it online may make one printout of the material and may use that printout only for his or her personal, non-commercial reference. This material may not otherwise be downloaded, copied, printed, stored, transmitted or reproduced in any medium, whether now known or later invented, except as authorized in writing by the AAFP.  See permissions  for copyright questions and/or permission requests.

Copyright © 2024 American Academy of Family Physicians. All Rights Reserved.

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

Definition and Introduction

Journal article analysis assignments require you to summarize and critically assess the quality of an empirical research study published in a scholarly [a.k.a., academic, peer-reviewed] journal. The article may be assigned by the professor, chosen from course readings listed in the syllabus, or you must locate an article on your own, usually with the requirement that you search using a reputable library database, such as, JSTOR or ProQuest . The article chosen is expected to relate to the overall discipline of the course, specific course content, or key concepts discussed in class. In some cases, the purpose of the assignment is to analyze an article that is part of the literature review for a future research project.

Analysis of an article can be assigned to students individually or as part of a small group project. The final product is usually in the form of a short paper [typically 1- 6 double-spaced pages] that addresses key questions the professor uses to guide your analysis or that assesses specific parts of a scholarly research study [e.g., the research problem, methodology, discussion, conclusions or findings]. The analysis paper may be shared on a digital course management platform and/or presented to the class for the purpose of promoting a wider discussion about the topic of the study. Although assigned in any level of undergraduate and graduate coursework in the social and behavioral sciences, professors frequently include this assignment in upper division courses to help students learn how to effectively identify, read, and analyze empirical research within their major.

Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students make the most of Scholarly Articles." Library Management 33 (2012): 525-535.

Benefits of Journal Article Analysis Assignments

Analyzing and synthesizing a scholarly journal article is intended to help students obtain the reading and critical thinking skills needed to develop and write their own research papers. This assignment also supports workplace skills where you could be asked to summarize a report or other type of document and report it, for example, during a staff meeting or for a presentation.

There are two broadly defined ways that analyzing a scholarly journal article supports student learning:

Improve Reading Skills

Conducting research requires an ability to review, evaluate, and synthesize prior research studies. Reading prior research requires an understanding of the academic writing style , the type of epistemological beliefs or practices underpinning the research design, and the specific vocabulary and technical terminology [i.e., jargon] used within a discipline. Reading scholarly articles is important because academic writing is unfamiliar to most students; they have had limited exposure to using peer-reviewed journal articles prior to entering college or students have yet to gain exposure to the specific academic writing style of their disciplinary major. Learning how to read scholarly articles also requires careful and deliberate concentration on how authors use specific language and phrasing to convey their research, the problem it addresses, its relationship to prior research, its significance, its limitations, and how authors connect methods of data gathering to the results so as to develop recommended solutions derived from the overall research process.

Improve Comprehension Skills

In addition to knowing how to read scholarly journals articles, students must learn how to effectively interpret what the scholar(s) are trying to convey. Academic writing can be dense, multi-layered, and non-linear in how information is presented. In addition, scholarly articles contain footnotes or endnotes, references to sources, multiple appendices, and, in some cases, non-textual elements [e.g., graphs, charts] that can break-up the reader’s experience with the narrative flow of the study. Analyzing articles helps students practice comprehending these elements of writing, critiquing the arguments being made, reflecting upon the significance of the research, and how it relates to building new knowledge and understanding or applying new approaches to practice. Comprehending scholarly writing also involves thinking critically about where you fit within the overall dialogue among scholars concerning the research problem, finding possible gaps in the research that require further analysis, or identifying where the author(s) has failed to examine fully any specific elements of the study.

In addition, journal article analysis assignments are used by professors to strengthen discipline-specific information literacy skills, either alone or in relation to other tasks, such as, giving a class presentation or participating in a group project. These benefits can include the ability to:

• Effectively paraphrase text, which leads to a more thorough understanding of the overall study;
• Identify and describe strengths and weaknesses of the study and their implications;
• Relate the article to other course readings and in relation to particular research concepts or ideas discussed during class;
• Think critically about the research and summarize complex ideas contained within;
• Plan, organize, and write an effective inquiry-based paper that investigates a research study, evaluates evidence, expounds on the author’s main ideas, and presents an argument concerning the significance and impact of the research in a clear and concise manner;
• Model the type of source summary and critique you should do for any college-level research paper; and,
• Increase interest and engagement with the research problem of the study as well as with the discipline.

Kershaw, Trina C., Jennifer Fugate, and Aminda J. O'Hare. "Teaching Undergraduates to Understand Published Research through Structured Practice in Identifying Key Research Concepts." Scholarship of Teaching and Learning in Psychology . Advance online publication, 2020; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students make the most of Scholarly Articles." Library Management 33 (2012): 525-535; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946.

Structure and Organization

A journal article analysis paper should be written in paragraph format and include an instruction to the study, your analysis of the research, and a conclusion that provides an overall assessment of the author's work, along with an explanation of what you believe is the study's overall impact and significance. Unless the purpose of the assignment is to examine foundational studies published many years ago, you should select articles that have been published relatively recently [e.g., within the past few years].

Since the research has been completed, reference to the study in your paper should be written in the past tense, with your analysis stated in the present tense [e.g., “The author portrayed access to health care services in rural areas as primarily a problem of having reliable transportation. However, I believe the author is overgeneralizing this issue because...”].

Introduction Section

The first section of a journal analysis paper should describe the topic of the article and highlight the author’s main points. This includes describing the research problem and theoretical framework, the rationale for the research, the methods of data gathering and analysis, the key findings, and the author’s final conclusions and recommendations. The narrative should focus on the act of describing rather than analyzing. Think of the introduction as a more comprehensive and detailed descriptive abstract of the study.

Possible questions to help guide your writing of the introduction section may include:

• Who are the authors and what credentials do they hold that contributes to the validity of the study?
• What was the research problem being investigated?
• What type of research design was used to investigate the research problem?
• What theoretical idea(s) and/or research questions were used to address the problem?
• What was the source of the data or information used as evidence for analysis?
• What methods were applied to investigate this evidence?
• What were the author's overall conclusions and key findings?

Critical Analysis Section

The second section of a journal analysis paper should describe the strengths and weaknesses of the study and analyze its significance and impact. This section is where you shift the narrative from describing to analyzing. Think critically about the research in relation to other course readings, what has been discussed in class, or based on your own life experiences. If you are struggling to identify any weaknesses, explain why you believe this to be true. However, no study is perfect, regardless of how laudable its design may be. Given this, think about the repercussions of the choices made by the author(s) and how you might have conducted the study differently. Examples can include contemplating the choice of what sources were included or excluded in support of examining the research problem, the choice of the method used to analyze the data, or the choice to highlight specific recommended courses of action and/or implications for practice over others. Another strategy is to place yourself within the research study itself by thinking reflectively about what may be missing if you had been a participant in the study or if the recommended courses of action specifically targeted you or your community.

Possible questions to help guide your writing of the analysis section may include:

Introduction

• Did the author clearly state the problem being investigated?
• What was your reaction to and perspective on the research problem?
• Was the study’s objective clearly stated? Did the author clearly explain why the study was necessary?
• How well did the introduction frame the scope of the study?
• Did the introduction conclude with a clear purpose statement?

Literature Review

• Did the literature review lay a foundation for understanding the significance of the research problem?
• Did the literature review provide enough background information to understand the problem in relation to relevant contexts [e.g., historical, economic, social, cultural, etc.].
• Did literature review effectively place the study within the domain of prior research? Is anything missing?
• Was the literature review organized by conceptual categories or did the author simply list and describe sources?
• Did the author accurately explain how the data or information were collected?
• Was the data used sufficient in supporting the study of the research problem?
• Was there another methodological approach that could have been more illuminating?
• Give your overall evaluation of the methods used in this article. How much trust would you put in generating relevant findings?

Results and Discussion

• Were the results clearly presented?
• Did you feel that the results support the theoretical and interpretive claims of the author? Why?
• What did the author(s) do especially well in describing or analyzing their results?
• Was the author's evaluation of the findings clearly stated?
• How well did the discussion of the results relate to what is already known about the research problem?
• Was the discussion of the results free of repetition and redundancies?
• What interpretations did the authors make that you think are in incomplete, unwarranted, or overstated?
• Did the conclusion effectively capture the main points of study?
• Did the conclusion address the research questions posed? Do they seem reasonable?
• Were the author’s conclusions consistent with the evidence and arguments presented?
• Has the author explained how the research added new knowledge or understanding?

Overall Writing Style

• If the article included tables, figures, or other non-textual elements, did they contribute to understanding the study?
• Were ideas developed and related in a logical sequence?
• Were transitions between sections of the article smooth and easy to follow?

Overall Evaluation Section

The final section of a journal analysis paper should bring your thoughts together into a coherent assessment of the value of the research study . This section is where the narrative flow transitions from analyzing specific elements of the article to critically evaluating the overall study. Explain what you view as the significance of the research in relation to the overall course content and any relevant discussions that occurred during class. Think about how the article contributes to understanding the overall research problem, how it fits within existing literature on the topic, how it relates to the course, and what it means to you as a student researcher. In some cases, your professor will also ask you to describe your experiences writing the journal article analysis paper as part of a reflective learning exercise.

Possible questions to help guide your writing of the conclusion and evaluation section may include:

• Was the structure of the article clear and well organized?
• Was the topic of current or enduring interest to you?
• What were the main weaknesses of the article? [this does not refer to limitations stated by the author, but what you believe are potential flaws]
• Was any of the information in the article unclear or ambiguous?
• What did you learn from the research? If nothing stood out to you, explain why.
• Assess the originality of the research. Did you believe it contributed new understanding of the research problem?
• Were you persuaded by the author’s arguments?
• If the author made any final recommendations, will they be impactful if applied to practice?
• In what ways could future research build off of this study?
• What implications does the study have for daily life?
• Was the use of non-textual elements, footnotes or endnotes, and/or appendices helpful in understanding the research?
• What lingering questions do you have after analyzing the article?

NOTE: Avoid using quotes. One of the main purposes of writing an article analysis paper is to learn how to effectively paraphrase and use your own words to summarize a scholarly research study and to explain what the research means to you. Using and citing a direct quote from the article should only be done to help emphasize a key point or to underscore an important concept or idea.

Business: The Article Analysis . Fred Meijer Center for Writing, Grand Valley State University; Bachiochi, Peter et al. "Using Empirical Article Analysis to Assess Research Methods Courses." Teaching of Psychology 38 (2011): 5-9; Brosowsky, Nicholaus P. et al. “Teaching Undergraduate Students to Read Empirical Articles: An Evaluation and Revision of the QALMRI Method.” PsyArXi Preprints , 2020; Holster, Kristin. “Article Evaluation Assignment”. TRAILS: Teaching Resources and Innovations Library for Sociology . Washington DC: American Sociological Association, 2016; Kershaw, Trina C., Jennifer Fugate, and Aminda J. O'Hare. "Teaching Undergraduates to Understand Published Research through Structured Practice in Identifying Key Research Concepts." Scholarship of Teaching and Learning in Psychology . Advance online publication, 2020; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Reviewer's Guide . SAGE Reviewer Gateway, SAGE Journals; Sego, Sandra A. and Anne E. Stuart. "Learning to Read Empirical Articles in General Psychology." Teaching of Psychology 43 (2016): 38-42; Kershaw, Trina C., Jordan P. Lippman, and Jennifer Fugate. "Practice Makes Proficient: Teaching Undergraduate Students to Understand Published Research." Instructional Science 46 (2018): 921-946; Gyuris, Emma, and Laura Castell. "To Tell Them or Show Them? How to Improve Science Students’ Skills of Critical Reading." International Journal of Innovation in Science and Mathematics Education 21 (2013): 70-80; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36; MacMillan, Margy and Allison MacKenzie. "Strategies for Integrating Information Literacy and Academic Literacy: Helping Undergraduate Students Make the Most of Scholarly Articles." Library Management 33 (2012): 525-535.

Writing Tip

Not All Scholarly Journal Articles Can Be Critically Analyzed

There are a variety of articles published in scholarly journals that do not fit within the guidelines of an article analysis assignment. This is because the work cannot be empirically examined or it does not generate new knowledge in a way which can be critically analyzed.

If you are required to locate a research study on your own, avoid selecting these types of journal articles:

• Theoretical essays which discuss concepts, assumptions, and propositions, but report no empirical research;
• Statistical or methodological papers that may analyze data, but the bulk of the work is devoted to refining a new measurement, statistical technique, or modeling procedure;
• Articles that review, analyze, critique, and synthesize prior research, but do not report any original research;
• Brief essays devoted to research methods and findings;
• Articles written by scholars in popular magazines or industry trade journals;
• Pre-print articles that have been posted online, but may undergo further editing and revision by the journal's editorial staff before final publication; and
• Academic commentary that discusses research trends or emerging concepts and ideas, but does not contain citations to sources.

Journal Analysis Assignment - Myers . Writing@CSU, Colorado State University; Franco, Josue. “Introducing the Analysis of Journal Articles.” Prepared for presentation at the American Political Science Association’s 2020 Teaching and Learning Conference, February 7-9, 2020, Albuquerque, New Mexico; Woodward-Kron, Robyn. "Critical Analysis and the Journal Article Review Assignment." Prospect 18 (August 2003): 20-36.

• << Previous: Annotated Bibliography
• Next: Giving an Oral Presentation >>
• Last Updated: Mar 6, 2024 1:00 PM
• URL: https://libguides.usc.edu/writingguide/assignments

• Open access
• Published: 24 April 2024

Breast cancer screening motivation and behaviours of women aged over 75 years: a scoping review

• Virginia Dickson-Swift 1 ,
• Joanne Adams 1 ,
• Evelien Spelten 1 ,
• Irene Blackberry 2 ,
• Carlene Wilson 3 , 4 , 5 &
• Eva Yuen 3 , 6 , 7 , 8  

BMC Women's Health volume  24 , Article number:  256 ( 2024 ) Cite this article

100 Accesses

Metrics details

This scoping review aimed to identify and present the evidence describing key motivations for breast cancer screening among women aged ≥ 75 years. Few of the internationally available guidelines recommend continued biennial screening for this age group. Some suggest ongoing screening is unnecessary or should be determined on individual health status and life expectancy. Recent research has shown that despite recommendations regarding screening, older women continue to hold positive attitudes to breast screening and participate when the opportunity is available.

All original research articles that address motivation, intention and/or participation in screening for breast cancer among women aged ≥ 75 years were considered for inclusion. These included articles reporting on women who use public and private breast cancer screening services and those who do not use screening services (i.e., non-screeners).

The Joanna Briggs Institute (JBI) methodology for scoping reviews was used to guide this review. A comprehensive search strategy was developed with the assistance of a specialist librarian to access selected databases including: the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Medline, Web of Science and PsychInfo. The review was restricted to original research studies published since 2009, available in English and focusing on high-income countries (as defined by the World Bank). Title and abstract screening, followed by an assessment of full-text studies against the inclusion criteria was completed by at least two reviewers. Data relating to key motivations, screening intention and behaviour were extracted, and a thematic analysis of study findings undertaken.

A total of fourteen (14) studies were included in the review. Thematic analysis resulted in identification of three themes from included studies highlighting that decisions about screening were influenced by: knowledge of the benefits and harms of screening and their relationship to age; underlying attitudes to the importance of cancer screening in women's lives; and use of decision aids to improve knowledge and guide decision-making.

The results of this review provide a comprehensive overview of current knowledge regarding the motivations and screening behaviour of older women about breast cancer screening which may inform policy development.

Peer Review reports

Introduction

Breast cancer is now the most commonly diagnosed cancer in the world overtaking lung cancer in 2021 [ 1 ]. Across the globe, breast cancer contributed to 25.8% of the total number of new cases of cancer diagnosed in 2020 [ 2 ] and accounts for a high disease burden for women [ 3 ]. Screening for breast cancer is an effective means of detecting early-stage cancer and has been shown to significantly improve survival rates [ 4 ]. A recent systematic review of international screening guidelines found that most countries recommend that women have biennial mammograms between the ages of 40–70 years [ 5 ] with some recommending that there should be no upper age limit [ 6 , 7 , 8 , 9 , 10 , 11 , 12 ] and others suggesting that benefits of continued screening for women over 75 are not clear [ 13 , 14 , 15 ].

Some guidelines suggest that the decision to end screening should be determined based on the individual health status of the woman, their life expectancy and current health issues [ 5 , 16 , 17 ]. This is because the benefits of mammography screening may be limited after 7 years due to existing comorbidities and limited life expectancy [ 18 , 19 , 20 , 21 ], with some jurisdictions recommending breast cancer screening for women ≥ 75 years only when life expectancy is estimated as at least 7–10 years [ 22 ]. Others have argued that decisions about continuing with screening mammography should depend on individual patient risk and health management preferences [ 23 ]. This decision is likely facilitated by a discussion between a health care provider and patient about the harms and benefits of screening outside the recommended ages [ 24 , 25 ]. While mammography may enable early detection of breast cancer, it is clear that false-positive results and overdiagnosis Footnote 1 may occur. Studies have estimated that up to 25% of breast cancer cases in the general population may be over diagnosed [ 26 , 27 , 28 ].

The risk of being diagnosed with breast cancer increases with age and approximately 80% of new cases of breast cancer in high-income countries are in women over the age of 50 [ 29 ]. The average age of first diagnosis of breast cancer in high income countries is comparable to that of Australian women which is now 61 years [ 2 , 4 , 29 ]. Studies show that women aged ≥ 75 years generally have positive attitudes to mammography screening and report high levels of perceived benefits including early detection of breast cancer and a desire to stay healthy as they age [ 21 , 30 , 31 , 32 ]. Some women aged over 74 participate, or plan to participate, in screening despite recommendations from health professionals and government guidelines advising against it [ 33 ]. Results of a recent review found that knowledge of the recommended guidelines and the potential harms of screening are limited and many older women believed that the benefits of continued screening outweighed the risks [ 30 ].

Very few studies have been undertaken to understand the motivations of women to screen or to establish screening participation rates among women aged ≥ 75 and older. This is surprising given that increasing age is recognised as a key risk factor for the development of breast cancer, and that screening is offered in many locations around the world every two years up until 74 years. The importance of this topic is high given the ambiguity around best practice for participation beyond 74 years. A preliminary search of Open Science Framework, PROSPERO, Cochrane Database of Systematic Reviews and JBI Evidence Synthesis in May 2022 did not locate any reviews on this topic.

This scoping review has allowed for the mapping of a broad range of research to explore the breadth and depth of the literature, summarize the evidence and identify knowledge gaps [ 34 , 35 ]. This information has supported the development of a comprehensive overview of current knowledge of motivations of women to screen and screening participation rates among women outside the targeted age of many international screening programs.

Materials and methods

Research question.

The research question for this scoping review was developed by applying the Population—Concept—Context (PCC) framework [ 36 ]. The current review addresses the research question “What research has been undertaken in high-income countries (context) exploring the key motivations to screen for breast cancer and screening participation (concepts) among women ≥ 75 years of age (population)?

Eligibility criteria

Participants.

Women aged ≥ 75 years were the key population. Specifically, motivations to screen and screening intention and behaviour and the variables that discriminate those who screen from those who do not (non-screeners) were utilised as the key predictors and outcomes respectively.

From a conceptual perspective it was considered that motivation led to behaviour, therefore articles that described motivation and corresponding behaviour were considered. These included articles reporting on women who use public (government funded) and private (fee for service) breast cancer screening services and those who do not use screening services (i.e., non-screeners).

The scope included high-income countries using the World Bank definition [ 37 ]. These countries have broadly similar health systems and opportunities for breast cancer screening in both public and private settings.

Types of sources

All studies reporting original research in peer-reviewed journals from January 2009 were eligible for inclusion, regardless of design. This date was selected due to an evaluation undertaken for BreastScreen Australia recommending expansion of the age group to include 70–74-year-old women [ 38 ]. This date was also indicative of international debate regarding breast cancer screening effectiveness at this time [ 39 , 40 ]. Reviews were also included, regardless of type—scoping, systematic, or narrative. Only sources published in English and available through the University’s extensive research holdings were eligible for inclusion. Ineligible materials were conference abstracts, letters to the editor, editorials, opinion pieces, commentaries, newspaper articles, dissertations and theses.

This scoping review was registered with the Open Science Framework database ( https://osf.io/fd3eh ) and followed Joanna Briggs Institute (JBI) methodology for scoping reviews [ 35 , 36 ]. Although ethics approval is not required for scoping reviews the broader study was approved by the University Ethics Committee (approval number HEC 21249).

Search strategy

A pilot search strategy was developed in consultation with an expert health librarian and tested in MEDLINE (OVID) and conducted on 3 June 2022. Articles from this pilot search were compared with seminal articles previously identified by the members of the team and used to refine the search terms. The search terms were then searched as both keywords and subject headings (e.g., MeSH) in the titles and abstracts and Boolean operators employed. A full MEDLINE search was then carried out by the librarian (see Table  1 ). This search strategy was adapted for use in each of the following databases: Cumulative Index to Nursing and Allied Health Literature (CINAHL), Medical Literature Analysis and Retrieval System Online (MEDLINE), Web of Science and PsychInfo databases. The references of included studies have been hand-searched to identify any additional evidence sources.

Study/source of evidence selection

Following the search, all identified citations were collated and uploaded into EndNote v.X20 (Clarivate Analytics, PA, USA) and duplicates removed. The resulting articles were then imported into Covidence – Cochrane’s systematic review management software [ 41 ]. Duplicates were removed once importation was complete, and title and abstract screening was undertaken against the eligibility criteria. A sample of 25 articles were assessed by all reviewers to ensure reliability in the application of the inclusion and exclusion criteria. Team discussion was used to ensure consistent application. The Covidence software supports blind reviewing with two reviewers required at each screening phase. Potentially relevant sources were retrieved in full text and were assessed against the inclusion criteria by two independent reviewers. Conflicts were flagged within the software which allows the team to discuss those that have disagreements until a consensus was reached. Reasons for exclusion of studies at full text were recorded and reported in the scoping review. The Preferred Reporting Items of Systematic Reviews extension for scoping reviews (PRISMA-ScR) checklist was used to guide the reporting of the review [ 42 ] and all stages were documented using the PRISMA-ScR flow chart [ 42 ].

Data extraction

A data extraction form was created in Covidence and used to extract study characteristics and to confirm the study’s relevance. This included specific details such as article author/s, title, year of publication, country, aim, population, setting, data collection methods and key findings relevant to the review question. The draft extraction form was modified as needed during the data extraction process.

Data analysis and presentation

Extracted data were summarised in tabular format (see Table  2 ). Consistent with the guidelines for the effective reporting of scoping reviews [ 43 ] and the JBI framework [ 35 ] the final stage of the review included thematic analysis of the key findings of the included studies. Study findings were imported into QSR NVivo with coding of each line of text. Descriptive codes reflected key aspects of the included studies related to the motivations and behaviours of women > 75 years about breast cancer screening.

In line with the reporting requirements for scoping reviews the search results for this review are presented in Fig.  1 [ 44 ].

PRISMA Flowchart. From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71

A total of fourteen [ 14 ] studies were included in the review with studies from the following countries, US n  = 12 [ 33 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 ], UK n  = 1 [ 23 ] and France n  = 1 [ 56 ]. Sample sizes varied, with most containing fewer than 50 women ( n  = 8) [ 33 , 45 , 46 , 48 , 51 , 52 , 55 ]. Two had larger samples including a French study with 136 women (a sub-set of a larger sample) [ 56 ], and one mixed method study in the UK with a sample of 26 women undertaking interviews and 479 women completing surveys [ 23 ]. One study did not report exact numbers [ 50 ]. Three studies [ 47 , 53 , 54 ] were undertaken by a group of researchers based in the US utilising the same sample of women, however each of the papers focused on different primary outcomes. The samples in the included studies were recruited from a range of locations including primary medical care clinics, specialist medical clinics, University affiliated medical clinics, community-based health centres and community outreach clinics [ 47 , 53 , 54 ].

Data collection methods varied and included: quantitative ( n  = 8), qualitative ( n  = 5) and mixed methods ( n  = 1). A range of data collection tools and research designs were utilised; pre/post, pilot and cross-sectional surveys, interviews, and secondary analysis of existing data sets. Seven studies focused on the use of a Decision Aids (DAs), either in original or modified form, developed by Schonberg et al. [ 55 ] as a tool to increase knowledge about the harms and benefits of screening for older women [ 45 , 47 , 48 , 49 , 52 , 54 , 55 ]. Three studies focused on intention to screen [ 33 , 53 , 56 ], two on knowledge of, and attitudes to, screening [ 23 , 46 ], one on information needs relating to risks and benefits of screening discontinuation [ 51 ], and one on perceptions about discontinuation of screening and impact of social interactions on screening [ 50 ].

The three themes developed from the analysis of the included studies highlighted that decisions about screening were primarily influenced by: (1) knowledge of the benefits and harms of screening and their relationship to age; (2) underlying attitudes to the importance of cancer screening in women's lives; and (3) exposure to decision aids designed to facilitate informed decision-making. Each of these themes will be presented below drawing on the key findings of the appropriate studies. The full dataset of extracted data can be found in Table  2 .

Knowledge of the benefits and harms of screening ≥ 75 years

The decision to participate in routine mammography is influenced by individual differences in cognition and affect, interpersonal relationships, provider characteristics, and healthcare system variables. Women typically perceive mammograms as a positive, beneficial and routine component of care [ 46 ] and an important aspect of taking care of themselves [ 23 , 46 , 49 ]. One qualitative study undertaken in the US showed that few women had discussed mammography cessation or the potential harms of screening with their health care providers and some women reported they would insist on receiving mammography even without a provider recommendation to continue screening [ 46 ].

Studies suggested that ageing itself, and even poor health, were not seen as reasonable reasons for screening cessation. For many women, guidance from a health care provider was deemed the most important influence on decision-making [ 46 ]. Preferences for communication about risk and benefits were varied with one study reporting women would like to learn more about harms and risks and recommended that this information be communicated via physicians or other healthcare providers, included in brochures/pamphlets, and presented outside of clinical settings (e.g., in community-based seniors groups) [ 51 ]. Others reported that women were sometimes sceptical of expert and government recommendations [ 33 ] although some were happy to participate in discussions with health educators or care providers about breast cancer screening harms and benefits and potential cessation [ 52 ].

Underlying attitudes to the importance of cancer screening at and beyond 75 years

Included studies varied in describing the importance of screening, with some attitudes based on past attendance and some based on future intentions to screen. Three studies reported findings indicating that some women intended to continue screening after 75 years of age [ 23 , 45 , 46 ], with one study in the UK reporting that women supported an extension of the automatic recall indefinitely, regardless of age or health status. In this study, failure to invite older women to screen was interpreted as age discrimination [ 23 ]. The desire to continue screening beyond 75 was also highlighted in a study from France that found that 60% of the women ( n  = 136 aged ≥ 75) intended to pursue screening in the future, and 27 women aged ≥ 75, who had never undergone mammography previously (36%), intended to do so in the future [ 56 ]. In this same study, intentions to screen varied significantly [ 56 ]. There were no sociodemographic differences observed between screened and unscreened women with regard to level of education, income, health risk behaviour (smoking, alcohol consumption), knowledge about the importance and the process of screening, or psychological features (fear of the test, fear of the results, fear of the disease, trust in screening impact) [ 56 ]. Further analysis showed that three items were statistically correlated with a higher rate of attendance at screening: (1) screening was initiated by a physician; (2) the women had a consultation with a gynaecologist during the past 12 months; and (3) the women had already undergone at least five screening mammograms. Analysis highlighted that although average income, level of education, psychological features or other types of health risk behaviours did not impact screening intention, having a mammogram previously impacted likelihood of ongoing screening. There was no information provided that explained why women who had not previously undergone screening might do so in the future.

A mixed methods study in the UK reported similar findings [ 23 ]. Utilising interviews ( n  = 26) and questionnaires ( n  = 479) with women ≥ 70 years (median age 75 years) the overwhelming result (90.1%) was that breast screening should be offered to all women indefinitely regardless of age, health status or fitness [ 23 ], and that many older women were keen to continue screening. Both the interview and survey data confirmed women were uncertain about eligibility for breast screening. The survey data showed that just over half the women (52.9%) were unaware that they could request mammography or knew how to access it. Key reasons for screening discontinuation were not being invited for screening (52.1%) and not knowing about self-referral (35.1%).

Women reported that not being invited to continue screening sent messages that screening was no longer important or required for this age group [ 23 ]. Almost two thirds of the women completing the survey (61.6%) said they would forget to attend screening without an invitation. Other reasons for screening discontinuation included transport difficulties (25%) and not wishing to burden family members (24.7%). By contrast, other studies have reported that women do not endorse discontinuation of screening mammography due to advancing age or poor health, but some may be receptive to reducing screening frequency on recommendation from their health care provider [ 46 , 51 ].

Use of Decision Aids (DAs) to improve knowledge and guide screening decision-making

Many women reported poor knowledge about the harms and benefits of screening with studies identifying an important role for DAs. These aids have been shown to be effective in improving knowledge of the harms and benefits of screening [ 45 , 54 , 55 ] including for women with low educational attainment; as compared to women with high educational attainment [ 47 ]. DAs can increase knowledge about screening [ 47 , 49 ] and may decrease the intention to continue screening after the recommended age [ 45 , 52 , 54 ]. They can be used by primary care providers to support a conversation about breast screening intention and reasons for discontinuing screening. In one pilot study undertaken in the US using a DA, 5 of the 8 women (62.5%) indicated they intended to continue to receive mammography; however, 3 participants planned to get them less often [ 45 ]. When asked whether they thought their physician would want them to get a mammogram, 80% said “yes” on pre-test; this figure decreased to 62.5% after exposure to the DA. This pilot study suggests that the use of a decision-aid may result in fewer women ≥ 75 years old continuing to screen for breast cancer [ 45 ].

Similar findings were evident in two studies drawing on the same data undertaken in the US [ 48 , 53 ]. Using a larger sample ( n  = 283), women’s intentions to screen prior to a visit with their primary care provider and then again after exposure to the DA were compared. Results showed that 21.7% of women reduced their intention to be screened, 7.9% increased their intentions to be screened, and 70.4% did not change. Compared to those who had no change or increased their screening intentions, women who had a decrease in screening intention were significantly less likely to receive screening after 18 months. Generally, studies have shown that women aged 75 and older find DAs acceptable and helpful [ 47 , 48 , 49 , 55 ] and using them had the potential to impact on a women’s intention to screen [ 55 ].

Cadet and colleagues [ 49 ] explored the impact of educational attainment on the use of DAs. Results highlight that education moderates the utility of these aids; women with lower educational attainment were less likely to understand all the DA’s content (46.3% vs 67.5%; P < 0.001); had less knowledge of the benefits and harms of mammography (adjusted mean ± standard error knowledge score, 7.1 ± 0.3 vs 8.1 ± 0.3; p < 0.001); and were less likely to have their screening intentions impacted (adjusted percentage, 11.4% vs 19.4%; p  = 0.01).

This scoping review summarises current knowledge regarding motivations and screening behaviours of women over 75 years. The findings suggest that awareness of the importance of breast cancer screening among women aged ≥ 75 years is high [ 23 , 46 , 49 ] and that many women wish to continue screening regardless of perceived health status or age. This highlights the importance of focusing on motivation and screening behaviours and the multiple factors that influence ongoing participation in breast screening programs.

The generally high regard attributed to screening among women aged ≥ 75 years presents a complex challenge for health professionals who are focused on potential harm (from available national and international guidelines) in ongoing screening for women beyond age 75 [ 18 , 20 , 57 ]. Included studies highlight that many women relied on the advice of health care providers regarding the benefits and harms when making the decision to continue breast screening [ 46 , 51 , 52 ], however there were some that did not [ 33 ]. Having a previous pattern of screening was noted as being more significant to ongoing intention than any other identified socio-demographic feature [ 56 ]. This is perhaps because women will not readily forgo health care practices that they have always considered important and that retain ongoing importance for the broader population.

For those women who had discontinued screening after the age of 74 it was apparent that the rationale for doing so was not often based on choice or receipt of information, but rather on factors that impact decision-making in relation to screening. These included no longer receiving an invitation to attend, transport difficulties and not wanting to be a burden on relatives or friends [ 23 , 46 , 51 ]. Ongoing receipt of invitations to screen was an important aspect of maintaining a capacity to choose [ 23 ]. This was particularly important for those women who had been regular screeners.

Women over 75 require more information to make decisions regarding screening [ 23 , 52 , 54 , 55 ], however health care providers must also be aware that the element of choice is important for older women. Having a capacity to choose avoids any notion of discrimination based on age, health status, gender or sociodemographic difference and acknowledges the importance of women retaining control over their health [ 23 ]. It was apparent that some women would choose to continue screening at a reduced frequency if this option was available and that women should have access to information facilitating self-referral [ 23 , 45 , 46 , 51 , 56 ].

Decision-making regarding ongoing breast cancer screening has been facilitated via the use of Decision Aids (DAs) within clinical settings [ 54 , 55 ]. While some studies suggest that women will make a decision regardless of health status, the use of DAs has impacted women’s decision to screen. While this may have limited benefit for those of lower educational attainment [ 48 ] they have been effective in improving knowledge relating to harms and benefits of screening particularly where they have been used to support a conversation with women about the value of screening [ 54 , 55 , 56 ].

Women have identified challenges in engaging in conversations with health care providers regarding ongoing screening, because providers frequently draw on projections of life expectancy and over-diagnosis [ 17 , 51 ]. As a result, these conversations about screening after age 75 years often do not occur [ 46 ]. It is likely that health providers may need more support and guidance in leading these conversations. This may be through the use of DAs or standardised checklists. It may be possible to incorporate these within existing health preventive measures for this age group. The potential for advice regarding ongoing breast cancer screening to be available outside of clinical settings may provide important pathways for conversations with women regarding health choices. Provision of information and advice in settings such as community based seniors groups [ 51 ] offers a potential platform to broaden conversations and align sources of information, not only with health professionals but amongst women themselves. This may help to address any misconception regarding eligibility and access to services [ 23 ]. It may also be aligned with other health promotion and lifestyle messages provided to this age group.

Limitations of the review

The searches that formed the basis of this review were carried in June 2022. Although the search was comprehensive, we have only captured those studies that were published in the included databases from 2009. There may have been other studies published outside of these periods. We also limited the search to studies published in English with full-text availability.

The emphasis of a scoping review is on comprehensive coverage and synthesis of the key findings, rather than on a particular standard of evidence and, consequently a quality assessment of the included studies was not undertaken. This has resulted in the inclusion of a wide range of study designs and data collection methods. It is important to note that three studies included in the review drew on the same sample of women (283 over > 75)[ 49 , 53 , 54 ]. The results of this review provide valuable insights into motivations and behaviours for breast cancer screening for older women, however they should be interpreted with caution given the specific methodological and geographical limitations.

Conclusion and recommendations

This scoping review highlighted a range of key motivations and behaviours in relation to breast cancer screening for women ≥ 75 years of age. The results provide some insight into how decisions about screening continuation after 74 are made and how informed decision-making can be supported. Specifically, this review supports the following suggestions for further research and policy direction:

Further research regarding breast cancer screening motivations and behaviours for women over 75 would provide valuable insight for health providers delivering services to women in this age group.

Health providers may benefit from the broader use of decision aids or structured checklists to guide conversations with women over 75 regarding ongoing health promotion/preventive measures.

Providing health-based information in non-clinical settings frequented by women in this age group may provide a broader reach of information and facilitate choices. This may help to reduce any perception of discrimination based on age, health status or socio-demographic factors.

Availability of data and materials

All data generated or analysed during this study is included in this published article (see Table  2 above).

Cancer Australia, in their 2014 position statement, define “overdiagnosis” in the following way. ‘’Overdiagnosis’ from breast screening does not refer to error or misdiagnosis, but rather refers to breast cancer diagnosed by screening that would not otherwise have been diagnosed during a woman’s lifetime. “Overdiagnosis” includes all instances where cancers detected through screening (ductal carcinoma in situ or invasive breast cancer) might never have progressed to become symptomatic during a woman’s life, i.e., cancer that would not have been detected in the absence of screening. It is not possible to precisely predict at diagnosis, to which cancers overdiagnosis would apply.” (accessed 22. nd August 2022; https://www.canceraustralia.gov.au/resources/position-statements/overdiagnosis-mammographic-screening ).

World Health Organization. Breast Cancer Geneva: WHO; 2021 [Available from: https://www.who.int/news-room/fact-sheets/detail/breast-cancer#:~:text=Reducing%20global%20breast%20cancer%20mortality,and%20comprehensive%20breast%20cancer%20management .

International Agency for Research on Cancer (IARC). IARC Handbooks on Cancer Screening: Volume 15 Breast Cancer Geneva: IARC; 2016 [Available from: https://publications.iarc.fr/Book-And-Report-Series/Iarc-Handbooks-Of-Cancer-Prevention/Breast-Cancer-Screening-2016 .

Australian Institute of Health and Welfare. Cancer in Australia 2021 [Available from: https://www.canceraustralia.gov.au/cancer-types/breast-cancer/statistics .

Breast Cancer Network Australia. Current breast cancer statistics in Australia 2020 [Available from: https://www.bcna.org.au/media/7111/bcna-2019-current-breast-cancer-statistics-in-australia-11jan2019.pdf .

Ren W, Chen M, Qiao Y, Zhao F. Global guidelines for breast cancer screening: A systematic review. The Breast. 2022;64:85–99.

Article   PubMed   PubMed Central   Google Scholar  

Cardoso F, Kyriakides S, Ohno S, Penault-Llorca F, Poortmans P, Rubio IT, et al. Early breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2019;30(8):1194–220.

Article   CAS   PubMed   Google Scholar  

Hamashima C, Hattori M, Honjo S, Kasahara Y, Katayama T, Nakai M, et al. The Japanese guidelines for breast cancer screening. Jpn J Clin Oncol. 2016;46(5):482–92.

Article   PubMed   Google Scholar  

Bevers TB, Helvie M, Bonaccio E, Calhoun KE, Daly MB, Farrar WB, et al. Breast cancer screening and diagnosis, version 3.2018, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Net. 2018;16(11):1362–89.

Article   Google Scholar  

He J, Chen W, Li N, Shen H, Li J, Wang Y, et al. China guideline for the screening and early detection of female breast cancer (2021, Beijing). Zhonghua Zhong liu za zhi [Chinese Journal of Oncology]. 2021;43(4):357–82.

CAS   PubMed   Google Scholar  

Cancer Australia. Early detection of breast cancer 2021 [cited 2022 25 July]. Available from: https://www.canceraustralia.gov.au/resources/position-statements/early-detection-breast-cancer .

Schünemann HJ, Lerda D, Quinn C, Follmann M, Alonso-Coello P, Rossi PG, et al. Breast Cancer Screening and Diagnosis: A Synopsis of the European Breast Guidelines. Ann Intern Med. 2019;172(1):46–56.

World Health Organization. WHO Position Paper on Mammography Screening Geneva WHO. 2016.

Google Scholar  

Lansdorp-Vogelaar I, Gulati R, Mariotto AB. Personalizing age of cancer screening cessation based on comorbid conditions: model estimates of harms and benefits. Ann Intern Med. 2014;161:104.

Lee CS, Moy L, Joe BN, Sickles EA, Niell BL. Screening for Breast Cancer in Women Age 75 Years and Older. Am J Roentgenol. 2017;210(2):256–63.

Broeders M, Moss S, Nystrom L. The impact of mammographic screening on breast cancer mortality in Europe: a review of observational studies. J Med Screen. 2012;19(suppl 1):14.

Oeffinger KC, Fontham ETH, Etzioni R, Herzig A, Michaelson JS, Shih YCT, et al. Breast cancer screening for women at average risk: 2015 Guideline update from the American cancer society. JAMA - Journal of the American Medical Association. 2015;314(15):1599–614.

Walter LC, Schonberg MA. Screening mammography in older women: a review. JAMA. 2014;311:1336.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Braithwaite D, Walter LC, Izano M, Kerlikowske K. Benefits and harms of screening mammography by comorbidity and age: a qualitative synthesis of observational studies and decision analyses. J Gen Intern Med. 2016;31:561.

Braithwaite D, Mandelblatt JS, Kerlikowske K. To screen or not to screen older women for breast cancer: a conundrum. Future Oncol. 2013;9(6):763–6.

Demb J, Abraham L, Miglioretti DL, Sprague BL, O’Meara ES, Advani S, et al. Screening Mammography Outcomes: Risk of Breast Cancer and Mortality by Comorbidity Score and Age. Jnci-Journal of the National Cancer Institute. 2020;112(6):599–606.

Demb J, Akinyemiju T, Allen I, Onega T, Hiatt RA, Braithwaite D. Screening mammography use in older women according to health status: a systematic review and meta-analysis. Clin Interv Aging. 2018;13:1987–97.

Qaseem A, Lin JS, Mustafa RA, Horwitch CA, Wilt TJ. Screening for Breast Cancer in Average-Risk Women: A Guidance Statement From the American College of Physicians. Ann Intern Med. 2019;170(8):547–60.

Collins K, Winslow M, Reed MW, Walters SJ, Robinson T, Madan J, et al. The views of older women towards mammographic screening: a qualitative and quantitative study. Br J Cancer. 2010;102(10):1461–7.

Welch HG, Black WC. Overdiagnosis in cancer. J Natl Cancer Inst. 2010;102(9):605–13.

Hersch J, Jansen J, Barratt A, Irwig L, Houssami N, Howard K, et al. Women’s views on overdiagnosis in breast cancer screening: a qualitative study. BMJ : British Medical Journal. 2013;346:f158.

De Gelder R, Heijnsdijk EAM, Van Ravesteyn NT, Fracheboud J, Draisma G, De Koning HJ. Interpreting overdiagnosis estimates in population-based mammography screening. Epidemiol Rev. 2011;33(1):111–21.

Monticciolo DL, Helvie MA, Edward HR. Current issues in the overdiagnosis and overtreatment of breast cancer. Am J Roentgenol. 2018;210(2):285–91.

Shepardson LB, Dean L. Current controversies in breast cancer screening. Semin Oncol. 2020;47(4):177–81.

National Cancer Control Centre. Cancer incidence in Australia 2022 [Available from: https://ncci.canceraustralia.gov.au/diagnosis/cancer-incidence/cancer-incidence .

Austin JD, Shelton RC, Lee Argov EJ, Tehranifar P. Older Women’s Perspectives Driving Mammography Screening Use and Overuse: a Narrative Review of Mixed-Methods Studies. Current Epidemiology Reports. 2020;7(4):274–89.

Austin JD, Tehranifar P, Rodriguez CB, Brotzman L, Agovino M, Ziazadeh D, et al. A mixed-methods study of multi-level factors influencing mammography overuse among an older ethnically diverse screening population: implications for de-implementation. Implementation Science Communications. 2021;2(1):110.

Demb J, Allen I, Braithwaite D. Utilization of screening mammography in older women according to comorbidity and age: protocol for a systematic review. Syst Rev. 2016;5(1):168.

Housten AJ, Pappadis MR, Krishnan S, Weller SC, Giordano SH, Bevers TB, et al. Resistance to discontinuing breast cancer screening in older women: A qualitative study. Psychooncology. 2018;27(6):1635–41.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Peters M, Godfrey C, McInerney P, Munn Z, Tricco A, Khalil HAE, et al. Chapter 11: Scoping reviews. JBI Manual for Evidence Synthesis 2020 [Available from: https://jbi-global-wiki.refined.site/space/MANUAL .

Peters MD, Godfrey C, McInerney P, Khalil H, Larsen P, Marnie C, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI evidence synthesis. 2022;20(4):953–68.

Fantom NJ, Serajuddin U. The World Bank’s classification of countries by income. World Bank Policy Research Working Paper; 2016.

Book   Google Scholar  

BreastScreen Australia Evaluation Taskforce. BreastScreen Australia Evaluation. Evaluation final report: Screening Monograph No 1/2009. Canberra; Australia Australian Government Department of Health and Ageing; 2009.

Nelson HD, Cantor A, Humphrey L. Screening for breast cancer: a systematic review to update the 2009 U.S. Preventive Services Task Force recommendation2016.

Woolf SH. The 2009 breast cancer screening recommendations of the US Preventive Services Task Force. JAMA. 2010;303(2):162–3.

Covidence systematic review software. [Internet]. Veritas-Health-Innovation 2020. Available from: https://www.covidence.org/ .

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Beckmeyer A, Smith RM, Miles L, Schonberg MA, Toland AE, Hirsch H. Pilot Evaluation of Patient-centered Survey Tools for Breast Cancer Screening Decision-making in Women 75 and Older. Health Behavior and Policy Review. 2020;7(1):13–8.

Brotzman LE, Shelton RC, Austin JD, Rodriguez CB, Agovino M, Moise N, et al. “It’s something I’ll do until I die”: A qualitative examination into why older women in the U.S. continue screening mammography. Canc Med. 2022;11(20):3854–62.

Article   CAS   Google Scholar  

Cadet T, Pinheiro A, Karamourtopoulos M, Jacobson AR, Aliberti GM, Kistler CE, et al. Effects by educational attainment of a mammography screening patient decision aid for women aged 75 years and older. Cancer. 2021;127(23):4455–63.

Cadet T, Aliberti G, Karamourtopoulos M, Jacobson A, Gilliam EA, Primeau S, et al. Evaluation of a mammography decision aid for women 75 and older at risk for lower health literacy in a pretest-posttest trial. Patient Educ Couns. 2021;104(9):2344–50.

Cadet T, Aliberti G, Karamourtopoulos M, Jacobson A, Siska M, Schonberg MA. Modifying a mammography decision aid for older adult women with risk factors for low health literacy.  Health Lit Res Prac. 2021;5(2):e78–90.

Gray N, Picone G. Evidence of Large-Scale Social Interactions in Mammography in the United States. Atl Econ J. 2018;46(4):441–57.

Hoover DS, Pappadis MR, Housten AJ, Krishnan S, Weller SC, Giordano SH, et al. Preferences for Communicating about Breast Cancer Screening Among Racially/Ethnically Diverse Older Women. Health Commun. 2019;34(7):702–6.

Salzman B, Bistline A, Cunningham A, Silverio A, Sifri R. Breast Cancer Screening Shared Decision-Making in Older African-American Women. J Natl Med Assoc. 2020;112(5):556–60.

PubMed   Google Scholar  

Schoenborn NL, Pinheiro A, Kistler CE, Schonberg MA. Association between Breast Cancer Screening Intention and Behavior in the Context of Screening Cessation in Older Women. Med Decis Making. 2021;41(2):240–4.

Schonberg MA, Kistler CE, Pinheiro A, Jacobson AR, Aliberti GM, Karamourtopoulos M, et al. Effect of a Mammography Screening Decision Aid for Women 75 Years and Older: A Cluster Randomized Clinical Trial. JAMA Intern Med. 2020;180(6):831–42.

Schonberg MA, Hamel MB, Davis RB. Development and evaluation of a decision aid on mammography screening for women 75 years and older. JAMA Intern Med. 2014;174:417.

Eisinger F, Viguier J, Blay J-Y, Morère J-F, Coscas Y, Roussel C, et al. Uptake of breast cancer screening in women aged over 75years: a controversy to come? Eur J Cancer Prev. 2011;20(Suppl 1):S13-5.

Schonberg MA, Breslau ES, McCarthy EP. Targeting of Mammography Screening According to Life Expectancy in Women Aged 75 and Older. J Am Geriatr Soc. 2013;61(3):388–95.

Download references

Acknowledgements

We would like to acknowledge Ange Hayden-Johns (expert librarian) who assisted with the development of the search criteria and undertook the relevant searches and Tejashree Kangutkar who assisted with some of the Covidence work.

This work was supported by funding from the Australian Government Department of Health and Aged Care (ID: Health/20–21/E21-10463).

Author information

Authors and affiliations.

Violet Vines Centre for Rural Health Research, La Trobe Rural Health School, La Trobe University, P.O. Box 199, Bendigo, VIC, 3552, Australia

Virginia Dickson-Swift, Joanne Adams & Evelien Spelten

Care Economy Research Institute, La Trobe University, Wodonga, Australia

Irene Blackberry

Olivia Newton-John Cancer Wellness and Research Centre, Austin Health, Melbourne, Australia

Carlene Wilson & Eva Yuen

Melbourne School of Population and Global Health, Melbourne University, Melbourne, Australia

Carlene Wilson

School of Psychology and Public Health, La Trobe University, Bundoora, Australia

Institute for Health Transformation, Deakin University, Burwood, Australia

Centre for Quality and Patient Safety, Monash Health Partnership, Monash Health, Clayton, Australia

You can also search for this author in PubMed   Google Scholar

Contributions

VDS conceived and designed the scoping review. VDS & JA developed the search strategy with librarian support, and all authors (VDS, JA, ES, IB, CW, EY) participated in the screening and data extraction stages and assisted with writing the review. All authors provided editorial support and read and approved the final manuscript prior to submission.

Corresponding author

Correspondence to Joanne Adams .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval and consent to participate

Ethics approval and consent to participate was not required for this study.

Consent for publication

Consent for publication was not required for this study.

Competing interest

The authors declare they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Dickson-Swift, V., Adams, J., Spelten, E. et al. Breast cancer screening motivation and behaviours of women aged over 75 years: a scoping review. BMC Women's Health 24 , 256 (2024). https://doi.org/10.1186/s12905-024-03094-z

Download citation

Received : 06 September 2023

Accepted : 15 April 2024

Published : 24 April 2024

DOI : https://doi.org/10.1186/s12905-024-03094-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Breast cancer
• Mammography
• Older women
• Scoping review

BMC Women's Health

ISSN: 1472-6874

• Submission enquiries: [email protected]
• General enquiries: [email protected]

This paper is in the following e-collection/theme issue:

Published on 25.4.2024 in Vol 26 (2024)

Effect of Prosocial Behaviors on e-Consultations in a Web-Based Health Care Community: Panel Data Analysis

Authors of this article:

Original Paper

• Xiaoxiao Liu 1, 2 , PhD   ; 
• Huijing Guo 3 , PhD   ; 
• Le Wang 4 , PhD   ; 
• Mingye Hu 5 , PhD   ; 
• Yichan Wei 1 , BBM   ; 
• Fei Liu 6 , PhD   ; 
• Xifu Wang 7 , MCM  

1 School of Management, Xi’an Jiaotong University, Xi'an, China

2 China Institute of Hospital Development and Reform, Xi'an Jiaotong University, Xi'an, China

3 School of Economics and Management, China University of Mining and Technology, Xuzhou, China

4 College of Business, City University of Hong Kong, Hong Kong, China (Hong Kong)

5 School of Economics and Management, Xi’an University of Technology, Xi'an, China

6 School of Management, Harbin Engineering University, Harbin, China

7 Healthcare Simulation Center, Guangzhou First People’s Hospital, Guangzhou, China

Corresponding Author:

Xifu Wang, MCM

Healthcare Simulation Center

Guangzhou First People’s Hospital

1 Pan Fu Road

Yuexiu District

Guangzhou, 510180

Phone: 86 13560055951

Email: [email protected]

Background: Patients using web-based health care communities for e-consultation services have the option to choose their service providers from an extensive digital market. To stand out in this crowded field, doctors in web-based health care communities often engage in prosocial behaviors, such as proactive and reactive actions, to attract more users. However, the effect of these behaviors on the volume of e-consultations remains unclear and warrants further exploration.

Objective: This study investigates the impact of various prosocial behaviors on doctors’ e-consultation volume in web-based health care communities and the moderating effects of doctors’ digital and offline reputations.

Methods: A panel data set containing information on 2880 doctors over a 22-month period was obtained from one of the largest web-based health care communities in China. Data analysis was conducted using a 2-way fixed effects model with robust clustered SEs. A series of robustness checks were also performed, including alternative measurements of independent variables and estimation methods.

Results: Results indicated that both types of doctors’ prosocial behaviors, namely, proactive and reactive actions, positively impacted their e-consultation volume. In terms of the moderating effects of external reputation, doctors’ offline professional titles were found to negatively moderate the relationship between their proactive behaviors and their e-consultation volume. However, these titles did not significantly affect the relationship between doctors’ reactive behaviors and their e-consultation volume ( P =.45). Additionally, doctors’ digital recommendations from patients negatively moderated both the relationship between doctors’ proactive behaviors and e-consultation volume and the relationship between doctors’ reactive behaviors and e-consultation volume.

Conclusions: Drawing upon functional motives theory and social exchange theory, this study categorizes doctors’ prosocial behaviors into proactive and reactive actions. It provides empirical evidence that prosocial behaviors can lead to an increase in e-consultation volume. This study also illuminates the moderating roles doctors’ digital and offline reputations play in the relationships between prosocial behaviors and e-consultation volume.

Introduction

e-Consultations, offered through web-based health care communities [ 1 ], are increasingly becoming vital complements to traditional hospital services [ 2 - 4 ]. In hospital consultations, patients can only passively accept treatment [ 5 ] from a limited pool of medical resources within a geographical radius. However, when engaging with web-based health care communities, patients can search for primary care solutions [ 6 ] from an extensive digital market in a relatively short time [ 7 ]. Given that the diagnostic accuracy of e-consultations matches that of hospital consultations [ 8 - 10 ], e-consultations are becoming increasingly attractive to patients [ 3 , 11 ].

Doctors are also showing a growing interest in e-consultations, motivated by economic and social benefits. First, doctors can achieve economic gains by participating in e-consultations [ 7 , 12 ]. Web-based consultation platforms facilitate an efficient reputation system, enabling patients to easily provide feedback about doctors. Consequently, doctors can use e-consultation to strengthen their relationship with patients [ 13 , 14 ] and foster positive word-of-mouth [ 15 ]. More e-consultations can benefit doctors by retaining current patients, attracting new ones, and boosting in-person hospital visits [ 16 , 17 ]. Second, doctors could also receive social returns from engaging in e-consultation [ 7 ]. Active participation in e-consultations allows doctors to demonstrate their skills, attitude, and experience, aiding in accumulating professional capital [ 7 ], building their reputation [ 18 ], and increasing their social influence [ 19 ]. Given these tangible and intangible benefits, it is essential for doctors to diligently provide the desired e-consultations and make additional efforts to highlight their service attributes to stand out [ 6 , 20 , 21 ]. This involves engaging in prosocial behaviors in web-based health care communities, which is the primary research focus of this study.

Prior studies have examined the effects of prosocial behaviors on financial outcomes, such as actions reflecting social responsibility in the workplace [ 22 ]. In the health care sector, previous research has explored doctors’ prosocial behaviors within traditional, offline medical services. Doctors, working in established medical institutes and serving patients with limited choices of clinical service providers, often aim for self-satisfaction and patient satisfaction with their offline prosocial behaviors. For example, research indicates that doctors may act prosocially to regulate their self-oriented feelings [ 23 ] and foster a caring and understanding attitude toward patients [ 24 , 25 ]. Additionally, doctors who demonstrate more empathy and care can elicit positive emotions in patients and improve the doctor-patient relationship [ 26 , 27 ].

Compared to the offline context, doctors’ prosocial behaviors in a digital context may differ in 2 aspects. First, the internet allows patients to choose from a broader, more diverse range of doctors without the constraints of time and space [ 7 ]. However, the uncertainty inherent in the digital environment creates a more pronounced information asymmetry between patients and doctors [ 28 ], consequently making it more challenging for patients to establish trust. Therefore, doctors’ prosocial behaviors are crucial in building their self-image, establishing patients’ trust, and assisting patients in identifying suitable doctors [ 29 , 30 ]. Second, unlike offline environments, web-based medical platforms offer a range of functions, including asynchronous activities such as publishing articles, as well as real-time interactional actions such as answering questions during live streams. This array of functions facilitates the adoption of more diverse prosocial behaviors by doctors.

Although these differences underscore the importance of studying doctors’ prosocial behavior, there has been limited research focusing on the impact of such behaviors in the digital context. One previous study has scrutinized the impact of prosocial behaviors, such as answering patients’ questions freely, on patient engagement within web-based health care communities [ 31 ]. An aspect that requires further exploration is how doctors’ motivations and patients’ involvement vary in doctors’ helping behaviors. Consequently, studies on web-based health care communities should differentiate between diverse prosocial actions to understand their effects on doctors’ web-based service outcomes. This study aims to contribute new knowledge regarding the full breadth of doctors’ prosocial behaviors.

Unlike the previous study that exclusively investigated doctors’ asynchronous behaviors in web-based health care communities [ 31 ], this study also explores the role of synchronous reactive actions in achieving optimal doctors’ e-consultation volume. Recently, web-based health care communities have developed and released live-streaming functions to assist doctors in providing voluntary interactions with patients. The effect of doctors’ engagement in medical live streaming on e-consultation services remains unexplored. While these behaviors could demonstrate doctors’ ethical traits and ability to fulfill an e-consultation workflow, a potential trade-off with e-consultations may exist when doctors engage in prosocial behaviors.

In summary, this study examines the effects of doctors’ proactive and reactive prosocial behaviors, considering their digital and offline reputations as potential moderating factors. First, drawing from functional motives theory (FMT), we explore the impact of doctors’ web-based proactive actions on their e-consultation volume. Proactive behaviors are actions in which individuals exceed their assigned work, focusing on long-term goals to prevent future problems [ 32 , 33 ]. According to FMT, these behaviors reflect helping actions that satisfy personal needs [ 34 ], driven by self-focused motivations [ 35 ], such as impression management and the realization of self-worth goals. For example, knowledge-based proactive behaviors, such as disseminating expertise to preempt future issues, are self-initiated and not reactions to immediate requests [ 36 ]. This study categorizes doctors’ sharing of professional articles as a form of proactive behavior that creates a professional image for their patient audience. This is because these actions aim to assist patients with future health concerns rather than directly responding to patients’ immediate needs.

Second, this study explores the role of doctors’ reactive prosocial behaviors in increasing e-consultations, guided by social exchange theory (SET). Unlike proactive behaviors, reactive behaviors are characterized by instances of individuals engaging in helping activities [ 35 ], typically in response to others’ needs [ 34 ]. SET posits that individuals incurring additional social costs in relationships may anticipate reciprocal value [ 37 , 38 ]. Reactive prosocial behaviors, per SET, are initiated by the motivation to satisfy others’ desires, leading to the development of cooperative social values. In our context, medical live streams facilitate real-time, synchronized interactions, enabling patients to ask questions and doctors to provide immediate responses. Patients’ health questions during these streams indicate their immediate needs. Thus, a higher frequency of live streams within a certain period suggests doctors are increasingly responding to patients’ needs during that time. Therefore, this study uses the number of medical live-streaming sessions conducted by doctors as a measure for their synchronous reactive behaviors.

Finally, considering that doctors’ reputations play a crucial role in their workflow on web-based health care communities [ 39 , 40 ], we test the moderating roles of digital and offline reputation—measured by doctors’ offline professional titles and patients’ recommendations in the digital context, respectively—on the main effects.

Based on previous studies and practices within web-based health care communities, we aim to extend the literature by testing the impact of 2 types of web-based prosocial behaviors by doctors: proactive and synchronous reactive actions on e-consultation volume. We then explore the moderating roles of doctors’ offline and digital reputations on these main effects.

Research Framework and Hypothesis Development

We have developed a research framework, shown in Figure 1 , to identify effective prosocial strategies used by doctors within web-based health care communities to achieve a preferred e-consultation volume from the supply side.

Primarily, we explore the relationships between doctors’ prosocial behaviors and e-consultation volume, drawing on FMT and SET. These theories are widely adopted for measuring and classifying the outcomes of prosocial behaviors from 2 fundamental perspectives based on human nature [ 34 ]. While doctors’ offline prosocial behaviors may help satisfy patients [ 24 , 25 ], who are already service acceptors, the outcomes of doctors’ web-based prosocial behaviors still need careful distinction. It is essential to clearly differentiate between various types of doctors’ prosocial behaviors to identify their nature. In this study, following the leads of FMT and SET, we test 2 kinds of prosocial behavior: proactive (posting professional articles to achieve self-worth) and reactive (conducting medical live streaming to create cooperative social values).

Subsequently, we examine how doctors’ external reputation moderates the impacts of doctors’ proactive and reactive prosocial behaviors. This examination is conducted from the perspectives of reducing uncertainty and building trust, respectively.

Doctors’ Proactive Behaviors and e-Consultation Volume

FMT places emphasis on the primary motivations behind individuals’ behaviors, adopting an atheoretical stance [ 41 ]. Through the exploratory process, previous studies have provided examples to identify the functional motivations behind prosocial behaviors [ 42 ], such as expressing important personal values. In web-based health care communities, doctors have the opportunity to demonstrate personal traits through proactive behaviors. According to FMT, these proactive behaviors stem from the actors’ active efforts to satisfy their own needs and achieve self-worth [ 34 , 35 ].

Doctors might post professional articles, such as clinical notes and scientific papers, on web-based health care communities to help patient readers handle future health problems. These proactive prosocial behaviors are primarily driven by a desire to showcase personal medical competence, a crucial characteristic of a professional image [ 43 ], in medical consultations. By posting professional articles, doctors can display their medical knowledge, care delivery capability, and service quality, thereby enhancing their professional image. We hypothesize that this effort will lead to an increase in the e-consultation volume. Therefore, we propose the following hypothesis:

• Hypothesis 1: The posting of professional articles by doctors positively impacts their e-consultation volume on web-based health care communities.

Doctors’ Reactive Behaviors and e-Consultation Volume

Considering the social environment in the working context, SET suggests that reactive prosocial behaviors stem from responding to others’ needs [ 34 ]. Engaging in such behaviors can foster positive perceptions among the audience and build cooperative social values [ 44 ] through reactive social exchange. People with a high orientation toward cooperative social values act to maximize mutual interests [ 45 ], a trait highly valued in the medical field.

We use medical live streaming as a measure of doctors’ reactive behaviors on web-based health care communities. Volunteering to provide interactional live streaming, a typical reactive behavior that may generate cooperative social value, gives the patient audience the impression that the doctors will prioritize demand-side interests during e-consultation services. Additionally, engaging in medical live streaming allows doctors to present themselves as authentic and recognized experts. This enhances their social presence [ 46 ], potentially leading to increased service use [ 47 ] and greater popularity [ 48 ]. Consequently, patients are more likely to perceive doctors who participate in medical live streaming as trustworthy for consultations. Given that e-consultations are closely related to the health conditions of the demand side, a credible doctor is likely to attract more e-consultations. Therefore, we propose the following hypothesis:

• Hypothesis 2: The conduct of medical live streaming by doctors positively impacts their e-consultation volume on web-based health care communities.

Moderating Roles of Offline and Digital Reputation

As doctors’ proactive and reactive behaviors potentially affect their consultation performance, based on 2 distinct theoretical foundations of human nature, there exists a discrepancy in how doctors’ reputations influence the relationship between various prosocial behaviors and e-consultation.

We formulate hypotheses regarding the moderating effects within the context of digital health care, by taking into account the inherent information asymmetry and the significance of establishing patient trust. Specifically, our hypotheses explore the influence of reputation on the relationship between doctors’ proactive behaviors and e-consultation volume, with a focus on reducing uncertainty. Additionally, we examine how reputation moderates the impact of doctors’ reactive behaviors, emphasizing the perspective of trust building.

First, in the marketing literature, service providers’ reputations, which can reduce information asymmetry and purchase uncertainty [ 49 ], are key factors influencing purchasing behavior and sales performance in the digital context [ 50 - 52 ]. Similarly, for doctors, reputations are related to the experiences and beliefs of other stakeholders [ 53 ]. As health care services are credence goods [ 54 ]—whose quality patients cannot discern even after experiencing the services—and given the nature of web-based platforms (eg, the absence of face-to-face meetings), there is a significant information asymmetry [ 51 ]. This increases patients’ uncertainty regarding the quality of doctors. Consequently, doctors’ reputations play crucial roles in patients’ decision-making processes [ 18 , 39 ]. We use doctors’ professional titles and patients’ recommendations on web-based health care communities to measure doctors’ offline and digital reputations.

Proactive behaviors by low-reputation doctors can create deeper professional impressions [ 34 , 35 ] to reduce uncertainty in e-consultations than high-reputation doctors, who are less uncertain in medical services. Then, doctors’ reputations—measured by offline professional titles and digital patients’ recommendations on web-based health care communities—will negatively moderate the relationship between proactive behavior and e-consultation volume. Thus, we propose the following hypotheses:

• Hypothesis 3a: Doctors’ offline professional titles negatively moderate the relationship between the posting of professional articles and e-consultation volume on web-based health care communities.
• Hypothesis 3b: Doctors’ digital recommendations from patients negatively moderate the relationship between the posting of professional articles and e-consultation volume on web-based health care communities.

Second, one of the central elements of SET is the concept of trust between actors in the exchange process [ 55 - 58 ]. In the context of digital health, patient’s trust in doctors is important to establish in order to refine the doctor-patient relationship. Doctors’ reputations can reflect their personality traits [ 39 ] and promote trust from patients [ 53 ]. Conducting medical live streaming, a form of reactive prosocial behavior, includes doctors’ cooperative social value orientations that are preferred in e-consultations. For low-reputation doctors, such as those with relatively junior professional titles and few digital patient recommendations, conducting medical live streaming will build patients’ confidence in e-consultations to a greater extent than doctors with high reputations, who are usually already highly trusted. Then, offline and digital reputation may negatively moderate the relationship between engaging in medical live streaming and e-consultation volume. Thus, we propose the final hypotheses:

• Hypothesis 4a: Doctors’ offline professional titles will negatively moderate the relationship between conducting medical live streaming and e-consultation volume on web-based health care communities.
• Hypothesis 4b: Doctors’ digital recommendations from patients will negatively moderate the relationship between conducting medical live streaming and e-consultation volume on web-based health care communities.

Research Context and Data Collection

Our research context is one of the largest web-based health care communities in China. This platform, established in 2006, offers e-consultation services to patients. As of July 2023, it boasts over 260,000 active doctors from 10,000 hospitals nationwide and has provided web-based medical services to 79 million patients.

The platform allows doctors to create home pages where they can display relevant information such as offline professional titles, experiences shared by other patients, and personal introductions. Patients can select doctors for e-consultation by browsing this information. Besides e-consultation, doctors can engage in prosocial behavior primarily focused on knowledge sharing. This includes posting professional articles in various formats (text, voice, and short videos) and conducting medical live streams for real-time interaction with patients.

We collected data over a 22-month period, from January 2021 to October 2022, focusing on common diseases such as diabetes, depression, infertility, skin diseases, and gynecological diseases. To ensure that our findings are generalizable to a typical and active doctor on the platform, we included doctors who had posted at least 1 article and conducted at least 1 live stream before the end of the study period in our analysis [ 59 - 61 ]. Our sample consists of 2880 doctors and includes the following information for each doctor: professional title, patient recommendations, records of experiences shared by the doctor’s patients, records of professional articles posted, records of live streams conducted, and records of the doctor’s e-consultations.

Variable Operationalization

Our unit of analysis is each doctor. We investigate how doctors’ prosocial behaviors, including proactive behaviors (posting professional articles) and reactive behaviors (conducting medical live streams), influence their e-consultation volume.

Dependent Variable

Our dependent variable is the doctors’ e-consultation volume, denoted as Consultation it , which is measured by the number of e-consultations of doctor i in month t .

Independent Variables

Our independent variables are doctors’ proactive behaviors and reactive behaviors. Doctors’ proactive behavior is operationalized as the posting of professional articles. Specifically, we denote proactive behavior as Articles it , which is measured by the number of professional articles posted by doctor i in month t . Doctors’ reactive behavior is operationalized as medical live streaming. This variable is denoted as LiveStreaming it , which is calculated as the number of medical live streams conducted by doctor i in month t .

Moderating Variables

We are also interested in how doctors’ external reputation, including their offline professional titles and digital recommendations from patients, influences the relationship between prosocial behaviors and e-consultation volume. A doctor’s offline professional title is denoted as Title i , which is a dummy variable indicating whether doctor i is a chief doctor ( Title i =1 indicates the doctor is a chief doctor, and Title i =0 indicates the doctor has a lower-ranked title). Digital recommendations are captured by Recommendations i , which is the digital recommendation level of doctor i as calculated by the platform based on the recommendations provided by their past patients.

Control Variables

We incorporated several control variables to account for factors that may influence patient’s choices of doctors in the digital context. The shared experiences of patients regarding a doctor’s treatment [ 39 ], as well as the number of patients who have previously consulted with the doctors [ 17 , 62 ], can indicate the doctor’s overall popularity. This, in turn, may affect patient choice. Therefore, we controlled for (1) the total number of patients who consulted with doctor i in the digital context before month t ( TotalPatients it ) and (2) the total number of patient-shared experiences about offline treatment by doctor i before month t ( TotalExperiences it ). Furthermore, doctors’ past behaviors, including article publishing and live streaming, can influence their current practices in posting articles and conducting live streams. Simultaneously, these factors may also act as signals affecting patients’ judgments and selection of doctors [ 12 ]. To account for these influences, we also controlled for (1) the total number of articles posted by doctor i before month t ( TotalArticles it ) and (2) the total number of medical live streams conducted by doctor i before month t ( TotalLiveStreaming it ).

To control for both observed and unobserved doctor-specific factors that do not change over time, individual-fixed effects were added. Additionally, time-fixed effects were introduced into our analysis to account for both observed and unobserved factors that vary over time but remain constant across doctors. Table 1 shows the variables and their definitions.

Estimation Model

To estimate the direct impact of doctors’ proactive behaviors and reactive behaviors on their e-consultation volume, the following 2-way fixed effects regression model was used:

Consultation it = β 0 + β 1 Articles it + β 2 LiveStreaming it + β 3 TotalPatients it + β 4 TotalExperiences it + β 5 TotalArticles it + β 6 TotalLiveStreaming it + α i + δ t + μ it (1)

where i denotes doctor, t denotes month, α i is doctor-fixed effects, δ t is month-fixed effects, Consultation it is the number of e-consultations of doctor i in month t , Articles it is the number of professional articles posted by doctor i in month t , LiveStreaming it is the number of medical live streams conducted by doctor i in month t , TotalPatients it is the total number of patients who consulted doctor i in the digital context before month t , TotalExperiences it is the total number of patient-shared experiences about offline treatment by doctor i before month t , TotalArticles it is the total number of articles posted by doctor i before month t , TotalLiveStreaming it is the total number of medical live streams doctor i conducted before month t , β is the coefficient, and μ it is the error term. We took the log transformation for our continuous variables in the model to reduce the skewness of the variables [ 63 ].

Next, the moderating effects of doctors’ offline professional titles and digital recommendations by patients were investigated based on the following specification:

Consultation it = β 0 + β 1 Articles it + β 2 LiveStreaming it + β 3 Articles it × Title i + β 4 LiveStreaming it × Title i + β 5 Articles it × Recommendation i + β 6 LiveStreaming it × Recommendation i + β 7 TotalPatients it + β 8 TotalExperiences it + β 9 TotalArticles it + β 10 TotalLiveStreaming it + α i + δ t + μ it (2)

where Title i indicates whether doctor i is a chief doctor ( Title i =1 indicates the doctor is a chief doctor, and Title i =0 indicates the doctor has a lower-ranked title). Recommendations i is the digital recommendation level of doctor i by other patients.

Ethical Considerations

This study used secondary publicly available data obtained from a website and did not involve the collection of original data pertaining to human participants. As such, there is no evidence of unethical behavior in the study. Consequently, ethics approval by an ethics committee or institutional review board was not deemed necessary.

In this section, we present our empirical results. The descriptive statistics are shown in Table 2 , and the correlation matrix is shown in Table 3 .

Empirical Results

Results for direct effects.

The analysis was conducted progressively. We first estimated the equation without control variables (model 1) and then added control variables in model 2. The estimated results are shown in Table 4 . From the results, we can see that the coefficient of Articles is significant and positive in model 2 (β=.093; P <.001), indicating that doctors’ proactive behaviors (ie, posting professional articles) can help them obtain more e-consultations. Thus, hypothesis 1 is supported. Regarding doctors’ engagement in medical live streaming, the results show that the coefficient of LiveStreaming is significantly positive (β=.214; P <.001), which suggests that doctors’ reactive behaviors (ie, conducting medical live streaming) can increase their e-consultation volume. This supports hypothesis 2.

a All models include doctor-fixed effects and month-fixed effects; robust SEs clustered by doctors are reported; the number of doctors is 2880, and the number of observations is 63,360.

b R 2 =0.843; F 2,2879 =175.98; P <.001.

c R 2 =0.851; F 6,2879 =119.72; P <.001.

d N/A: not applicable.

Results for Moderating Effects

The results for moderating effects are shown in Table 5 . In model 1, interaction terms were initially introduced between Title and Articles , as well as between Title and LiveStreaming , to estimate the moderating effect of doctors’ offline professional titles. The interaction terms were then added between Recommendations and Articles , as well as between Recommendations and LiveStreaming , to estimate the moderating effect of doctors’ digital recommendations in model 2. Finally, a full model was estimated by incorporating all interaction terms. We find that the results are consistent across all models. Wald tests and likelihood ratio were used to compare the fit among nested models [ 64 , 65 ], and the results show that the inclusion of moderating variables significantly enhances the model’s fit.

Regarding the moderating effect of doctors’ offline professional titles, we find that the coefficient of Articles × Title in model 1 of Table 5 is significantly negative (β=–.058; P <.001), which supports hypothesis 3a that doctors’ offline professional titles have a negative moderating effect on the relationship between doctors’ proactive behaviors and e-consultation volume. However, the coefficient of LiveStreaming × Title is insignificant (β=–.024; P =.45), which suggests that doctors’ offline professional titles have no moderating effect on the relationship between doctors’ reactive behaviors and e-consultation volume. Thus, hypothesis 4a is not supported.

b R 2 =0.851; F 8,2879 =89.98; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

c R 2 =0.851; F 8,2879 =89.13; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

d R 2 =0.852; F 10,2879 =71.44; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

e N/A: not applicable.

For the moderating effect of digital patient recommendations, we find that both of the coefficients of Articles​ × Recommendations and LiveStreaming × Recommendations are negative and significant (β=–.055; P <.001 and β=–.100; P <.001, respectively, in model 2 of Table 5 ). This indicates that digital recommendations from patients have negative moderating effects on the relationship between doctors’ proactive behaviors and e-consultation volume as well as on the relationship between doctors’ reactive behaviors and e-consultation volume; this finding supports hypotheses 3b and 4b.

Robustness Check

First, additional analysis was performed to check whether our findings are robust to different measures of doctors’ reactive behaviors. In the main analysis, we used the number of medical live streams to construct doctors’ reactive behaviors. In the robustness check, doctors’ reactive behaviors were measured using the following measures: (1) the length of time spent in medical live streaming ( LSDuration it ), which is calculated as the total duration of all medical live streams conducted by doctor i in month t ; and (2) the number of doctor-patient interactions in the medical live streams ( LSInteractions it ), which is calculated as the total number of interactions between doctor i and patients in medical live streams in month t . This measure is likely to more effectively capture the reactive element of the behavior. The estimated results are shown in Table 6 , and we can see that the results are consistent with the main results.

Second, in the above analysis, the total number of articles posted by the doctors was used to measure doctors’ proactive behaviors. As doctors can post articles that are either their own original work or reposts from others, we further used the number of original articles ( OriArticles it ) to measure doctors’ proactive prosocial behaviors. Specifically, the number of articles was replaced with the number of original articles posted by doctor i in month t ( OriArticles it ). Models 1 and 2 in Table 7 show the results. We can see that using this alternative measure of proactive behavior does not materially change the results.

Third, as our dependent variable takes nonnegative values, negative binomial regression was further used to re-estimate our models. We find that the results (models 3 and 4 in Table 7 ) are similar to the main results.

Fourth, to further enhance the robustness and validity of our findings, article quality was used as a measure of doctors’ proactive behaviors. This approach is based on the premise that article quality more accurately reflects the effort and time invested by doctors in content creation. Specifically, we assessed article quality based on either the length of each article or the number of likes it received and then re-estimated our model. As indicated in Table 8 , the results remain consistent with our main findings, thereby further reinforcing the validity of our conclusions.

b R 2 =0.851; F 6,2879 =131.71; P <.001.

c R 2 =0.851; F 10,2879 =79.29; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

d R 2 =0.850; F 6,2879 =112.52; P <.001.

e R 2 =0.851; F 10,2879 =68.43; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

f N/A: not applicable.

a All models include doctor-fixed effects and month-fixed effects; robust SEs clustered by doctors are reported in models 1 and 2; bootstrap SEs in models 3 and 4.

b R 2 =0.851; F 6,2879 =118.99; P <.001.

c R 2 =0.852; F 10,2879 =71.04; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

d Log likelihood=–150,015.36.

e Log likelihood=–149,888.24.

b R 2 =0.851; F 6,2879 =127.75; P <.001.

c R 2 =0.851; F 10,2879 =89.97; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

d R 2 =0.851; F 6 , 2879 =133.39; P <.001.

e R 2 =0.852; F 10 , 2879 =84.94; P <.001; Wald test: P <.001; likelihood ratio: P <.001.

Analysis of Results

Web-based medical platforms offer a variety of functions to support doctors’ engagement in different types of prosocial behaviors. However, few studies have investigated the effects of these behaviors. Drawing on FMT and SET, this study categorized doctors’ prosocial practices in web-based health care communities into proactive and reactive actions and examined their effects on e-consultation volume. Briefly, prosocial behaviors positively impact on e-consultation, and a doctor’s digital and offline reputation moderates the relationship between prosocial behavior and e-consultation, albeit with some nuances.

First, we expanded upon existing literature on proactive prosocial behaviors, concluding that these actions can help doctors create professional images [ 43 ] in the medical consultation context. Our panel data analysis reveals that doctors’ posting of professional articles, which contribute to their professional image in the digital context, attracts more e-consultations. This finding aligns with the prior study [ 31 ], which observed that a health professional’s previous asynchronous prosocial behavior positively influences their future economic performance.

Second, drawing from SET, we analyzed the impact of synchronous reactive prosocial behaviors, a less explored area in prior literature. Our findings confirm that engaging in medical live streaming, a form of reactive prosocial behavior, leads to higher e-consultation volumes. Interestingly, we found that the positive impact of conducting a live stream exceeds that of posting an article.

Third, we expanded our research by testing the moderating roles of digital and offline reputations, measured by doctors’ offline professional titles and patients’ recommendations on web-based health care communities. We found that digital reputations significantly moderate the relationships between both types of prosocial behaviors and e-consultation volume. Specifically, doctors who post professional articles or conduct medical live streams attract more e-consultations when they have fewer patient recommendations compared to those with higher recommendations. Regarding offline professional titles, our results indicate a significant moderating effect on the relationship between proactive prosocial behaviors and e-consultation volume. Notably, junior doctors should focus more on posting articles in web-based health care communities to compensate for limitations associated with their titles [ 66 ]. However, the moderating effect of offline titles on the impact of reactive prosocial behaviors was found to be insignificant. We attribute this to the unique dynamics of trust conversion in Chinese health care settings. As doctors’ offline titles are granted by medical institutions, these titles could enhance patients’ trust in doctors only if there is a conversion of trust from the organization to the individual doctor, which represents different types of trust [ 67 ]. Consequently, doctors with the same offline titles from different hospitals may be perceived differently. For example, a senior doctor from a 3-A hospital is usually seen as highly professional in their clinical field, while a doctor with the same title in a 1-A hospital might typically handle primary diseases. Due to this trust conversion phenomenon, patients may not uniformly trust doctors from different hospitals with the same offline titles, leading to the insignificant moderating effect of offline titles on the impact of reactive prosocial behaviors.

In summary, this study underscores the importance of prosocial behaviors and reputation in shaping doctors’ e-consultation volumes on web-based health care communities, offering valuable insights for health care professionals aiming to increase their consultation outreach.

Implications

This study makes several theoretical implications. First, this study contributes to web-based health care community literature by offering a nuanced understanding of how doctors’ prosocial behaviors enhance e-consultation volume. While a limited number of studies have examined the effects of doctors’ freely provided behaviors in the digital context [ 31 ], the specific impact of different types of prosocial behaviors on e-consultation volume remains largely unexplored. This study addresses this knowledge gap by theoretically categorizing doctors’ prosocial behaviors in web-based health care communities into proactive and reactive types and exploring their impacts on e-consultations.

Second, this study enriches web-based health care communities and live streaming literature by validating the role of medical live streaming in web-based health services. Prior research on live streaming has mainly concentrated on e-commerce [ 68 ], web-based gaming [ 69 ], and web-based learning [ 70 ]. Our study extends this research to the health care context, highlighting the importance of live streaming on web-based health care platforms. Specifically, this study delves into how doctors’ synchronous, reactive volunteer interactions via live streaming influence patient decision-making.

Finally, this study advances FMT and SET by highlighting the importance of context in theory development and providing guidance for context-specific theorizing on web-based health platforms. It also sheds light on how the impact of different prosocial behaviors on e-consultation volume varies depending on a doctor’s offline and digital reputations. Notably, this study validates that proactive behaviors work more effectively in promoting e-consultations for doctors with lower titles or fewer digital recommendations, while reactive behaviors are more effective for doctors with fewer digital recommendations.

This study offers several practical implications for doctors and platform managers. First, the beneficial effects of prosocial behaviors suggest that doctors should adapt their engagement activities when participating in web-based health care platforms. Nowadays, an increasing number of doctors are joining web-based health care communities and focusing on e-consultations, attracted by the economic and social benefits. Based on our results, posting professional articles can help doctors establish a professional image, potentially leading to more e-consultations. Additionally, conducting medical live streams can bolster e-consultations by fostering cooperative social value for doctors and enhancing their credibility among patient audiences. Therefore, doctors may prefer engaging in both proactive and reactive prosocial activities in web-based health care communities to attract more patients to their e-consultation services.

Second, the boundary conditions of the effects of prosocial behaviors imply that doctors should strategically leverage the beneficial effect of proactive and reactive behaviors according to their offline and digital reputations. Doctors with fewer digital recommendations should focus more on prosocial behavior to attract patients to e-consultations. Meanwhile, doctors with lower titles should devote their efforts to proactive behaviors to demonstrate their capability in fulfilling the e-consultations, thereby reducing information asymmetry between patients and themselves.

Third, our findings offer implications for web-based health care platform managers in designing effective functions. An increasing number of platforms are launching various features to better serve doctors and patients, meeting the needs of both groups more effectively. Our empirical findings suggest that doctors’ proactive and reactive prosocial behaviors, such as posting professional articles and conducting medical live streams, can help them establish professional image and enhance patient trust, leading to improved performance. Importantly, these behaviors also benefit patients by enhancing their health knowledge and literacy. Thus, platform managers could introduce functions (eg, article posting, live streaming, and doctor-driven communities) to encourage more prosocial behaviors by doctors. Additionally, platform managers might consider incorporating guidelines or incentive mechanisms for prosocial behaviors into their platforms. For example, it is recommended that platforms collect and analyze doctors’ proactive and reactive prosocial behaviors and guide them on how to effectively use these functions and engage in different types of activities.

Limitations

Despite its contributions, this study also presents several limitations that future research should consider. First, various classifications of prosocial behavior are available; for instance, Richaud et al [ 71 ] classified such behavior as altruistic, compliant, emotional, public, anonymous, or dire actions. Given the intricacy of web-based medical services, future studies would benefit from further exploring the roles of these other types of prosocial behavior exhibited by doctors on web-based health care communities. Second, our research model was constructed primarily from the doctor’s perspective and thus did not investigate the influence of doctors’ prosocial behaviors on patients’ satisfaction and well-being. Future research should delve into these relationships to obtain a more comprehensive understanding of the impacts of doctors’ prosocial behaviors. Finally, this study focused only on the quantity of medical live-streaming sessions, overlooking the quality aspect, which could be a crucial factor influencing e-consultation volume. Future research will concentrate on exploring this aspect.

Conclusions

Building upon prior studies on doctors’ prosocial behaviors on web-based health care communities, this study further delineates doctors’ beneficial actions into proactive and synchronous reactive behaviors. This distinction is based on the divergence in doctors’ motives for engaging and patients’ levels of involvement. Drawing from FMT and SET, this study offers insights that could aid doctors in increasing their e-consultation volume by adopting these beneficial behaviors. Concurrently, this research augments our understanding of the roles a doctor’s reputation plays in the relationships between various prosocial behaviors—specifically, proactive and reactive actions—and their e-consultation volume. This study may inspire doctors with comparatively lower offline professional titles and digital popularity to achieve their desired e-consultation volume.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (72001170 and 72102179), the Fundamental Research Funds for the Central Universities (SK2024028), the Ministry of Education in China Project of Humanities and Social Sciences (21XJC630003), the China Postdoctoral Science Foundation (2022T150515, 2023M742818, and 2020M673432), the National Natural Science Foundation of China (72004042), and the Heilongjiang Natural Science Foundation (YQ2023G003), and the grants from City University of Hong Kong (projects 7005959, 7006152, and 7200725).

Conflicts of Interest

None declared.

• Qi M, Cui J, Li X, Han Y. Influence of e-consultation on the intention of first-visit patients to select medical services: results of a scenario survey. J Med Internet Res. 2023;25:e40993. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cao B, Huang W, Chao N, Yang G, Luo N. Patient activeness during online medical consultation in China: multilevel analysis. J Med Internet Res. 2022;24(5):e35557. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang S. Talk to your doctors online: an internet-based intervention in China. Health Commun. 2021;36(4):405-411. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang J, Cameron AF, Yang M. Analysis of massive online medical consultation service data to understand physicians' economic return: observational data mining study. JMIR Med Inform. 2020;8(2):e16765. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Street RL, Millay B. Analyzing patient participation in medical encounters. Health Commun. 2001;13(1):61-73. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ren D, Ma B. Effectiveness of interactive tools in online health care communities: social exchange theory perspective. J Med Internet Res. 2021;23(3):e21892. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guo S, Guo X, Fang Y, Vogel D. How doctors gain social and economic returns in online health-care communities: a professional capital perspective. J Manag Inf Syst. 2017;34(2):487-519. [ FREE Full text ] [ CrossRef ]
• Tande AJ, Berbari EF, Ramar P, Ponamgi SP, Sharma U, Philpot L, et al. Association of a remotely offered infectious diseases eConsult service with improved clinical outcomes. Open Forum Infect Dis. 2020;7(1):ofaa003. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Richardson BR, Truter P, Blumke R, Russell TG. Physiotherapy assessment and diagnosis of musculoskeletal disorders of the knee via telerehabilitation. J Telemed Telecare. 2017;23(1):88-95. [ CrossRef ] [ Medline ]
• Castaneda PR, Duffy B, Andraska EA, Stevens J, Reschke K, Osborne N, et al. Outcomes and safety of electronic consult use in vascular surgery. J Vasc Surg. 2020;71(5):1726-1732. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang S. The relationship between face-to-face and online patient-provider communication: examining the moderating roles of patient trust and patient satisfaction. Health Commun. 2020;35(3):341-349. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li J, Tang J, Jiang L, Yen DC, Liu X. Economic success of physicians in the online consultation market: a signaling theory perspective. Int J Electron Commer. 2019;23(2):244-271. [ FREE Full text ] [ CrossRef ]
• Guo S, Guo X, Zhang X, Vogel D. Doctor–patient relationship strength’s impact in an online healthcare community. Inf Technol Dev. 2017;24(2):279-300. [ FREE Full text ] [ CrossRef ]
• Audrain-Pontevia AF, Menvielle L. Do online health communities enhance patient-physician relationship? an assessment of the impact of social support and patient empowerment. Health Serv Manage Res. 2018;31(3):154-162. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yang Y, Zhang X, Lee PK. Improving the effectiveness of online healthcare platforms: an empirical study with multi-period patient-doctor consultation data. Int J Prod Econ. 2019;207:70-80. [ FREE Full text ] [ CrossRef ]
• Fan W, Zhou Q, Qiu L, Kumar S. Should doctors open online consultation services? An empirical investigation of their impact on offline appointments. Inf Syst Res. 2023;34(2):629-651. [ FREE Full text ] [ CrossRef ]
• Liu QB, Liu X, Guo X. The effects of participating in a physician-driven online health community in managing chronic disease: evidence from two natural experiments. MIS Q. 2020;44(1):391-419. [ FREE Full text ] [ CrossRef ]
• Yan Z, Wang T, Chen Y, Zhang H. Knowledge sharing in online health communities: a social exchange theory perspective. Inf Manag. 2016;53(5):643-653. [ FREE Full text ] [ CrossRef ]
• Luo P, Chen K, Wu C, Li Y. Exploring the social influence of multichannel access in an online health community. J Assoc Inf Sci Technol. 2018;69(1):98-109. [ FREE Full text ] [ CrossRef ]
• Chai S, Bagchi-Sen S, Morrell C, Rao HR, Upadhyaya SJ. Internet and online information privacy: an exploratory study of preteens and early teens. IEEE Trans Profess Commun. 2009;52(2):167-182. [ FREE Full text ] [ CrossRef ]
• Ahearne M, Rapp A, Hughes DE, Jindal R. Managing sales force product perceptions and control systems in the success of new product introductions. J Mark Res. 2010;47(4):764-776. [ FREE Full text ] [ CrossRef ]
• Awaysheh A, Heron RA, Perry T, Wilson JI. On the relation between corporate social responsibility and financial performance. Strateg Manag J. 2020;41(6):965-987. [ FREE Full text ] [ CrossRef ]
• Coll MP, Grégoire M, Eugène F, Jackson PL. Neural correlates of prosocial behavior towards persons in pain in healthcare providers. Biol Psychol. 2017;128:1-10. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kapıkıran NA. Sources of ethnocultural empathy: personality, intergroup relations, affects. Curr Psychol. 2021;42(14):11510-11528. [ FREE Full text ] [ CrossRef ]
• Yin Y, Wang Y. Is empathy associated with more prosocial behaviour? a meta‐analysis. Asian J Soc Psychol. 2023;26(1):3-22. [ FREE Full text ] [ CrossRef ]
• Finset A. "I am worried, Doctor!" emotions in the doctor-patient relationship. Patient Educ Couns. 2012;88(3):359-363. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Larson EB, Yao X. Clinical empathy as emotional labor in the patient-physician relationship. JAMA. 2005;293(9):1100-1106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bloom G, Standing H, Lloyd R. Markets, information asymmetry and health care: towards new social contracts. Soc Sci Med. 2008;66(10):2076-2087. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guo F, Zou B, Guo J, Shi Y, Bo Q, Shi L. What determines academic entrepreneurship success? A social identity perspective. Int Entrep Manag J. 2019;15(3):929-952. [ FREE Full text ] [ CrossRef ]
• Yang H, Zhang X. Investigating the effect of paid and free feedback about physicians' telemedicine services on patients' and physicians' behaviors: panel data analysis. J Med Internet Res. 2019;21(3):e12156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yan Z, Kuang L, Qiu L. Prosocial behaviors and economic performance: evidence from an online mental healthcare platform. Prod Oper Manag. 2022;31(10):3859-3876. [ FREE Full text ] [ CrossRef ]
• Frese M, Fay D. 4. Personal initiative: an active performance concept for work in the 21st century. Res Organ Behav. 2001;23:133-187. [ FREE Full text ] [ CrossRef ]
• Frese M, Kring W, Soose A, Zempel J. Personal initiative at work: differences between East and West Germany. Acad Manage J. 1996;39(1):37-63. [ FREE Full text ] [ CrossRef ]
• Spitzmuller M, Van Dyne L. Proactive and reactive helping: contrasting the positive consequences of different forms of helping. J Organ Behavior. 2013;34(4):560-580. [ FREE Full text ] [ CrossRef ]
• Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol. 2001;52(1):1-26. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mittal S, Sengupta A, Agrawal NM, Gupta S. How prosocial is proactive: developing and validating a scale and process model of knowledge-based proactive helping. J Manag Organ. 2018;26(4):625-650. [ FREE Full text ] [ CrossRef ]
• Walter A, Ritter T, Gemünden HG. Value creation in buyer–seller relationships: theoretical considerations and empirical results from a supplier's perspective. Ind Mark Manag. 2001;30(4):365-377. [ FREE Full text ] [ CrossRef ]
• Eggert A, Ulaga W, Schultz F. Value creation in the relationship life cycle: a quasi-longitudinal analysis. Ind Mark Manag. 2006;35(1):20-27. [ FREE Full text ] [ CrossRef ]
• Deng Z, Hong Z, Zhang W, Evans R, Chen Y. The effect of online effort and reputation of physicians on patients' choice: 3-wave data analysis of China's good doctor website. J Med Internet Res. 2019;21(3):e10170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kurihara H, Maeno T, Maeno T. Importance of physicians' attire: factors influencing the impression it makes on patients, a cross-sectional study. Asia Pac Fam Med. 2014;13(1):2. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cooper ML, Shapiro CM, Powers AM. Motivations for sex and risky sexual behavior among adolescents and young adults: a functional perspective. J Pers Soc Psychol. 1998;75(6):1528-1558. [ CrossRef ] [ Medline ]
• Clary EG, Snyder M, Ridge RD, Copeland J, Stukas AA, Haugen J, et al. Understanding and assessing the motivations of volunteers: a functional approach. J Pers Soc Psychol. 1998;74(6):1516-1530. [ CrossRef ] [ Medline ]
• Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287(2):226-235. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang L, Qiu Y, Zhang N, Li S. How difficult doctor‒patient relationships impair physicians' work engagement: the roles of prosocial motivation and problem-solving pondering. Psychol Rep. 2020;123(3):885-902. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dabholkar PA, Johnston WJ, Cathey AS. The dynamics of long-term business-to-business exchange relationships. J Acad Mark Sci. 1994;22(2):130-145. [ FREE Full text ] [ CrossRef ]
• Ochs M, Mestre D, de Montcheuil G, Pergandi JM, Saubesty J, Lombardo E, et al. Training doctors’ social skills to break bad news: evaluation of the impact of virtual environment displays on the sense of presence. J Multimodal User Interfaces. 2019;13(1):41-51. [ FREE Full text ] [ CrossRef ]
• Schroeder T, Seaman K, Nguyen A, Gewald H, Georgiou A. Enablers and inhibitors to the adoption of mHealth apps by patients—a qualitative analysis of German doctors' perspectives. Patient Educ Couns. 2023;114:107865. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu X, Hu M, Xiao BS, Shao J. Is my doctor around me? investigating the impact of doctors’ presence on patients’ review behaviors on an online health platform. J Assoc Inf Sci Technol. 2022;73(9):1279-1296. [ FREE Full text ] [ CrossRef ]
• Chen Y, Xie J. Online consumer review: word-of-mouth as a new element of marketing communication mix. Manage Sci. 2008;54(3):477-491. [ FREE Full text ] [ CrossRef ]
• Ye Q, Li Y, Kiang M, Wu W. The impact of seller reputation on the performance of online sales: evidence from TaoBao Buy-It-Now (BIN) data. SIGMIS Database. 2009;40(1):12-19. [ FREE Full text ] [ CrossRef ]
• Liu X, Guo X, Wu H, Wu T. The impact of individual and organizational reputation on physicians’ appointments online. Int J Electron Commer. 2016;20(4):551-577. [ FREE Full text ] [ CrossRef ]
• Dewan S, Hsu V. Adverse selection in electronic markets: evidence from online stamp auctions. J Ind Econ. 2004;52(4):497-516. [ CrossRef ]
• Torres E, Vasquez-Parraga AZ, Barra C. The path of patient loyalty and the role of doctor reputation. Health Mark Q. 2009;26(3):183-197. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gottschalk F, Mimra W, Waibel C. Health services as credence goods: a field experiment. Econ J. 2020;130(629):1346-1383. [ FREE Full text ] [ CrossRef ]
• Nunkoo R, Ramkissoon H. Power, trust, social exchange and community support. Ann Tour Res. 2012;39(2):997-1023. [ FREE Full text ] [ CrossRef ]
• Blau PM. Exchange and Power in Social Life. New Brunswick. Transaction Publishers; 1964.
• Molm LD, Takahashi N, Peterson G. Risk and trust in social exchange: an experimental test of a classical proposition. Am J Sociol. 2000;105(5):1396-1427. [ FREE Full text ] [ CrossRef ]
• Lioukas CS, Reuer JJ. Isolating trust outcomes from exchange relationships: social exchange and learning benefits of prior ties in alliances. Acad Manage J. 2015;58(6):1826-1847. [ FREE Full text ] [ CrossRef ]
• Malgonde OS, Saldanha TJ, Mithas S. Resilience in the open source software community: how pandemic and unemployment shocks influence contributions to others’ and one’s own projects. MIS Q. 2023;47(1):361-390. [ FREE Full text ] [ CrossRef ]
• Mayya R, Ye S, Viswanathan S, Agarwal R. Who forgoes screening in online markets and why? Evidence from Airbnb. MIS Q. 2021;45(4):1745-1776. [ FREE Full text ] [ CrossRef ]
• Moqri M, Mei X, Qiu L, Bandyopadhyay S. Effect of “following” on contributions to open source communities. J Manag Inf Syst. 2018;35(4):1188-1217. [ FREE Full text ] [ CrossRef ]
• Chen Q, Jin J, Zhang T, Yan X. The effects of log-in behaviors and web reviews on patient consultation in online health communities: longitudinal study. J Med Internet Res. 2021;23(6):e25367. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mo J, Sarkar S, Menon S. Competing tasks and task quality: an empirical study of crowdsourcing contests. MIS Q. 2021;45(4):1921-1948. [ FREE Full text ] [ CrossRef ]
• Gourieroux C, Holly A, Monfort A. Likelihood ratio test, Wald test, and Kuhn-Tucker test in linear models with inequality constraints on the regression parameters. Econometrica. 1982;50(1):63. [ CrossRef ]
• Peng CH, Yin D, Zhang H. More than words in medical question-and-answer sites: a content-context congruence perspective. Inf Syst Res. 2020;31(3):913-928. [ FREE Full text ] [ CrossRef ]
• Guo L, Jin B, Yao C, Yang H, Huang D, Wang F. Which doctor to trust: a recommender system for identifying the right doctors. J Med Internet Res. 2016;18(7):e186. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zheng S, Hui SF, Yang Z. Hospital trust or doctor trust? A fuzzy analysis of trust in the health care setting. J Bus Res. 2017;78:217-225. [ FREE Full text ] [ CrossRef ]
• Mao Z, Du Z, Yuan R, Miao Q. Short-term or long-term cooperation between retailer and MCN? new launched products sales strategies in live streaming e-commerce. J Retail Consum Serv. 2022;67:102996. [ FREE Full text ] [ CrossRef ]
• Hilvert-Bruce Z, Neill JT, Sjöblom M, Hamari J. Social motivations of live-streaming viewer engagement on Twitch. Comput Hum Behav. 2018;84:58-67. [ FREE Full text ] [ CrossRef ]
• Tang YM, Chen PC, Law KMY, Wu CH, Lau YY, Guan J, et al. Comparative analysis of student's live online learning readiness during the coronavirus (COVID-19) pandemic in the higher education sector. Comput Educ. 2021;168:104211. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Richaud MC, Mesurado B, Cortada AK. Analysis of dimensions of prosocial behavior in an Argentinean sample of children. Psychol Rep. 2012;111(3):687-696. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by G Eysenbach; submitted 11.09.23; peer-reviewed by P Luo, Y Zhu, C Fu; comments to author 05.10.23; revised version received 30.12.23; accepted 09.03.24; published 25.04.24.

©Xiaoxiao Liu, Huijing Guo, Le Wang, Mingye Hu, Yichan Wei, Fei Liu, Xifu Wang. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

A leader in implementing evidence-based programs

Thursday, Apr 25, 2024 • Cristal Gonzalez : contact

The University of Texas at Arlington has chosen to become a leadership partner with Excelencia in Education, the nation’s premier authority on efforts to accelerate Latino student success in higher education, in releasing a compilation of evidence-based programs that improve higher educational achievement for Latino and other students.

The compilation, “ Advancing What Works to Intentionally Serve Latino Students: Opportunities for Action – 2024 ,” emphasizes cultural responsiveness and asset-based approaches to serving Latinos, as well as other post-traditional students, in five key areas:

• Engagement to Prepare Students and Families in College
• First-Year Experiences Building Strong Foundations for Success
• Bilingualism Enhancing Academic and Career Connections
• Experiences and Partnerships Building Career Pathways
• Gender-Based Learning Communities to Support Success

“Excelencia increases awareness of programs that work for Latino students with the understanding that what serves our students ultimately serves our country,” said Deborah Santiago, co-founder and CEO of Excelencia. “The trailblazing institutions supporting the compilation know asset-based approaches and evidence-based practices are crucial to intentionally serving Latino students and show there is no excuse for inaction.”

UTA is one of 24 colleges and universities across the country supporting the release to inspire and support educators, community leaders, funders, and policymakers to replicate and bring to scale data-informed practices that work for Latinos in higher education.

UTA President Jennifer Cowley belongs to Excelencia’s national network of Presidents for Latino Student Success (P4LSS), composed of over 190 postsecondary leaders from 27 states, Washington, D.C., and Puerto Rico. Together, colleges and universities in this network—which represents only 6% of institutions yet enrolls 33% and graduates 35% of all Latino students—are transforming higher education.

“America’s future is brighter with the talents of Latinos,” said Sarita Brown, co-founder and president of Excelencia. “Excelencia brings national attention to these effective efforts advancing students’ higher education journeys with 24 college and university presidents and chancellors who serve as leaders for Latino student success. By sharing information and supporting replication, we build momentum for institutional transformation that accelerates Latino student success and thus grows our civic leadership and future workforce.

UTA leads Excelencia’s network as one of only 39 colleges and universities to have earned the Seal of Excelencia, a national certification for institutions that strive to go beyond enrollment to intentionally serve Latino students. Seal-certified institutions represent less than 1% of colleges and universities yet enroll 15% and graduate 17% of all Latino students. They also lead the way by implementing evidence-based practices that ensure Latino students thrive on their campuses.

Through this partnership and the P4LSS network, Cowley and UTA will continue collaborating with Excelencia, leveraging collective expertise and resources, fostering working relationships, and amplifying current efforts at the national level to accelerate Latino student success and ensure America’s future with the talent of Latino students.

To learn more about “Advancing What Works to Intentionally Serve Latino Students: Opportunities for Action – 2024,” visit EdExcelencia.org/advancing-what-works-latino-students-2024 .

About Excelencia in Education

In 2024, Excelencia in Education marks 20 years of service to accelerate Latino student success in higher education by promoting Latino student achievement, conducting analysis to inform educational policies, advancing institutional practices, and collaborating with those committed and ready to meet the mission. Launched in 2004 in the nation’s capital, Excelencia has established a network of results-oriented educators and policymakers to address the U.S. economy’s needs for a highly educated workforce and engaged civic leaders. For more information, visit EdExcelencia.org.

About The University of Texas at Arlington

The University of Texas at Arlington, located in the heart of the Dallas-Fort Worth-Arlington metroplex, is a comprehensive teaching, research, and public service institution dedicated to the advancement of knowledge through scholarship and creative work. The University is committed to providing access and ensuring student success, and to a culture of innovation, entrepreneurship, and commercialization of discoveries by our community of scholars. With an enrollment of approximately 41,000 students, UTA is the second-largest institution in the UT System.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts

Latest science news, discoveries and analysis

Could a rare mutation that causes dwarfism also slow ageing?

Bird flu in US cows: is the milk supply safe?

Future of Humanity Institute shuts: what's next for ‘deep future’ research?

Judge dismisses superconductivity physicist’s lawsuit against university

Nih pay raise for postdocs and phd students could have us ripple effect, hello puffins, goodbye belugas: changing arctic fjord hints at our climate future, china's moon atlas is the most detailed ever made, ‘shut up and calculate’: how einstein lost the battle to explain quantum reality, ecologists: don’t lose touch with the joy of fieldwork chris mantegna.

Should the Maldives be creating new land?

Lethal AI weapons are here: how can we control them?

Algorithm ranks peer reviewers by reputation — but critics warn of bias

How gliding marsupials got their ‘wings’

Bird flu virus has been spreading in us cows for months, rna reveals, audio long read: why loneliness is bad for your health, nato is boosting ai and climate research as scientific diplomacy remains on ice, rat neurons repair mouse brains — and restore sense of smell.

Retractions are part of science, but misconduct isn’t — lessons from a superconductivity lab

Any plan to make smoking obsolete is the right step

Citizenship privilege harms science

European ruling linking climate change to human rights could be a game changer — here’s how charlotte e. blattner, will ai accelerate or delay the race to net-zero emissions, current issue.

The Maldives is racing to create new land. Why are so many people concerned?

Surprise hybrid origins of a butterfly species, stripped-envelope supernova light curves argue for central engine activity, optical clocks at sea, research analysis.

Ancient DNA traces family lines and political shifts in the Avar empire

A chemical method for selective labelling of the key amino acid tryptophan

Robust optical clocks promise stable timing in a portable package

Targeting RNA opens therapeutic avenues for Timothy syndrome

Bioengineered ‘mini-colons’ shed light on cancer progression, galaxy found napping in the primordial universe, tumours form without genetic mutations, marsupial genomes reveal how a skin membrane for gliding evolved.

Scientists urged to collect royalties from the ‘magic money tree’

Breaking ice, and helicopter drops: winning photos of working scientists

Shrouded in secrecy: how science is harmed by the bullying and harassment rumour mill

Want to make a difference try working at an environmental non-profit organization, how ground glass might save crops from drought on a caribbean island, books & culture.

How volcanoes shaped our planet — and why we need to be ready for the next big eruption

Dogwhistles, drilling and the roots of Western civilization: Books in brief

Cosmic rentals

Las borinqueñas remembers the forgotten puerto rican women who tested the first pill, dad always mows on summer saturday mornings, nature podcast.

Latest videos

Nature briefing.

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Analysis Suggests Additional Evidence Necessary for Surrogate Markers Used as Endpoints Used to Support FDA Drug Approvals

  By Kelly Jordan

Clinical trials supporting the approval of drugs by the U.S. Food and Drug Administration (FDA) increasingly use surrogate markers, such as imaging findings or laboratory measures, as primary endpoints. Surrogate markers, which offer the advantage of reducing the duration and size of clinical trials, are expected to predict target clinical outcomes of interest that directly measure how people feel, function, or survive. In 2018, the FDA released an Adult Surrogate Endpoint Table of surrogate markers that may be used as primary endpoints in clinical trials that form the basis of FDA’s approval of drugs.

A recent investigation published in JAMA  and led by Joshua Wallach, PhD, assistant professor of epidemiology, looked at the available research to support the use of 37 surrogate markers listed as potential primary endpoints in clinical trials for 32 chronic diseases (excluding cancer) in the FDA’s table. In particular, the authors conducted independent systematic reviews for each surrogate marker to identify meta-analyses of clinical trials quantifying the strength of association between the treatment effects measured using the relevant surrogate markers and any clinical outcome.

What They Found

The researchers’ analysis showed 22 of the 37 (59%) surrogate markers that may be used as endpoints in clinical trials to support FDA approval of drugs treating chronic diseases had no published meta-analyses examining the strength of their association with any clinical outcome. Furthermore, among those with at least one meta-analysis, few reported high-strength evidence of associations between the surrogates and target clinical outcomes.

“The development of the Adult Surrogate Endpoint Table was an important and necessary step for increasing the transparency around the surrogate markers that could be used to inform the approval of new drugs and biologics,” says Wallach. “However, our findings highlight the need for adding a summary of evidence supporting marker surrogacy for clinical benefit, or clarify when no studies are available.”  

Why it Matters

These findings highlight the importance of making publicly available a summary of the evidence supporting surrogate endpoints that may be used to support FDA approval of drugs treating chronic disease.

“Ultimately, this can aid drug sponsors in choosing appropriate surrogate endpoints for trials and guide physicians and their patients in accurately interpreting the clinical benefit of drugs approved according to listed surrogate markers,” says Wallach.

MEDIA CONTACT

Rob Spahr Director of Public Relations, Rollins School of Public Health 470.590.8055 [email protected]

Associated Topics:

• Epidemiology
• Rollins News
• In the Media
• Rollins Experts

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Effective fall prevention exercise in residential aged care: an intervention component analysis from an updated systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-9652-4711 Rik Dawson 1 , 2 ,
• Jenni Suen 3 ,
• http://orcid.org/0000-0001-8934-4368 Catherine Sherrington 1 , 2 ,
• http://orcid.org/0000-0002-6806-3011 Wing Kwok 1 , 2 ,
• http://orcid.org/0000-0001-7459-5105 Marina B Pinheiro 1 , 2 ,
• Abby Haynes 1 , 2 ,
• Charlotte McLennan 1 , 2 ,
• Katy Sutcliffe 4 ,
• Dylan Kneale 4 ,
• Suzanne Dyer 3
• 1 Sydney Musculoskeletal Health , School of Public Health, Faculty of Medicine and Health, The University of Sydney , Sydney , New South Wales , Australia
• 2 Institute for Musculoskeletal Health , Sydney Local Health District , Camperdown , New South Wales , Australia
• 3 Flinders Health and Medical Research Institute , Flinders University , Adelaide , South Australia , Australia
• 4 Social Research Institute , University College London , London , UK
• Correspondence to Rik Dawson, Sydney Musculoskeletal Health, Institute for Musculoskeletal Health, The University of Sydney Faculty of Medicine and Health, Sydney, NSW 2050, Australia; rik.dawson{at}sydney.edu.au

Objective The effect of fall prevention exercise programmes in residential aged care (RAC) is uncertain. This paper reports on an intervention component analysis (ICA) of randomised controlled trials (RCTs), from an update of a Cochrane review, to develop a theory of features of successful fall prevention exercise in RAC.

Methods Trial characteristics were extracted from RCTs testing exercise interventions in RAC identified from an update of a Cochrane review to December 2022 (n=32). Eligible trials included RCTs or cluster RCTs in RAC, focusing on participants aged 65 or older, assessing fall outcomes with stand-alone exercise interventions. ICA was conducted on trials with >30 participants per treatment arm compared with control (n=17). Two authors coded trialists’ perceptions on intervention features that may have contributed to the observed effect on falls. Inductive thematic analysis was used to identify the key differences between the trials which might account for positive and negative outcomes.

Results 32 RCTs involving 3960 residents including people with cognitive (57%) and mobility (41%) impairments were included. ICA on the 17 eligible RCTs informed the development of a theory that (1) effective fall prevention exercise delivers the right exercise by specifically targeting balance and strength, tailored to the individual and delivered simply at a moderate intensity and (2) successful implementation needs to be sufficiently resourced to deliver structured and supervised exercise at an adequate dose.

Conclusions This analysis suggests that delivering the right exercise, sufficiently resourced, is important for preventing falls in RAC. This clinical guidance requires confirmation in larger trials.

• Accidental Falls
• Qualitative Research

Data availability statement

Data are available upon reasonable request. Data are available on reasonable request. Proposals for data should be directed to the corresponding author ([email protected]).

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bjsports-2023-107505

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Exercise reduces the risk of falls for older people in the community with tailored balance and functional exercise having the strongest fall prevention effect.

The evidence concerning the effects of fall prevention exercise in residential aged care (RAC) is less clear, limiting guidance for clinicians working in RAC.

WHAT THIS STUDY ADDS

This intervention component analysis (ICA) used a mixed method approach to develop a theory of potential features of effective fall prevention exercise in RAC that can be explored and validated in future analyses. This theory indicates that (1) exercise interventions for fall prevention in RAC should include balance and strength exercise delivered at a moderate intensity and tailored to the individual and (2) exercise programmes in RAC are more likely to be successful if structured, supervised and resourced to deliver an adequate dose.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Findings from this ICA will assist implementation of fall prevention programmes in RAC; guide a qualitative comparative analysis and subgroup meta-analysis in RAC and inform the design of interventions to be tested in future large clinical trials in RAC.

Introduction

Falls are prevalent in residential aged care (RAC), affecting half of residents annually, diminishing independence, increasing care burden and imposing economic costs. 1 2 Implementing effective interventions holds the potential to benefit older individuals and alleviate healthcare burden. 3

The 2019 Cochrane Review in community-dwelling older adults reported exercise prevents falls, particularly with balance and functional exercises, reducing rates by 24% (rate ratio (RaR) 0.76, 95% CI 0.70 to 0.81; 7920 participants, 39 studies; I 2 =29%, high-certainty evidence). 4 Programmes combining balance, functional and resistance exercises reduced the rate of falls by 34% (RaR 0.66, 95% CI 0.50 to 0.88; 1374 participants, 11 studies; I 2 =65%, moderate-certainty evidence). Conversely, the 2018 Cochrane Review for residents in aged care reported uncertain effects of exercise on falls (RaR=0.93, 95% CI 0.72 to 1.20; 2002 participants, 10 studies; I 2 =76%, very low-quality evidence). 5 Subgroup analyses couldn't explain high outcome heterogeneity, challenging clinicians in selecting effective exercise programmes. 6

In this study, we conducted an intervention component analysis (ICA), a method utilising inductive qualitative analysis to collate trialists’ reflections on the factors influencing the success or failure of an intervention. Our objective is to answer three research questions. (1) What are the characteristics of trials included in the updated Cochrane Review? 5 (2) Using ICA, what are the intervention and implementation features that are present in successful fall prevention exercise programmes in RAC? and (3) What explanatory theory does the information in the first two questions suggest for supporting knowledge translation of fall prevention programmes and informing future research in RAC? 7

This systematic review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. 8 This ICA was developed according to the methodology detailed in an existing published theory. 7

Selection of studies

Criteria followed Cameron et al ’s 2018 Cochrane Review on preventing falls in older care facility residents. 5 New trials from 2017 to December 2022 were sourced from CENTRAL, MEDLINE, Embase and CINAHL databases, adhering to the Cochrane Review’s search methods. 5 Only randomised controlled trials (RCTs) and cluster RCTs with participants aged 65+ in RAC, reporting fall rates or risk and comparing exercise interventions to usual care or control were considered. Trials with ≥30 participants per arm were included to mitigate small sample bias, 9 encompassing outcomes that reduced, had no effect neutral, or increased falls ( figure 1 ).

• Download figure
• Open in new tab
• Download powerpoint

PRISMA flowchart of trial selection. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Data extraction

Trial features.

Two aged care physiotherapists (RD and WK) extracted trial and participant characteristics (trial design, sample size, age, gender, cognitive and mobility status), exercise features (type, duration, dose, delivery, supervision, tailoring, intensity) and fall outcomes from new trial’s final endpoint or the Cochrane Review. 5 Adherence was classified as good if the participant exercise session attendance rate exceeded 50%, and/or if 75% or more of the participants attended 50% or more of the exercise sessions. 10 Exercise classifications adhered to the Prevention of Falls Network Europe (ProFANE) exercise taxonomy 11 and include standardised intensity ratings like the Borg Rating of Perceived Exertion Scale 12 and trialist’s self-classification. Low-intensity exercise refers to gentle physical activity where the heart rate and breathing are low; moderate-intensity exercise elevates heart rate and breathing moderately; while high-intensity exercise elevates heart rate and breathing to near maximal effort, making it difficult to sustain a conversation. 12 The ProFANE group’s exercise descriptors cover structured programmes including strength exercises (resistance training with weights and/or body weight), gait, balance and functional exercises mirroring daily movements (eg, stepping, sit-to-stand), flexibility, general physical activity and 3D exercises like Tai Chi. Disagreements were resolved through discussion and involvement of a third author as required (SD and JS).

Fall outcomes were expressed as rate or risk ratio with 95% CI. They were coded positive (<0.75), neutral (0.75 to 1.25) or negative (>1.25) based on Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) group guidance for consideration of ‘appreciable benefit or harm’. 13 Data analysed in Excel.

Trial quality

Two trained independent physiotherapists (RD, WK) assessed study quality using the Physiotherapy Evidence Database (PEDro) scale, which evaluates 11 criteria: inclusion criteria, random allocation, concealed allocation, baseline similarity, blinding of subjects, therapists and assessors, completeness of follow-up, intention-to-treat analysis, between-group statistical comparisons and outcome measures. 14 Disagreements were resolved through discussion. External validity, the first item, does not contribute to the score, yielding scores from 0 to 10. Ratings: 0–4 (poor), 4–5 (fair) and 6–8 (good). Scores of 9–10 (excellent) are n't feasible in exercise trials due to blinding constraints.

Intervention component analysis

We conducted ICA over four stages:

Authors (RD and WK) described the trial features, quality and classified fall outcomes in Excel. 13

Authors (RD and JS) coded trialists’ reflections on effective and ineffective features in eligible publications’ discussion and conclusion sections, including any additional trial-related documents (eg, protocols, process evaluations) identified through systematic search, pearling and hand searching. A selection of publications was coded independently in duplicate; the remainder were extracted by RD and checked by JS, with disagreements resolved through discussion. Codes were stored in NVivo V.12. 15

RD conducted an inductive thematic analysis in NVivo on included trials, grouping codes to develop themes and subthemes on successful intervention features and implementation. 16 JS and SD reviewed groupings to ensure thematic agreement. Both JS and SD experienced qualitative and falls researchers, respectively, offered critical perspectives to the thematic analysis as non-exercise professionals.

RD reviewed the themes against the trial outcomes and the intervention features to develop a theory regarding the types of intervention features and implementation strategies most likely to be effective in preventing falls in RAC. All authors were involved with developing the final theory.

Equity, diversity and inclusion statement

The author group consists of junior, mid-career and senior researchers from different countries and disciplines, most of whom are women. Our study population included both male and female older adults from different socioeconomic and geographical backgrounds.

Trial and participant characteristics from an updated systematic review

Thirty-two trials were included across 16 countries involving 3960 participants with 35 exercise intervention arms. Many trials have small sample sizes, with a median number of 71 participants and a range from 16 17 to 682 participants. 18 Participant ages ranged from 68 19 to 92 years, 20 21 with a median of 82 years; 74% were women. Eighteen (57%) trials included people with cognitive impairment and 13 (41%) trials included those with mobility disabilities. Trial length varied from 4 weeks 22 to 104 weeks, 23 with a median of 12 weeks (IQR 12–26). Total intervention hours ranged from 1.5 hours 24 to 112 hours, 25 with a median trial dose of 22 hours (IQR 14–36) or 1.8 hours per week, comprising a median of three sessions per week (IQR 2–3) with a median session duration of 33 min (IQR 25–48). Sixteen (52%) trials met good exercise adherence criteria. Ten (32%) trials reported adverse events, including two falls during exercise. 26 27 Most trials compared an exercise intervention with usual care, defined as seated low-intensity range of movement exercise programmes, self-directed or social programmes ( online supplemental material A ).

Supplemental material

Intervention components.

The most common exercise intervention was gait, balance and functional training combined with strength training in 18 (51%) intervention arms. The most common single intervention was gait, balance and functional training in five (14%) arms. In 20 (63%) trials, exercise interventions were tailored to the individual. Fifteen (47%) trials delivered exercise at moderate intensity, 14 (44%) at low intensity and three (9%) at high intensity. Thirteen (41%) trials were led by physiotherapists, three (9%) by other health professionals and trained non-exercise professionals led 14 (44%) trials; five trials (16%) did not report who led the intervention. There was a mix of supervised and unsupervised individual and group exercise, with supervised groups being the most common in 19 trials (59%) with a median of five participants per group (IQR 5–8) ( online supplemental material B )

Quality assessment

The PEDro study quality summary is displayed in online supplemental material C . The median PEDro quality score for the trials included in the ICA was ‘good’ (6/10).

The ICA included 17 trials involving 3293 participants with a median age of 82 years (ranging from 75 to 86 years); 73% were female participants. Ten trials (59%) included people with cognitive impairment and seven trials (41%) included people with mobility disability ( table 1 ). The participant characteristics of the ICA trials reflected the 32 trials included in the updated Cochrane review. ICA identified two major themes related to exercise features and implementation, each with three subthemes ( table 2 ). ICA Codebook outlines the coding framework, which is displayed in online supplemental material D . The correspondence between the themes and the trial’s effectiveness synthesis are summarised in table 3 . The ICA also identified some study design features that could be associated with intervention effects.

• View inline

Summary characteristics of 17 ICA trials at inclusion

Development of intervention and implementation themes in the ICA

Presence of themes and subthemes in ICA trials (effectiveness synthesis)

The right exercise

The most common theme supported by the trialist’s commentary focused on providing the right exercise to reduce falls. The results of our thematic analysis suggest that the right exercise is a combination of exercise that targets balance and strength, tailored to the individual’s physical and cognitive comorbidities and delivers moderate-intensity exercise ( table 2 ). Ten trials targeted progressive standing balance and strength exercise, 27–36 12 studies delivered tailored exercise prescription 18 27–35 37 38 and 6 studies delivered moderate-intensity exercise subtheme 27–32 ( table 2 ). Six of eight effective trials 27–32 supported the right exercise theme, while zero out of nine neutral or negative trials did not ( table 3 ).

Supporting exercise engagement

The second theme centred on supporting exercise engagement . However, there was less trialist commentary on this theme ( table 2 ) and only two of eight effective trials included all the subthemes. 27 31 Only the first subtheme had sufficient evidence to progress to the ICA theory ( table 3 ). Four of eight effective trials provided sufficient resourcing as defined as trials that delivered structured and supervised exercise at a dose greater than 30 hours 27 28 30 32 compared with two of nine neutral or negative trials. 34 38 Group exercise that offers socialisation opportunities were offered in five of eight effective trials 27 28 30–32 but was also offered in four of nine neutral or ineffective trials. 33 34 36 38 Fall prevention education was a feature in three of eight effective trials 27 29 31 and one of nine neutral or negative trials. 23 The effectiveness synthesis demonstrated that there was insufficient evidence to involve the group exercise and fall education subthemes in the final ICA theory displayed in figure 2 .

ICA theory of effective fall prevention exercise in residential aged care. ICA, intervention component analysis.

Trial methodological design features

Two features emerged as themes relating to trial design and their impact on effective fall prevention trials. Seven authors commented that some trials are too small to detect significant changes in falls. 23 28 30 33 34 36 38 Five authors commented that trial designs that involved an active control diluted the fall effect. 30–33 36

ICA is a valuable tool for researchers, clinicians and policymakers, enabling the identification of promising intervention components and their implementation strategies. Through ICA methodology, trialist perspectives are plotted against trial outcomes, culminating in the development a theory of effectiveness that can be explored and validated though subsequent analyses. This ICA theory indicates that effective fall prevention programmes within RAC centre on the delivery of the right exercise when sufficiently resourced . However, larger trials are required to expand the evidence base for effective fall prevention exercise programmes in RAC.

In this ICA, we propose that right exercise is tailored, moderate intensity, balance and strength exercise. In general, this is in accordance with the evidence for effective exercise programmes in the community. 4 However, Sherrington and colleagues’ subgroup analysis revealed that balance and functional exercises with or without resistance exercises had the greatest fall prevention effects in the community. 4 , 5 Cameron and colleagues’ subgroup analysis for combination exercises in RAC, including balance and strength exercise 30 32 33 35 38 or physical activity plus strength, 17 did not find a significant reduction in falls (RaR 0.94, 95% CI 0.6 to 1.47); however, there were only six trials in this subgroup analysis. 5 Our ICA was based on an update of Cameron et al ’s Cochrane review, which reported that balance and strength exercises were the most prescribed interventions in effective trials (88%) compared with those with neutral or negative fall outcomes (33%). Becker and colleagues’ multifactorial trial in RAC also supports this finding. They reported that progressive standing balance and strength exercises reduced the rate of falls by 45%. 39

Our review of trial characteristics revealed that 20 out of the 32 trials were tailored to the individual. In the ICA, 88% of positive trials tailored the exercise programmes compared with 56% of trials with neutral or negative fall outcomes. Several trialists stated that exercise needs to be tailored throughout the programme to cater for participants’ changing physical and cognitive capabilities to learn the programme and to maintain exercise adherence, which is a key factor in reducing falls. 33 35 38 40 Contrary to this finding, a recent review of tailored exercise delivered in the community found no significant reduction in fall outcomes. 41

Six effective trials delivered exercise at a moderate intensity 27–32 compared with two effective trials that delivered exercise at a low intensity. 37 40 Among the trials with neutral or negative fall outcomes, both low or high intensity models were tested. Kerse et al revealed that low-intensity exercise delivered in small doses throughout the day by care workers as part of the resident’s usual activities was ineffective. 18 Faber et al reported that high-intensity individual training was neither effective nor sustainable for long-term exercise due to the resident’s high level of comorbidities. 38 Nowalk, Yokoi and Sakamoto and colleagues all commented that successful exercise interventions should be delivered in a way that is simple and easy to learn to maximise exercise compliance in RAC and increase exercise intensity over time. 23 40 42

Trialists provided limited comments on ideal implementation strategies. However, the ICA suggested the importance of adequate resources to support exercise structure, supervision and dosage. Analysis of trial characteristics revealed a median weekly dose of 1.8 hours across 32 trials from this updated Cochrane search, significantly lower than the 3 hours/week recommended by Sherrington et al ’s meta-regression for effective community-based fall prevention. 43 Kerse et al emphasised the need for funding to support more intensive interventions and supervision, 18 while Kovacs et al stressed the importance of sufficient physiotherapy resources for effective fall prevention exercise. 32 More research is required to better understand the implementation of an ideal fall prevention exercise programme in RAC. However, there is likely a higher need for supervision in this setting, given the high level of frailty and comorbidities in this population.

This ICA suggests future RCTs on fall prevention exercises in RAC should increase their sample size, enhance study quality by reducing bias in the design and improve reporting. A total PEDro score of 8 is optimal for multifaceted interventions like exercise trials, contrasting with the median trial score of 6. 44 In the 32 trials identified in the Cochrane review update, 15 enrolled fewer than 60 participants, requiring a larger sample size to detect differences in fall rates. 9 Additionally, many trials did not meet the standards of the Consolidated Standards of Reporting Trials (CONSORT) statement. 45 The reporting of almost two-thirds of trials did not describe allocation concealment, with some not clearly describing their control group. This poor reporting made coding exercise features difficult and reduces the generalisability of this ICA.

Hewitt et al ’s Sunbeam trial achieved an optimal PEDro score of 8, 27 demonstrating effective fall prevention through a physiotherapy-led tailored, progressive standing balance and strength exercises. With a 55% fall rate reduction and improved mobility outcomes, the trial implemented exercise engagement supports detailed in our thematic analysis ( table 3 ). It provided a mean exercise dose of 36 hours, employing user-friendly electronic equipment, fostering high programme adherence via staff and resident education and supervised group exercise for socialisation. Similarly, the Otago Exercise Program, known for its balance and strength focus, demonstrated effectiveness in community settings and merits investigation in RAC. 46

Research implications

The ICA theory will inform a qualitative comparative analysis (QCA), which examines the ICA theory’s consistency and identifies conditions contributing to effective falls prevention programmes. 7 While both ICA and QCA facilitate identifying processes and potential mechanisms that link intervention features and outcomes, we acknowledge that causation cannot be definitively established using probabilistic or counterfactual accounts using these approaches. A recent study employing ICA and QCA methodologies on multifactorial fall prevention interventions highlighted the importance of incorporating exercise and engaging aged care staff and managers in implementing tailored strategies in RAC. 47 These findings emphasise the need for future research to explore multifactorial interventions in RAC settings, explore specific programme needs for recurrent fallers who may have different needs compared with single fallers and in other settings to enhance the generalisability of our ICA and bolster the quality of evidence.

To enhance reporting, future trials should adhere to recognised trial reporting guidelines such as the CONSORT statement, 45 Standard Protocol Items: Recommendations for Interventional Trials statement, 48 Template for Intervention Description and Replication checklist 49 and Consensus on Exercise Reporting guidelines 50 to enhance reporting.

Strengths and Limitations

This updated systematic review, incorporating ICA, ensures rigorous methodology. Nonetheless, limitations include the exclusion of conference abstracts, trial records and studies on exercise within multifactorial interventions. Some trials excluded older adults with high cognitive and mobility impairments, potentially limiting result generalisability. Additionally, exercise descriptions were often broad, lacking specificity like sensorimotor training. Incomplete reporting may have compromised ICA quality, hindering a more comprehensive explanatory theory.

Examining trial characteristics in the updated Cochrane Review on fall prevention exercise in RAC reveals intervention heterogeneity. This ICA enriches trial descriptors, aiding theory development and practical applications. Trialists suggest sufficiently resourced, tailored balance and strength exercises delivered at moderate intensity may prevent falls. Future larger trials should scrutinise this ICA theory, delivery mode, dose, different implementation and engagement strategies, and adhere to reporting guidelines.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

• Quigley PA ,
• Campbell RR ,
• Bulat T , et al
• Goodall S , et al
• Sherrington C ,
• Fairhall NJ ,
• Wallbank GK , et al
• Cameron ID ,
• Panagoda CE , et al
• Higgins JPT ,
• Thompson SG ,
• Deeks JJ , et al
• Sutcliffe K ,
• Stokes G , et al
• McKenzie JE ,
• Bossuyt PM , et al
• Dimairo M ,
• Shephard N , et al
• Fairhall N ,
• Wallbank G , et al
• Jørstad‐Stein EC ,
• Hauer K , et al
• Guyatt GH ,
• Kunz R , et al
• Moseley AM ,
• Wells GA , et al
• Schoenfelder DP
• Robinson E , et al
• da Silva Borges EG ,
• de Souza Vale RG ,
• Cader SA , et al
• Cadore EL ,
• Casas-Herrero A ,
• Zambom-Ferraresi F , et al
• Serra‐Rexach JA ,
• Bustamante‐Ara N ,
• Hierro Villarán M , et al
• Sihvonen S ,
• Taskinen S , et al
• Nowalk MP ,
• Prendergast JM ,
• Bayles CM , et al
• Buckinx F ,
• Beaudart C ,
• Maquet D , et al
• Cancela JM ,
• Seijo-Martinez M , et al
• Saravanakumar P ,
• Higgins IJ ,
• Van Der Riet PJ , et al
• Goodall S ,
• Clemson L , et al
• Arrieta H ,
• Rezola‐Pardo C ,
• Gil SM , et al
• Dhargave P ,
• Sendhilkumar R ,
• James TT , et al
• Ozdemir RA ,
• Evin R , et al
• Jahanpeyma P ,
• Kayhan Koçak FÖ ,
• Yıldırım Y , et al
• Sztruhár Jónásné I ,
• Karóczi CK , et al
• Rosendahl E ,
• Gustafson Y ,
• Nordin E , et al
• Wiklund R ,
• Littbrand H , et al
• Sitjà-Rabert M ,
• Martínez-Zapata MJ ,
• Fort Vanmeerhaeghe A , et al
• Tung AK , et al
• Bosscher RJ ,
• Chin A Paw MJ , et al
• Lindemann U , et al
• Yoshimasu K ,
• Takemura S , et al
• Hunter SW ,
• Batchelor FA , et al
• Sakamoto K ,
• Nakamura T ,
• Hagino H , et al
• Michaleff ZA ,
• Fairhall N , et al
• Cashin AG ,
• Schulz KF ,
• Altman DG ,
• Moher D , et al
• Robertson MC ,
• Campbell AJ ,
• Gardner MM , et al
• Sutcliffe K , et al
• Tetzlaff JM ,
• Altman DG , et al
• Hoffmann TC ,
• Glasziou PP ,
• Boutron I , et al
• Dionne CE ,
• Underwood M , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2
• Data supplement 3
• Data supplement 4

X @dawson_rik, @CathieSherr, @venisakwok, @mabpinheiro, @SuzanneMDyer

Contributors All authors conceptualised and designed the study and were involved in applying the ICA methodology in this study. Analysis of data was undertaken by RD, WSK, JS and SD. RD drafted the manuscript. RD is the guarantor and has access to the data, accepts full responsibility for the conduct of the study and controlled the decision to publish. All authors critically revised the manuscript for intellectual content, approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Funding The National Health and Medical Research Council (NHMRC) of Australia funded Centre for Research Excellence – Prevention of Fall related Injuries provides RD and WK with PhD scholarship funding, and JS and SD partial salary funding. MP receives salary funding from an NHMRC Fellowship.

Disclaimer This manuscript is based on Chapter Three of RD’s thesis by publication called 'Telehealth Physiotherapy to Deliver Exercise in Aged Care Settings' which was submitted to the University of Sydney 14 March 2024. The funding organisations listed above did not have a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer-reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

What the data says about crime in the U.S.

A growing share of Americans say reducing crime should be a top priority for the president and Congress to address this year. Around six-in-ten U.S. adults (58%) hold that view today, up from 47% at the beginning of Joe Biden’s presidency in 2021.

We conducted this analysis to learn more about U.S. crime patterns and how those patterns have changed over time.

The analysis relies on statistics published by the FBI, which we accessed through the Crime Data Explorer , and the Bureau of Justice Statistics (BJS), which we accessed through the  National Crime Victimization Survey data analysis tool .

To measure public attitudes about crime in the U.S., we relied on survey data from Pew Research Center and Gallup.

Additional details about each data source, including survey methodologies, are available by following the links in the text of this analysis.

With the issue likely to come up in this year’s presidential election, here’s what we know about crime in the United States, based on the latest available data from the federal government and other sources.

How much crime is there in the U.S.?

It’s difficult to say for certain. The  two primary sources of government crime statistics  – the Federal Bureau of Investigation (FBI) and the Bureau of Justice Statistics (BJS) – paint an incomplete picture.

The FBI publishes  annual data  on crimes that have been reported to law enforcement, but not crimes that haven’t been reported. Historically, the FBI has also only published statistics about a handful of specific violent and property crimes, but not many other types of crime, such as drug crime. And while the FBI’s data is based on information from thousands of federal, state, county, city and other police departments, not all law enforcement agencies participate every year. In 2022, the most recent full year with available statistics, the FBI received data from 83% of participating agencies .

BJS, for its part, tracks crime by fielding a  large annual survey of Americans ages 12 and older and asking them whether they were the victim of certain types of crime in the past six months. One advantage of this approach is that it captures both reported and unreported crimes. But the BJS survey has limitations of its own. Like the FBI, it focuses mainly on a handful of violent and property crimes. And since the BJS data is based on after-the-fact interviews with crime victims, it cannot provide information about one especially high-profile type of offense: murder.

All those caveats aside, looking at the FBI and BJS statistics side-by-side  does  give researchers a good picture of U.S. violent and property crime rates and how they have changed over time. In addition, the FBI is transitioning to a new data collection system – known as the National Incident-Based Reporting System – that eventually will provide national information on a much larger set of crimes , as well as details such as the time and place they occur and the types of weapons involved, if applicable.

Which kinds of crime are most and least common?

Property crime in the U.S. is much more common than violent crime. In 2022, the FBI reported a total of 1,954.4 property crimes per 100,000 people, compared with 380.7 violent crimes per 100,000 people.  

By far the most common form of property crime in 2022 was larceny/theft, followed by motor vehicle theft and burglary. Among violent crimes, aggravated assault was the most common offense, followed by robbery, rape, and murder/nonnegligent manslaughter.

BJS tracks a slightly different set of offenses from the FBI, but it finds the same overall patterns, with theft the most common form of property crime in 2022 and assault the most common form of violent crime.

How have crime rates in the U.S. changed over time?

Both the FBI and BJS data show dramatic declines in U.S. violent and property crime rates since the early 1990s, when crime spiked across much of the nation.

Using the FBI data, the violent crime rate fell 49% between 1993 and 2022, with large decreases in the rates of robbery (-74%), aggravated assault (-39%) and murder/nonnegligent manslaughter (-34%). It’s not possible to calculate the change in the rape rate during this period because the FBI  revised its definition of the offense in 2013 .

The FBI data also shows a 59% reduction in the U.S. property crime rate between 1993 and 2022, with big declines in the rates of burglary (-75%), larceny/theft (-54%) and motor vehicle theft (-53%).

Using the BJS statistics, the declines in the violent and property crime rates are even steeper than those captured in the FBI data. Per BJS, the U.S. violent and property crime rates each fell 71% between 1993 and 2022.

While crime rates have fallen sharply over the long term, the decline hasn’t always been steady. There have been notable increases in certain kinds of crime in some years, including recently.

In 2020, for example, the U.S. murder rate saw its largest single-year increase on record – and by 2022, it remained considerably higher than before the coronavirus pandemic. Preliminary data for 2023, however, suggests that the murder rate fell substantially last year .

How do Americans perceive crime in their country?

Americans tend to believe crime is up, even when official data shows it is down.

In 23 of 27 Gallup surveys conducted since 1993 , at least 60% of U.S. adults have said there is more crime nationally than there was the year before, despite the downward trend in crime rates during most of that period.

While perceptions of rising crime at the national level are common, fewer Americans believe crime is up in their own communities. In every Gallup crime survey since the 1990s, Americans have been much less likely to say crime is up in their area than to say the same about crime nationally.

Public attitudes about crime differ widely by Americans’ party affiliation, race and ethnicity, and other factors . For example, Republicans and Republican-leaning independents are much more likely than Democrats and Democratic leaners to say reducing crime should be a top priority for the president and Congress this year (68% vs. 47%), according to a recent Pew Research Center survey.

How does crime in the U.S. differ by demographic characteristics?

Some groups of Americans are more likely than others to be victims of crime. In the  2022 BJS survey , for example, younger people and those with lower incomes were far more likely to report being the victim of a violent crime than older and higher-income people.

There were no major differences in violent crime victimization rates between male and female respondents or between those who identified as White, Black or Hispanic. But the victimization rate among Asian Americans (a category that includes Native Hawaiians and other Pacific Islanders) was substantially lower than among other racial and ethnic groups.

The same BJS survey asks victims about the demographic characteristics of the offenders in the incidents they experienced.

In 2022, those who are male, younger people and those who are Black accounted for considerably larger shares of perceived offenders in violent incidents than their respective shares of the U.S. population. Men, for instance, accounted for 79% of perceived offenders in violent incidents, compared with 49% of the nation’s 12-and-older population that year. Black Americans accounted for 25% of perceived offenders in violent incidents, about twice their share of the 12-and-older population (12%).

As with all surveys, however, there are several potential sources of error, including the possibility that crime victims’ perceptions about offenders are incorrect.

How does crime in the U.S. differ geographically?

There are big geographic differences in violent and property crime rates.

For example, in 2022, there were more than 700 violent crimes per 100,000 residents in New Mexico and Alaska. That compares with fewer than 200 per 100,000 people in Rhode Island, Connecticut, New Hampshire and Maine, according to the FBI.

The FBI notes that various factors might influence an area’s crime rate, including its population density and economic conditions.

What percentage of crimes are reported to police? What percentage are solved?

Most violent and property crimes in the U.S. are not reported to police, and most of the crimes that  are  reported are not solved.

In its annual survey, BJS asks crime victims whether they reported their crime to police. It found that in 2022, only 41.5% of violent crimes and 31.8% of household property crimes were reported to authorities. BJS notes that there are many reasons why crime might not be reported, including fear of reprisal or of “getting the offender in trouble,” a feeling that police “would not or could not do anything to help,” or a belief that the crime is “a personal issue or too trivial to report.”

Most of the crimes that are reported to police, meanwhile,  are not solved , at least based on an FBI measure known as the clearance rate . That’s the share of cases each year that are closed, or “cleared,” through the arrest, charging and referral of a suspect for prosecution, or due to “exceptional” circumstances such as the death of a suspect or a victim’s refusal to cooperate with a prosecution. In 2022, police nationwide cleared 36.7% of violent crimes that were reported to them and 12.1% of the property crimes that came to their attention.

Which crimes are most likely to be reported to police? Which are most likely to be solved?

Around eight-in-ten motor vehicle thefts (80.9%) were reported to police in 2022, making them by far the most commonly reported property crime tracked by BJS. Household burglaries and trespassing offenses were reported to police at much lower rates (44.9% and 41.2%, respectively), while personal theft/larceny and other types of theft were only reported around a quarter of the time.

Among violent crimes – excluding homicide, which BJS doesn’t track – robbery was the most likely to be reported to law enforcement in 2022 (64.0%). It was followed by aggravated assault (49.9%), simple assault (36.8%) and rape/sexual assault (21.4%).

The list of crimes  cleared  by police in 2022 looks different from the list of crimes reported. Law enforcement officers were generally much more likely to solve violent crimes than property crimes, according to the FBI.

The most frequently solved violent crime tends to be homicide. Police cleared around half of murders and nonnegligent manslaughters (52.3%) in 2022. The clearance rates were lower for aggravated assault (41.4%), rape (26.1%) and robbery (23.2%).

When it comes to property crime, law enforcement agencies cleared 13.0% of burglaries, 12.4% of larcenies/thefts and 9.3% of motor vehicle thefts in 2022.

Are police solving more or fewer crimes than they used to?

Nationwide clearance rates for both violent and property crime are at their lowest levels since at least 1993, the FBI data shows.

Police cleared a little over a third (36.7%) of the violent crimes that came to their attention in 2022, down from nearly half (48.1%) as recently as 2013. During the same period, there were decreases for each of the four types of violent crime the FBI tracks:

• Police cleared 52.3% of reported murders and nonnegligent homicides in 2022, down from 64.1% in 2013.
• They cleared 41.4% of aggravated assaults, down from 57.7%.
• They cleared 26.1% of rapes, down from 40.6%.
• They cleared 23.2% of robberies, down from 29.4%.

The pattern is less pronounced for property crime. Overall, law enforcement agencies cleared 12.1% of reported property crimes in 2022, down from 19.7% in 2013. The clearance rate for burglary didn’t change much, but it fell for larceny/theft (to 12.4% in 2022 from 22.4% in 2013) and motor vehicle theft (to 9.3% from 14.2%).

Note: This is an update of a post originally published on Nov. 20, 2020.

• Criminal Justice

John Gramlich is an associate director at Pew Research Center

8 facts about Black Lives Matter

#blacklivesmatter turns 10, support for the black lives matter movement has dropped considerably from its peak in 2020, fewer than 1% of federal criminal defendants were acquitted in 2022, before release of video showing tyre nichols’ beating, public views of police conduct had improved modestly, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy