evaluate the reliability and validity of own literature review

Literature Review: Measures for Validity

According to Brown (2006) there are five criteria for the evaluation of the validity of literature review: purpose, scope, authority, audience and format. Accordingly, each of these criteria have been taken into account and appropriately addressed during the whole process of literature review.

McNabb (2008), on the other hand, formulates three fundamental purposes of literature review that are described below:

First, literature review shows the audience of the study that the author is familiar with the major contributions that have already been done to the research area by other authors. Second, literature helps to identify the key issues in the research area and obvious gaps in the current literature.

Third, the literature review assists the readers of the research in term of comprehending the principles and theories that have been used by the author in different parts of the study.

• Brown RB, 2006, Doing Your Dissertation in Business and Management: The Reality of Research and Writing, Sage Publications
• McNabb, DE, 2008, Research Methods in Public Administration and Non-Profit Management: Qualitative and Quantitative Approaches, 2 nd edition, ME Sharpe
• Wysocki, DK, 2007, Readings in Social Research Methods, Cengage Learning

Literature Review - what is a Literature Review, why it is important and how it is done

• Strategies to Find Sources

Evaluating Literature Reviews and Sources

Reading critically, tips to evaluate sources.

• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

• A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips. First part focus on Introductions and in page 10 in the PDF, 37 in the text, it focus on "literature reviews".
• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

To be able to write a good Literature Review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

• Who is the author? What is his/her standing in the field.
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved overtime?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

• How to Read a Paper (University of Waterloo, Canada) This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.

Criteria to evaluate sources:

• Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful site - Critically Analyzing Information Sources (Cornell University Library)

• << Previous: Strategies to Find Sources
• Next: Tips for Writing Literature Reviews >>
• Last Updated: Apr 10, 2024 3:27 PM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

Literature reviews

• Introduction
• Conducting your search
• Store and organise the literature

Evaluate the information you have found

Critique the literature.

• Different subject areas
• Find literature reviews

When conducting your searches you may find many references that will not be suitable to use in your literature review.

• Skim through the resource - a quick read through the table of contents, the introductory paragraph or the abstract should indicate whether you need to read further or whether you can immediately discard the result.
• Evaluate the quality and reliability of the references you find - our page on evaluating information outlines what you need to consider when evaluating the books, journal articles, news and websites you find to ensure they are suitable for use in your literature review.

Critiquing the literature involves looking at the strength and weaknesses of the paper and evaluating the statements made by the author/s.

Books and resources on reading critically

• CASP Checklists Critical appraisal tools designed to be used when reading research. Includes tools for Qualitative studies, Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies and Clinical Prediction Rule.
• How to read critically - business and management From Postgraduate research in business - the aim of this chapter is to show you how to become a critical reader of typical academic literature in business and management.
• Learning to read critically in language and literacy Aims to develop skills of critical analysis and research design. It presents a series of examples of `best practice' in language and literacy education research.
• Critical appraisal in health sciences See tools for critically appraising health science research.

• << Previous: Store and organise the literature
• Next: Different subject areas >>
• Last Updated: Dec 15, 2023 12:09 PM
• URL: https://guides.library.uq.edu.au/research-techniques/literature-reviews

• UNC Chapel Hill

Critical Analysis of Reliability and Validity in Literature Reviews

Chetwynd, E.J.

Introduction

Literature reviews can take many forms depending on the field of specialty and the specific purpose of the review. The evidence base for lactation integrates research that cuts across multiple specialties (Dodgson, 2019) but the most common literature reviews accepted in the Journal of Human Lactation include scoping reviews, systematic reviews, and meta-analyses. Scoping reviews map out the literature in a particular topic area or answer a question about a particular concept or characteristic of the literature about a particular topic. They are broad, detailed, often focused on emerging evidence, and can be used to determine whether a more rigorous systematic review would be useful (Munn et al., 2018). To this end, a scoping review can draw from various sources of evidence, including expert opinion and policy documents, sometimes referred to as “grey literature” (Tricco, et al., 2018). A systematic review has a different purpose to a scoping review. According to the Cochrane Library (www. cochranelibrary.com), under the the section heading “What is a Systemic Review?” the following is stated: it will “identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question” https://www.cochranelibrary.com/about/ about-cochrane-reviews.). Meta-analysis takes the process of systematic review one step further by pooling the data collected and presenting aggregated summary results (Ahn & Kang, 2018). Each type of analysis or review requires a critical analysis of the methodologies used in the reviewed articles (Dodgson, 2021). In a scoping review, the results of the critical analysis are integrated and reported descriptively since they are designed to broadly encapsulate all of the research in a topic area and identify the current state of the science, rather than including only research that meets specific established quality guidelines (Munn et al., 2018). Systematic reviews and meta-analyses use critical analysis differently. In these types of reviews and analyses, the quality of research methods and study instruments becomes an inclusion criterion for deciding which studies to include for analysis so that their authors can ensure rigor in their processes (Page et al., 2021). Reliability and validity are research specific terms that may be applied throughout scientific literature to assess many elements of research methods, designs, and outcomes; however, here we are focusing specifically on their use for assessing measurement in quantitative research methodology. Specifically, we will be examining how they are used within literature review analyses to describe the nature of the instruments applied to measure study variables. Within this framework, reliability refers to the reproducibility of the study results, should the same measurement instruments be applied in different situations (Revelle & Condon, 2019). Validity tests the interpretation of study instruments and refers to whether they measure what they have been reported to be measuring, as supported by evidence and theory in the topic area of investigation (Clark & Watson, 2019). Reliability and validity can exist separately in a study; however, robust studies are both reliable and valid (Urbina & Monks, 2021). In order to establish a benchmark for determining the quality and rigor across all methodologies and reporting guidelines (Dodgson, 2019), the Journal of Human Lactation requires that the authors of any type of literature review include two summary tables. The first table illustrates the study design broadly, asking for study aims, a description of the sample, and the research design for each of the reviewed articles. The second required table is focused on measurement. It guides authors to list each study’s variables, the instruments used to measure each variable, and the reliability and validity of each of these study instrument (https://journals .sagepub.com/author-instructions/jhl#LiteratureReview; Simera et. al., 2010). The techniques used to describe the measurement reliability and validity are sometimes described explicitly using either statistical testing or other recognized forms of testing (Duckett, 2021). However, there are times when the methods for evaluating the measurement used have not been explicitly stated. This situation requires the authors of the review to have a clear understanding of reliability and validity in measurement to extrapolate the methods researchers may have used. Lactation is a topic area that incorporates many fields of specialty; therefore, this article will not be an exhaustive exploration of all types of tests for measurement of reliability and validity. The aim, instead, is to provide readers with enough information to feel confident about finding and assessing implicit types of measurement reliability and validity within published manuscripts. Additionally, readers will be better able to evaluate the usefulness of reviews and the instruments included in those reviews. To that end, this article will: (1) describe types of reliability and validity used in measurement; (2) demonstrate how realiability and validity might be implemented; and (3) discuss how to critically review reliability and validity in literature reviews.

Chetwynd EM, Wasser HM, Poole C. Breastfeeding Support Interventions by International Board Certified Lactation Consultants: A Systemic Review and Meta-Analysis. J Hum Lact. 2019 Aug;35(3):424-440. doi: 10.1177/0890334419851482. Epub 2019 Jun 17. PMID: 31206317.

Publication Link

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Reliability vs Validity in Research | Differences, Types & Examples

Reliability vs Validity in Research | Differences, Types & Examples

Published on 3 May 2022 by Fiona Middleton . Revised on 10 October 2022.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique, or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research .

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect your data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Prevent plagiarism, run a free check.

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalisability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability, or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data .

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are of high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardised questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or the findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid generalisable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession). Ensure that you have enough participants and that they are representative of the population.

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible.

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations, clearly define how specific behaviours or responses will be counted, and make sure questions are phrased the same way each time.

• Standardise the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper. Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Middleton, F. (2022, October 10). Reliability vs Validity in Research | Differences, Types & Examples. Scribbr. Retrieved 29 April 2024, from https://www.scribbr.co.uk/research-methods/reliability-or-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, the 4 types of validity | types, definitions & examples, a quick guide to experimental design | 5 steps & examples, sampling methods | types, techniques, & examples.

Literature Reviews

• Step 1: Understanding Literature Reviews
• Step 2: Gathering Information
• Step 3: Organizing your Information (Intellectually and Physically)

Analyzing and Evaluating the Literature

Evaluating your sources, analysing your sources.

• Step 5: Writing Your Literature Review
• Other Resources

How to evaluate a source

Consider the more obvious elements of the paper:.

• Is its title clear? Does it accurately reflect the content of the paper?
• Is the abstract well-structured (providing an accurate, albeit brief, description of the purpose, method, theoretical background of the research, as well as its results or conclusions?)
• What does their bibliography look like? For example, if most of their references are quite old despite being a newer paper, you should see if they provide an explanation for that in the paper itself. If not, you may want to consider why they do not have any newer sources informing their research.
• Is the journal its published in prestigious and reputable, or does the journal stand to gain something from publishing this paper? You may need to consider the biases of not only the author, but the publisher!

Evaluate the content:

• Look for identifiable gaps in their method, as well as potential problems with their interpretation of the data.
• Look for any obvious manipulations of the data.
• Do they themselves identify any biases or limitations, or do you notice any that they haven’t identified? 

You are not just looking at what they are saying, but also at what they have NOT said. If they didn’t identify a clear gap or bias, why not? What does that say about the rest of the paper? If at all possible, you may want to see if you can identify where the funding for their study came from if you’re noticing these gaps, in case it is possible to spot a conflict of interest.

How do you analyze your sources?

It can be daunting to logically analyze the argument. If they are only showing one side and not addressing the topic from multiple perspectives, you may want to consider why, and if you feel they do a fair job of trying to present a holistic argument – if they don’t bring up conflicting information and demonstrate how they argument works against it, why not? You can also look for key red flags like:

• logical fallacies: mistakes in your reasoning that undermine the logic of the argument – usually identified due to a lack of evidence
• slippery slopes: a conclusion based on the idea that if one thing happens, it will trigger a series of other small steps leading to a drastic conclusion
• post hoc ergo propter hoc: a conclusion that says that if one event occurs after another, that it was the first event that caused the second due to chronology rather than evidence
• circular arguments: these are arguments that simply restate their premises rather than providing proof
• moral equivalence: this compares minor actions with major atrocities and concludes that both are equally immoral
• ad hominem: these are arguments that attack the character of the person making the argument rather than the argument itself

This is just a sample of the types or red flags that occur in academic writing. For more examples or further explanation, consult Purdue Owl’s academic writing guide, “ Logic in Argumentative Writing .”

• << Previous: Step 3: Organizing your Information (Intellectually and Physically)
• Next: Step 5: Writing Your Literature Review >>
• Last Updated: Feb 6, 2024 5:46 PM
• URL: https://yorkvilleu.libguides.com/LiteratureReviews

English Editing Research Services

Assessing Validity in Systematic Reviews (Internal and External)

The validity of results and conclusions is a critical aspect of a systematic review. A systematic review that doesn’t answer a valid question or hasn’t used valid methods won’t have a valid result. And then it won’t be generalizable to a larger population, which makes it have little impact or value in the literature.

So then, how can you be sure that your systematic review has an acceptable level of validity? Look at it from the perspective of both external validity and internal validity.

What you’ll learn in this post

• The definitions of internal validity and external validity in systematic reviews.

• Why validity is so important to consider and assess when you write a systematic literature review.

• How validity will help expand the impact and reach of your review paper.

• The key relationship of bias and validity.

• Where to take free courses to educate yourself on reviews, and how to speed your review to publication, while ensuring it’s valid and valuable.

What is validity and why is it important for systematic reviews?

Validity for systematic reviews is how trustworthy the review’s conclusions are for a reader.

Systematic reviews compile different studies and present a summary of a range of findings.

It’s strength in numbers – and this strength is why they’re at the top of the evidence pyramid , the strongest form of evidence.

Many health organizations, for instance, use evidence from systematic reviews, especially Cochrane reviews , to draft practice guidelines. This is precisely why your systematic review must have trustworthy conclusions. These will give it impact and value, which is why you spent all that time on it.

Validity measures this trustworthiness. It depends on the strength of your review methodology. External validity and internal validity are the two main means of evaluation, so let’s look at each.

External validity in systematic reviews

External validity is how generalizable the results of a systematic review are. Can you generalize the results to populations not included in your systematic review? If “yes,” then you’ve achieved good external validity.

If you’re a doctor and read a systematic review that found a particular drug effective, you may wonder if you can use that drug to treat your patients. For example, this systematic review concluded antidepressants worked better than placebo in adults with major depressive disorder. But…

• Can the results of this study also be applied to older patients with major depressive disorder?
• How about for adolescents or certain cultures?
• Is the treatment regimen self-manageable?

Various factors will impact the external validity. The main ones are…

Sample size

Sampling is key. The results of a systematic review with a larger sample size will typically be more generalizable than those with a smaller sample size.

This meta-analysis estimated how sample size affected treatment effects when different studies were pooled together. The authors found the treatment effects were 32% larger in studies with a smaller sample size vs. a larger one. Trials with smaller sample sizes could provide more exaggerated results than those with larger sample sizes and, by extension, the greater population.

Using a smaller sample size for your systematic review will lower its generalizability (and thus, external validity). The simple takeaway is:

Include as many studies as possible.

This will improve the external validity of your work.

Participant characteristics

Let’s say the conclusions of your systematic review are restricted to a specific sex, age, geographic region, socioeconomic profile, etc. This limits generalizability to participants with a different set of characteristics.

For example, this review concluded that a mean of 27.22% of medical students in China had anxiety (albeit with a range of 8.54% to 88.30%). That’s a key finding from 21 studies.

But what about medical students from a different country?

Or, for that matter, what about Chinese students not studying medicine? Will a similar percentage of them suffer from anxiety?

These questions don’t decrease the value of the findings. The review provides work to build on. But technically, its external validity faces some limitations.

Study setting

Let’s say that your systematic review examined a particular risk factor for a disease in a specific setting.

Can you extrapolate those findings to other settings?

For example, this study evaluated different determinants of population health in urban settings. The authors found that income, education, air quality, occupation status, mobility, and smoking habits impacted morbidity and mortality, in different urban settings.

Are the same findings valid in other urban settings in a different country? Are the findings adaptable to rural settings?

Comparators

With what are you comparing your treatment of interest in your systematic review?

If you compare a new treatment with a placebo, you may find a vast difference in treatment effects. But if you compare a new treatment with another active treatment, the difference in effects may be less prominent. See this systematic review and meta-analysis of treatments for hypertrophic scar and keloid. This review examined two treatments and a placebo to increase its external validity.

The comparator you chose for your systematic review should ideally be a close match to real-world practice. This is another way of upping its external validity.

Reporting external validity

Many systematic reviews insist that you report internal validity yet overlook external validity. In fact, researchers don’t usually use the very term external validity . Many authors use “generalizability,” “applicability,” “feasibility,” or “interchangeability.” They are essentially different terms for the same thing.

The PRISMA guidelines are (as of this writing) what your systematic reviews should follow. Read this article to learn about navigating PRISMA. But even PRISMA doesn’t insist on external validity as much as internal validity.

Authors usually don’t see the need to stress external validity in systematic reviews for all these reasons. Researchers have pointed this out and suggested the importance of reporting external validity.

Nevertheless, internal validity may receive greater attention and is also critical for your systematic review’s overall validity and worth.

Internal validity in systematic reviews

As the name implies, internal validity looks at the inside of the study rather than the external factors. It’s about how strong the study methodology is, and in a systematic review, it’s largely defined by the extent of bias.

Internal validity is easier to measure and achieve than external validity. This owes to the extensive work that’s gone into measuring it. Many organizations, such as Cochrane collaboration and the Joanna Briggs Institute , have developed tools for calculating bias (see below). A similar effort hasn’t gone into measuring external validity.

As a systematic reviewer, you must check the methodological quality of the studies in your systematic review and report the extent of different types of bias within them. This accumulates toward your own study’s internal validity.

Selection bias

Selection bias refers to the selection of participants in a trial.

If the baseline characteristics of two groups in a study are considerably different, selection bias is likely present.

For example, in a randomized controlled trial (RCT) of a new drug for heart failure, if one group has more diabetic patients than the other, then this group is likely to have lower treatment success.

Non-uniform allocation of intervention between two can negatively affect the results.

Strong randomization can reduce selection bias. This is why RCTs are considered the gold standard in evidence generation.

To check selection bias in an RCT in your systematic review, search for words that describe how randomization was done. If the study describes a random number table, sequence generation for randomization, and/or allocation concealment before patients are assigned to the different groups, then there’s probably no selection bias.

This neurological RCT is a good example of strong randomization, despite a relatively small population (n=35).

Performance bias

Performance bias checks if all treatment groups in a study have received a similar level of care. A varying level of care between the groups can bias the results. Researchers often blind or mask the study participants and caregivers to reduce performance bias. An RCT with no details about blinding or masking probably suffers from performance bias.

Blinding, however, isn’t always possible, so a non-blinded study may still have worth and still warrant inclusion in your review.

For example, a cancer drug trial may compare one drug given orally and another injected drug. Or a surgical technique trial may compare surgery with non-invasive treatment.

In both situations, blinding is not practical. The existing bias should be acknowledged in such cases.

Detection bias

Detection bias can occur if the outcome assessors are aware of the intervention the groups received. If an RCT mentions that the outcome assessors were blinded or masked, this suggests a low risk of detection bias.

Blinding of outcome assessors is important when an RCT measures subjective outcomes. This study , for instance, assessed postoperative pain after gallbladder surgery. The postoperative dressing was identical so that the patients would be unaware of (blinded from) the treatment received.

Attrition bias

Attrition bias results from incomplete study data.

Over the course of an RCT, patients may be excluded from analysis or may not return for follow-up. Both result in attrition. All RCTs have some attrition, but if the attrition rate is considerably different between the study groups, the results become skewed.

Attrition bias decreases when using intention-to-treat analysis. But in a per-protocol analysis, attrition bias is usually high. If a study uses both these analyses and finds the results are similar, the attrition bias is considered low.

For example, this RCT of a surgical procedure found that the intention-to-treat analysis and per-protocol analysis were similar. This suggests low attrition bias.

If you find the RCT included in your systematic review hasn’t performed an intention-to-treat analysis, then it’s likely that the included RCT suffers from attrition bias.

Reporting bias

When there are remarkable differences between reported and unreported findings in an RCT, that’s usually a case of reporting bias.

This bias can also arise when study authors report only statistically significant results, leaving out the non-significant ones. Many journals encourage authors to share their data sets to overcome this bias.

For an expert look at risk of bias in systematic reviews, see this article .

Calculating and reporting internal validity/bias

As bias can hurt your review’s internal validity, you must identify the different types of bias present in the studies you include.

Many tools now exist to help with this. Which tool you use depends on the nature of the studies in your review.

• For RCTs, try Cochrane’s risk-of-bias tool for randomized trials (RoB-2) tool .
• For non-randomized trials, try the ROBINS-I tool .
• For case-control studies, there’s the Newcastle–Ottawa Scale (NOS) .
• The AMSTAR-2 tool can be used for checking systematic review quality.

Do your systematic review or meta-analysis in less than one month

A systematic review is a valuable contribution to the literature. It’s top-level evidence and it will advance your research career.

We have published experts ready to assist you with all steps of the systematic review process. Go here to find how you can put Edanz on your team and get a systematic review in as little as 3 weeks!

And find how Edanz’s other research services can help you reach new heights .

• No results found

Validity and reliability

Chapter 2: literature review, 3.2. validity and reliability.

The importance of any research finding basically depends on how valid and reliable it is. However, Cohen et al. (2007: 134) claim that ‘threats to validity and reliability can never be erased completely’. Although early notions of validity and reliability focussed on ‘a demonstration that a particular instrument in fact measures what it purports to measure’ (ibid), some more recent interpretations concentrate on facts such as ‘honesty, depth, richness and scope of the data achieved, the participants approached, the extent

of triangulation and the disinterestedness or objectivity of the researcher’ (Winter; 2000 cited in Cohen et al., 2007: 133).

Maxwell (1992) argues for five kinds of validity in qualitative research: descriptive, interpretive, theoretical, generalizability and evaluative validity. This study adopts three of these dimensions which are relevant to a comprehension of how teachers construct professional identities.

· Descriptive validity: The factual accuracy of the account, that it is not made up, selective or distorted.

Descriptive validity was an issue of great importance in this study. The voices of pre-service teachers were heard through oral and written narratives which were carefully recorded, transcribed, translated and analysed. Assurances were made to reduce any possible alteration or misinterpretation of the data gathered. Both semi- structured interviews and the sessions of stimulated recall (SR) were literally transcribed, followed by a process of confirmation that was undertaken by a professional bilingual secretary who double-checked the Spanish transcriptions. Once they were ready, the drafts were shared with each research participant who had the chance to clarify or rectify any misleading transcription. The researcher and the secretary went through the whole body of the data again with the intention of ascertaining descriptive validity as fully as possible. The drafts were edited several times until agreement was reached that each transcript matched the context and meaning of what was narrated by the research participant. The data collected from on-line blogs were also incorporated in this step.

· Interpretive validity: The ability of the research to capture the meanings, interpretations, terms, and intentions about situations and events, i.e. the data, as expressed by the participants/subjects themselves, in their terms.

Interpretive validity was also strongly emphasized. The process of data analysis was a subject of permanent discussion with the research supervisor and members of the ‘Enletawa’ research group15 at Universidad Pedagógica y Tecnológica de Colombia (UPTC). Some valuable comments and feedback resulted from participation in conferences and workshops in the UK and Colombia, where early findings were shared, and this contributed enormously to assuring interpretive validity. That sense of the socially-constructed understanding of phenomena strengthened to a considerable extent the process of data collection and analysis.

· Theoretical validity: The theoretical constructions that the researcher brings to the research, including those of the researched theory. Theoretical validity is the extent to which the research explains observed phenomena.

Theoretical validity was assured by tracing relevant and recent theoretical constructs concerning the way teacher identity has been approached in the last decades. This study has enriched the possibilities for new and more critical perspectives in this field in the future. The research findings, conceptualisations, and interpretations of the reality approached in the context of pre-service teacher professional identity construction has been extensively discussed, refined, and shared with colleagues and scholars.

The concept of reliability assumes different meanings in qualitative and quantitative methodologies. Premises such as precision and accuracy, similar results in similar contexts or ‘dependability, consistency and replicability over time, over instruments and over groups of respondents’ are of paramount importance in quantitative research (Cohen et al., 2007: 146). Qualitative designs replace these notions with terms such as credibility, applicability, consistency or trustworthiness, for example (ibid). The debate here does not restrict the need for attenuating, yielding or validating findings in qualitative research, but rather reorients the discussion towards the analysis

ENLETAWA (English Language Teachers’ Awareness) is a research group at UPTC in Colombia. They are also the editors of ENLETAWA Journal.

of issues such as ‘the status position of the researcher, the choice of informants/respondents or the methods of data collection and analysis’ (Lecompte and Preissle, 1993: 334).

Transferability concerns whether or not the study could be undertaken in any other context, with a similar or different populations. This means that the ‘theory generated may be useful in understanding other similar situations’ (Cohen, et al., 2007: 135). The images of professional identity that are approached in this study are exploratory rather than conclusive. To gain a closer understanding of how identities are formed, sustained or transformed, further interpretations within groups that share similar circumstances will be required.

Credibility arose as another notion of reliability. This study deals with human beings who are engaged in processes of social and cognitive growth. These are two important premises in understanding the nature of identity construction. A sense of trustworthiness has to be built through the interaction between the researcher and the participants during the interviewing process. Some traces of interaction as they are revealed and discussed during the stimulated recall (SR) session are also a source of credibility. Pre-service teacher online blogs and their stories about teaching and learning have to be considered trustworthy. The next section examines case studies which were also part of the methodological framework of this study.

• The concept of identity
• A Historical overview of identity
• The concept of teacher identity
• Learning to teach and identity
• Beliefs and classroom practice
• Professional goals
• The research area in Colombia
• Validity and reliability (You are here)
• Stimulated recall

Related documents

A literature review to assess the reliability and validity of measures appropriate for use in research to evaluate the efficacy of a brief harm reduction strategy in reducing cannabis use among people with schizophrenia in acute inpatient settings

Affiliation.

• 1 Health Services Research, Institute of Psychiatry, King's College, London, UK. [email protected]
• PMID: 18844804
• DOI: 10.1111/j.1365-2850.2008.01297.x

There is a growing body of evidence looking at the effects of cannabis use on those with schizophrenia with concerning results. This has led to the development of a number of interventions that are intended to improve outcomes for this client group. However, the methodological quality of some dual diagnosis research has been questioned in reviews for using outcome measures that are not tested as reliable and valid in the population for which they are intended for use. This literature review assesses the self-report measures that have been reliability and validity tested in populations of people with schizophrenia who use cannabis and reports on their appropriateness for use in further research studies. An overview of the most appropriate biochemical tests for cannabis is also given.

Publication types

• Harm Reduction
• Inpatients*
• Marijuana Abuse / prevention & control*
• Nursing Research / standards*
• Randomized Controlled Trials as Topic
• Reproducibility of Results
• Risk Reduction Behavior
• Schizophrenia / complications
• Schizophrenia / nursing*
• Schizophrenia / therapy