research design limitations examples

How to Write Limitations of the Study (with examples)

This blog emphasizes the importance of recognizing and effectively writing about limitations in research. It discusses the types of limitations, their significance, and provides guidelines for writing about them, highlighting their role in advancing scholarly research.

Updated on August 24, 2023

No matter how well thought out, every research endeavor encounters challenges. There is simply no way to predict all possible variances throughout the process.

These uncharted boundaries and abrupt constraints are known as limitations in research . Identifying and acknowledging limitations is crucial for conducting rigorous studies. Limitations provide context and shed light on gaps in the prevailing inquiry and literature.

This article explores the importance of recognizing limitations and discusses how to write them effectively. By interpreting limitations in research and considering prevalent examples, we aim to reframe the perception from shameful mistakes to respectable revelations.

What are limitations in research?

In the clearest terms, research limitations are the practical or theoretical shortcomings of a study that are often outside of the researcher’s control . While these weaknesses limit the generalizability of a study’s conclusions, they also present a foundation for future research.

Sometimes limitations arise from tangible circumstances like time and funding constraints, or equipment and participant availability. Other times the rationale is more obscure and buried within the research design. Common types of limitations and their ramifications include:

• Theoretical: limits the scope, depth, or applicability of a study.
• Methodological: limits the quality, quantity, or diversity of the data.
• Empirical: limits the representativeness, validity, or reliability of the data.
• Analytical: limits the accuracy, completeness, or significance of the findings.
• Ethical: limits the access, consent, or confidentiality of the data.

Regardless of how, when, or why they arise, limitations are a natural part of the research process and should never be ignored . Like all other aspects, they are vital in their own purpose.

Why is identifying limitations important?

Whether to seek acceptance or avoid struggle, humans often instinctively hide flaws and mistakes. Merging this thought process into research by attempting to hide limitations, however, is a bad idea. It has the potential to negate the validity of outcomes and damage the reputation of scholars.

By identifying and addressing limitations throughout a project, researchers strengthen their arguments and curtail the chance of peer censure based on overlooked mistakes. Pointing out these flaws shows an understanding of variable limits and a scrupulous research process.

Showing awareness of and taking responsibility for a project’s boundaries and challenges validates the integrity and transparency of a researcher. It further demonstrates the researchers understand the applicable literature and have thoroughly evaluated their chosen research methods.

Presenting limitations also benefits the readers by providing context for research findings. It guides them to interpret the project’s conclusions only within the scope of very specific conditions. By allowing for an appropriate generalization of the findings that is accurately confined by research boundaries and is not too broad, limitations boost a study’s credibility .

Limitations are true assets to the research process. They highlight opportunities for future research. When researchers identify the limitations of their particular approach to a study question, they enable precise transferability and improve chances for reproducibility. 

Simply stating a project’s limitations is not adequate for spurring further research, though. To spark the interest of other researchers, these acknowledgements must come with thorough explanations regarding how the limitations affected the current study and how they can potentially be overcome with amended methods.

How to write limitations

Typically, the information about a study’s limitations is situated either at the beginning of the discussion section to provide context for readers or at the conclusion of the discussion section to acknowledge the need for further research. However, it varies depending upon the target journal or publication guidelines. 

Don’t hide your limitations

It is also important to not bury a limitation in the body of the paper unless it has a unique connection to a topic in that section. If so, it needs to be reiterated with the other limitations or at the conclusion of the discussion section. Wherever it is included in the manuscript, ensure that the limitations section is prominently positioned and clearly introduced.

While maintaining transparency by disclosing limitations means taking a comprehensive approach, it is not necessary to discuss everything that could have potentially gone wrong during the research study. If there is no commitment to investigation in the introduction, it is unnecessary to consider the issue a limitation to the research. Wholly consider the term ‘limitations’ and ask, “Did it significantly change or limit the possible outcomes?” Then, qualify the occurrence as either a limitation to include in the current manuscript or as an idea to note for other projects. 

Writing limitations

Once the limitations are concretely identified and it is decided where they will be included in the paper, researchers are ready for the writing task. Including only what is pertinent, keeping explanations detailed but concise, and employing the following guidelines is key for crafting valuable limitations:

1) Identify and describe the limitations : Clearly introduce the limitation by classifying its form and specifying its origin. For example:

• An unintentional bias encountered during data collection
• An intentional use of unplanned post-hoc data analysis

2) Explain the implications : Describe how the limitation potentially influences the study’s findings and how the validity and generalizability are subsequently impacted. Provide examples and evidence to support claims of the limitations’ effects without making excuses or exaggerating their impact. Overall, be transparent and objective in presenting the limitations, without undermining the significance of the research. 

3) Provide alternative approaches for future studies : Offer specific suggestions for potential improvements or avenues for further investigation. Demonstrate a proactive approach by encouraging future research that addresses the identified gaps and, therefore, expands the knowledge base.

Whether presenting limitations as an individual section within the manuscript or as a subtopic in the discussion area, authors should use clear headings and straightforward language to facilitate readability. There is no need to complicate limitations with jargon, computations, or complex datasets.

Examples of common limitations

Limitations are generally grouped into two categories , methodology and research process .

Methodology limitations

Methodology may include limitations due to:

• Sample size
• Lack of available or reliable data
• Lack of prior research studies on the topic
• Measure used to collect the data
• Self-reported data

The researcher is addressing how the large sample size requires a reassessment of the measures used to collect and analyze the data.

Research process limitations

Limitations during the research process may arise from:

• Access to information
• Longitudinal effects
• Cultural and other biases
• Language fluency
• Time constraints

The author is pointing out that the model’s estimates are based on potentially biased observational studies.

Final thoughts

Successfully proving theories and touting great achievements are only two very narrow goals of scholarly research. The true passion and greatest efforts of researchers comes more in the form of confronting assumptions and exploring the obscure.

In many ways, recognizing and sharing the limitations of a research study both allows for and encourages this type of discovery that continuously pushes research forward. By using limitations to provide a transparent account of the project's boundaries and to contextualize the findings, researchers pave the way for even more robust and impactful research in the future.

Charla Viera, MS

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

21 Research Limitations Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

Research limitations refer to the potential weaknesses inherent in a study. All studies have limitations of some sort, meaning declaring limitations doesn’t necessarily need to be a bad thing, so long as your declaration of limitations is well thought-out and explained.

Rarely is a study perfect. Researchers have to make trade-offs when developing their studies, which are often based upon practical considerations such as time and monetary constraints, weighing the breadth of participants against the depth of insight, and choosing one methodology or another.

In research, studies can have limitations such as limited scope, researcher subjectivity, and lack of available research tools.

Acknowledging the limitations of your study should be seen as a strength. It demonstrates your willingness for transparency, humility, and submission to the scientific method and can bolster the integrity of the study. It can also inform future research direction.

Typically, scholars will explore the limitations of their study in either their methodology section, their conclusion section, or both.

Research Limitations Examples

Qualitative and quantitative research offer different perspectives and methods in exploring phenomena, each with its own strengths and limitations. So, I’ve split the limitations examples sections into qualitative and quantitative below.

Qualitative Research Limitations

Qualitative research seeks to understand phenomena in-depth and in context. It focuses on the ‘why’ and ‘how’ questions.

It’s often used to explore new or complex issues, and it provides rich, detailed insights into participants’ experiences, behaviors, and attitudes. However, these strengths also create certain limitations, as explained below.

1. Subjectivity

Qualitative research often requires the researcher to interpret subjective data. One researcher may examine a text and identify different themes or concepts as more dominant than others.

Close qualitative readings of texts are necessarily subjective – and while this may be a limitation, qualitative researchers argue this is the best way to deeply understand everything in context.

Suggested Solution and Response: To minimize subjectivity bias, you could consider cross-checking your own readings of themes and data against other scholars’ readings and interpretations. This may involve giving the raw data to a supervisor or colleague and asking them to code the data separately, then coming together to compare and contrast results.

2. Researcher Bias

The concept of researcher bias is related to, but slightly different from, subjectivity.

Researcher bias refers to the perspectives and opinions you bring with you when doing your research.

For example, a researcher who is explicitly of a certain philosophical or political persuasion may bring that persuasion to bear when interpreting data.

In many scholarly traditions, we will attempt to minimize researcher bias through the utilization of clear procedures that are set out in advance or through the use of statistical analysis tools.

However, in other traditions, such as in postmodern feminist research , declaration of bias is expected, and acknowledgment of bias is seen as a positive because, in those traditions, it is believed that bias cannot be eliminated from research, so instead, it is a matter of integrity to present it upfront.

Suggested Solution and Response: Acknowledge the potential for researcher bias and, depending on your theoretical framework , accept this, or identify procedures you have taken to seek a closer approximation to objectivity in your coding and analysis.

3. Generalizability

If you’re struggling to find a limitation to discuss in your own qualitative research study, then this one is for you: all qualitative research, of all persuasions and perspectives, cannot be generalized.

This is a core feature that sets qualitative data and quantitative data apart.

The point of qualitative data is to select case studies and similarly small corpora and dig deep through in-depth analysis and thick description of data.

Often, this will also mean that you have a non-randomized sample size.

While this is a positive – you’re going to get some really deep, contextualized, interesting insights – it also means that the findings may not be generalizable to a larger population that may not be representative of the small group of people in your study.

Suggested Solution and Response: Suggest future studies that take a quantitative approach to the question.

4. The Hawthorne Effect

The Hawthorne effect refers to the phenomenon where research participants change their ‘observed behavior’ when they’re aware that they are being observed.

This effect was first identified by Elton Mayo who conducted studies of the effects of various factors ton workers’ productivity. He noticed that no matter what he did – turning up the lights, turning down the lights, etc. – there was an increase in worker outputs compared to prior to the study taking place.

Mayo realized that the mere act of observing the workers made them work harder – his observation was what was changing behavior.

So, if you’re looking for a potential limitation to name for your observational research study , highlight the possible impact of the Hawthorne effect (and how you could reduce your footprint or visibility in order to decrease its likelihood).

Suggested Solution and Response: Highlight ways you have attempted to reduce your footprint while in the field, and guarantee anonymity to your research participants.

5. Replicability

Quantitative research has a great benefit in that the studies are replicable – a researcher can get a similar sample size, duplicate the variables, and re-test a study. But you can’t do that in qualitative research.

Qualitative research relies heavily on context – a specific case study or specific variables that make a certain instance worthy of analysis. As a result, it’s often difficult to re-enter the same setting with the same variables and repeat the study.

Furthermore, the individual researcher’s interpretation is more influential in qualitative research, meaning even if a new researcher enters an environment and makes observations, their observations may be different because subjectivity comes into play much more. This doesn’t make the research bad necessarily (great insights can be made in qualitative research), but it certainly does demonstrate a weakness of qualitative research.

6. Limited Scope

“Limited scope” is perhaps one of the most common limitations listed by researchers – and while this is often a catch-all way of saying, “well, I’m not studying that in this study”, it’s also a valid point.

No study can explore everything related to a topic. At some point, we have to make decisions about what’s included in the study and what is excluded from the study.

So, you could say that a limitation of your study is that it doesn’t look at an extra variable or concept that’s certainly worthy of study but will have to be explored in your next project because this project has a clearly and narrowly defined goal.

Suggested Solution and Response: Be clear about what’s in and out of the study when writing your research question.

7. Time Constraints

This is also a catch-all claim you can make about your research project: that you would have included more people in the study, looked at more variables, and so on. But you’ve got to submit this thing by the end of next semester! You’ve got time constraints.

And time constraints are a recognized reality in all research.

But this means you’ll need to explain how time has limited your decisions. As with “limited scope”, this may mean that you had to study a smaller group of subjects, limit the amount of time you spent in the field, and so forth.

Suggested Solution and Response: Suggest future studies that will build on your current work, possibly as a PhD project.

8. Resource Intensiveness

Qualitative research can be expensive due to the cost of transcription, the involvement of trained researchers, and potential travel for interviews or observations.

So, resource intensiveness is similar to the time constraints concept. If you don’t have the funds, you have to make decisions about which tools to use, which statistical software to employ, and how many research assistants you can dedicate to the study.

Suggested Solution and Response: Suggest future studies that will gain more funding on the back of this ‘ exploratory study ‘.

9. Coding Difficulties

Data analysis in qualitative research often involves coding, which can be subjective and complex, especially when dealing with ambiguous or contradicting data.

After naming this as a limitation in your research, it’s important to explain how you’ve attempted to address this. Some ways to ‘limit the limitation’ include:

• Triangulation: Have 2 other researchers code the data as well and cross-check your results with theirs to identify outliers that may need to be re-examined, debated with the other researchers, or removed altogether.
• Procedure: Use a clear coding procedure to demonstrate reliability in your coding process. I personally use the thematic network analysis method outlined in this academic article by Attride-Stirling (2001).

Suggested Solution and Response: Triangulate your coding findings with colleagues, and follow a thematic network analysis procedure.

10. Risk of Non-Responsiveness

There is always a risk in research that research participants will be unwilling or uncomfortable sharing their genuine thoughts and feelings in the study.

This is particularly true when you’re conducting research on sensitive topics, politicized topics, or topics where the participant is expressing vulnerability .

This is similar to the Hawthorne effect (aka participant bias), where participants change their behaviors in your presence; but it goes a step further, where participants actively hide their true thoughts and feelings from you.

Suggested Solution and Response: One way to manage this is to try to include a wider group of people with the expectation that there will be non-responsiveness from some participants.

11. Risk of Attrition

Attrition refers to the process of losing research participants throughout the study.

This occurs most commonly in longitudinal studies , where a researcher must return to conduct their analysis over spaced periods of time, often over a period of years.

Things happen to people over time – they move overseas, their life experiences change, they get sick, change their minds, and even die. The more time that passes, the greater the risk of attrition.

Suggested Solution and Response: One way to manage this is to try to include a wider group of people with the expectation that there will be attrition over time.

12. Difficulty in Maintaining Confidentiality and Anonymity

Given the detailed nature of qualitative data , ensuring participant anonymity can be challenging.

If you have a sensitive topic in a specific case study, even anonymizing research participants sometimes isn’t enough. People might be able to induce who you’re talking about.

Sometimes, this will mean you have to exclude some interesting data that you collected from your final report. Confidentiality and anonymity come before your findings in research ethics – and this is a necessary limiting factor.

Suggested Solution and Response: Highlight the efforts you have taken to anonymize data, and accept that confidentiality and accountability place extremely important constraints on academic research.

13. Difficulty in Finding Research Participants

A study that looks at a very specific phenomenon or even a specific set of cases within a phenomenon means that the pool of potential research participants can be very low.

Compile on top of this the fact that many people you approach may choose not to participate, and you could end up with a very small corpus of subjects to explore. This may limit your ability to make complete findings, even in a quantitative sense.

You may need to therefore limit your research question and objectives to something more realistic.

Suggested Solution and Response: Highlight that this is going to limit the study’s generalizability significantly.

14. Ethical Limitations

Ethical limitations refer to the things you cannot do based on ethical concerns identified either by yourself or your institution’s ethics review board.

This might include threats to the physical or psychological well-being of your research subjects, the potential of releasing data that could harm a person’s reputation, and so on.

Furthermore, even if your study follows all expected standards of ethics, you still, as an ethical researcher, need to allow a research participant to pull out at any point in time, after which you cannot use their data, which demonstrates an overlap between ethical constraints and participant attrition.

Suggested Solution and Response: Highlight that these ethical limitations are inevitable but important to sustain the integrity of the research.

For more on Qualitative Research, Explore my Qualitative Research Guide

Quantitative Research Limitations

Quantitative research focuses on quantifiable data and statistical, mathematical, or computational techniques. It’s often used to test hypotheses, assess relationships and causality, and generalize findings across larger populations.

Quantitative research is widely respected for its ability to provide reliable, measurable, and generalizable data (if done well!). Its structured methodology has strengths over qualitative research, such as the fact it allows for replication of the study, which underpins the validity of the research.

However, this approach is not without it limitations, explained below.

1. Over-Simplification

Quantitative research is powerful because it allows you to measure and analyze data in a systematic and standardized way. However, one of its limitations is that it can sometimes simplify complex phenomena or situations.

In other words, it might miss the subtleties or nuances of the research subject.

For example, if you’re studying why people choose a particular diet, a quantitative study might identify factors like age, income, or health status. But it might miss other aspects, such as cultural influences or personal beliefs, that can also significantly impact dietary choices.

When writing about this limitation, you can say that your quantitative approach, while providing precise measurements and comparisons, may not capture the full complexity of your subjects of study.

Suggested Solution and Response: Suggest a follow-up case study using the same research participants in order to gain additional context and depth.

2. Lack of Context

Another potential issue with quantitative research is that it often focuses on numbers and statistics at the expense of context or qualitative information.

Let’s say you’re studying the effect of classroom size on student performance. You might find that students in smaller classes generally perform better. However, this doesn’t take into account other variables, like teaching style , student motivation, or family support.

When describing this limitation, you might say, “Although our research provides important insights into the relationship between class size and student performance, it does not incorporate the impact of other potentially influential variables. Future research could benefit from a mixed-methods approach that combines quantitative analysis with qualitative insights.”

3. Applicability to Real-World Settings

Oftentimes, experimental research takes place in controlled environments to limit the influence of outside factors.

This control is great for isolation and understanding the specific phenomenon but can limit the applicability or “external validity” of the research to real-world settings.

For example, if you conduct a lab experiment to see how sleep deprivation impacts cognitive performance, the sterile, controlled lab environment might not reflect real-world conditions where people are dealing with multiple stressors.

Therefore, when explaining the limitations of your quantitative study in your methodology section, you could state:

“While our findings provide valuable information about [topic], the controlled conditions of the experiment may not accurately represent real-world scenarios where extraneous variables will exist. As such, the direct applicability of our results to broader contexts may be limited.”

Suggested Solution and Response: Suggest future studies that will engage in real-world observational research, such as ethnographic research.

4. Limited Flexibility

Once a quantitative study is underway, it can be challenging to make changes to it. This is because, unlike in grounded research, you’re putting in place your study in advance, and you can’t make changes part-way through.

Your study design, data collection methods, and analysis techniques need to be decided upon before you start collecting data.

For example, if you are conducting a survey on the impact of social media on teenage mental health, and halfway through, you realize that you should have included a question about their screen time, it’s generally too late to add it.

When discussing this limitation, you could write something like, “The structured nature of our quantitative approach allows for consistent data collection and analysis but also limits our flexibility to adapt and modify the research process in response to emerging insights and ideas.”

Suggested Solution and Response: Suggest future studies that will use mixed-methods or qualitative research methods to gain additional depth of insight.

5. Risk of Survey Error

Surveys are a common tool in quantitative research, but they carry risks of error.

There can be measurement errors (if a question is misunderstood), coverage errors (if some groups aren’t adequately represented), non-response errors (if certain people don’t respond), and sampling errors (if your sample isn’t representative of the population).

For instance, if you’re surveying college students about their study habits , but only daytime students respond because you conduct the survey during the day, your results will be skewed.

In discussing this limitation, you might say, “Despite our best efforts to develop a comprehensive survey, there remains a risk of survey error, including measurement, coverage, non-response, and sampling errors. These could potentially impact the reliability and generalizability of our findings.”

Suggested Solution and Response: Suggest future studies that will use other survey tools to compare and contrast results.

6. Limited Ability to Probe Answers

With quantitative research, you typically can’t ask follow-up questions or delve deeper into participants’ responses like you could in a qualitative interview.

For instance, imagine you are surveying 500 students about study habits in a questionnaire. A respondent might indicate that they study for two hours each night. You might want to follow up by asking them to elaborate on what those study sessions involve or how effective they feel their habits are.

However, quantitative research generally disallows this in the way a qualitative semi-structured interview could.

When discussing this limitation, you might write, “Given the structured nature of our survey, our ability to probe deeper into individual responses is limited. This means we may not fully understand the context or reasoning behind the responses, potentially limiting the depth of our findings.”

Suggested Solution and Response: Suggest future studies that engage in mixed-method or qualitative methodologies to address the issue from another angle.

7. Reliance on Instruments for Data Collection

In quantitative research, the collection of data heavily relies on instruments like questionnaires, surveys, or machines.

The limitation here is that the data you get is only as good as the instrument you’re using. If the instrument isn’t designed or calibrated well, your data can be flawed.

For instance, if you’re using a questionnaire to study customer satisfaction and the questions are vague, confusing, or biased, the responses may not accurately reflect the customers’ true feelings.

When discussing this limitation, you could say, “Our study depends on the use of questionnaires for data collection. Although we have put significant effort into designing and testing the instrument, it’s possible that inaccuracies or misunderstandings could potentially affect the validity of the data collected.”

Suggested Solution and Response: Suggest future studies that will use different instruments but examine the same variables to triangulate results.

8. Time and Resource Constraints (Specific to Quantitative Research)

Quantitative research can be time-consuming and resource-intensive, especially when dealing with large samples.

It often involves systematic sampling, rigorous design, and sometimes complex statistical analysis.

If resources and time are limited, it can restrict the scale of your research, the techniques you can employ, or the extent of your data analysis.

For example, you may want to conduct a nationwide survey on public opinion about a certain policy. However, due to limited resources, you might only be able to survey people in one city.

When writing about this limitation, you could say, “Given the scope of our research and the resources available, we are limited to conducting our survey within one city, which may not fully represent the nationwide public opinion. Hence, the generalizability of the results may be limited.”

Suggested Solution and Response: Suggest future studies that will have more funding or longer timeframes.

How to Discuss Your Research Limitations

1. in your research proposal and methodology section.

In the research proposal, which will become the methodology section of your dissertation, I would recommend taking the four following steps, in order:

• Be Explicit about your Scope – If you limit the scope of your study in your research question, aims, and objectives, then you can set yourself up well later in the methodology to say that certain questions are “outside the scope of the study.” For example, you may identify the fact that the study doesn’t address a certain variable, but you can follow up by stating that the research question is specifically focused on the variable that you are examining, so this limitation would need to be looked at in future studies.
• Acknowledge the Limitation – Acknowledging the limitations of your study demonstrates reflexivity and humility and can make your research more reliable and valid. It also pre-empts questions the people grading your paper may have, so instead of them down-grading you for your limitations; they will congratulate you on explaining the limitations and how you have addressed them!
• Explain your Decisions – You may have chosen your approach (despite its limitations) for a very specific reason. This might be because your approach remains, on balance, the best one to answer your research question. Or, it might be because of time and monetary constraints that are outside of your control.
• Highlight the Strengths of your Approach – Conclude your limitations section by strongly demonstrating that, despite limitations, you’ve worked hard to minimize the effects of the limitations and that you have chosen your specific approach and methodology because it’s also got some terrific strengths. Name the strengths.

Overall, you’ll want to acknowledge your own limitations but also explain that the limitations don’t detract from the value of your study as it stands.

2. In the Conclusion Section or Chapter

In the conclusion of your study, it is generally expected that you return to a discussion of the study’s limitations. Here, I recommend the following steps:

• Acknowledge issues faced – After completing your study, you will be increasingly aware of issues you may have faced that, if you re-did the study, you may have addressed earlier in order to avoid those issues. Acknowledge these issues as limitations, and frame them as recommendations for subsequent studies.
• Suggest further research – Scholarly research aims to fill gaps in the current literature and knowledge. Having established your expertise through your study, suggest lines of inquiry for future researchers. You could state that your study had certain limitations, and “future studies” can address those limitations.
• Suggest a mixed methods approach – Qualitative and quantitative research each have pros and cons. So, note those ‘cons’ of your approach, then say the next study should approach the topic using the opposite methodology or could approach it using a mixed-methods approach that could achieve the benefits of quantitative studies with the nuanced insights of associated qualitative insights as part of an in-study case-study.

Overall, be clear about both your limitations and how those limitations can inform future studies.

In sum, each type of research method has its own strengths and limitations. Qualitative research excels in exploring depth, context, and complexity, while quantitative research excels in examining breadth, generalizability, and quantifiable measures. Despite their individual limitations, each method contributes unique and valuable insights, and researchers often use them together to provide a more comprehensive understanding of the phenomenon being studied.

Attride-Stirling, J. (2001). Thematic networks: an analytic tool for qualitative research. Qualitative research , 1 (3), 385-405. ( Source )

Atkinson, P., Delamont, S., Cernat, A., Sakshaug, J., & Williams, R. A. (2021).  SAGE research methods foundations . London: Sage Publications.

Clark, T., Foster, L., Bryman, A., & Sloan, L. (2021).  Bryman’s social research methods . Oxford: Oxford University Press.

Köhler, T., Smith, A., & Bhakoo, V. (2022). Templates in qualitative research methods: Origins, limitations, and new directions.  Organizational Research Methods ,  25 (2), 183-210. ( Source )

Lenger, A. (2019). The rejection of qualitative research methods in economics.  Journal of Economic Issues ,  53 (4), 946-965. ( Source )

Taherdoost, H. (2022). What are different research approaches? Comprehensive review of qualitative, quantitative, and mixed method research, their applications, types, and limitations.  Journal of Management Science & Engineering Research ,  5 (1), 53-63. ( Source )

Walliman, N. (2021).  Research methods: The basics . New York: Routledge.

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 10 Reasons you’re Perpetually Single
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 20 Montessori Toddler Bedrooms (Design Inspiration)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 21 Montessori Homeschool Setups
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 101 Hidden Talents Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Instant insights, infinite possibilities

How to present limitations in research

Last updated

30 January 2024

Reviewed by

Short on time? Get an AI generated summary of this article instead

Limitations don’t invalidate or diminish your results, but it’s best to acknowledge them. This will enable you to address any questions your study failed to answer because of them.

In this guide, learn how to recognize, present, and overcome limitations in research.

• What is a research limitation?

Research limitations are weaknesses in your research design or execution that may have impacted outcomes and conclusions. Uncovering limitations doesn’t necessarily indicate poor research design—it just means you encountered challenges you couldn’t have anticipated that limited your research efforts.

Does basic research have limitations?

Basic research aims to provide more information about your research topic . It requires the same standard research methodology and data collection efforts as any other research type, and it can also have limitations.

• Common research limitations

Researchers encounter common limitations when embarking on a study. Limitations can occur in relation to the methods you apply or the research process you design. They could also be connected to you as the researcher.

Methodology limitations

Not having access to data or reliable information can impact the methods used to facilitate your research. A lack of data or reliability may limit the parameters of your study area and the extent of your exploration.

Your sample size may also be affected because you won’t have any direction on how big or small it should be and who or what you should include. Having too few participants won’t adequately represent the population or groups of people needed to draw meaningful conclusions.

Research process limitations

The study’s design can impose constraints on the process. For example, as you’re conducting the research, issues may arise that don’t conform to the data collection methodology you developed. You may not realize until well into the process that you should have incorporated more specific questions or comprehensive experiments to generate the data you need to have confidence in your results.

Constraints on resources can also have an impact. Being limited on participants or participation incentives may limit your sample sizes. Insufficient tools, equipment, and materials to conduct a thorough study may also be a factor.

Common researcher limitations

Here are some of the common researcher limitations you may encounter:

Time: some research areas require multi-year longitudinal approaches, but you might not be able to dedicate that much time. Imagine you want to measure how much memory a person loses as they age. This may involve conducting multiple tests on a sample of participants over 20–30 years, which may be impossible.

Bias: researchers can consciously or unconsciously apply bias to their research. Biases can contribute to relying on research sources and methodologies that will only support your beliefs about the research you’re embarking on. You might also omit relevant issues or participants from the scope of your study because of your biases.

Limited access to data : you may need to pay to access specific databases or journals that would be helpful to your research process. You might also need to gain information from certain people or organizations but have limited access to them. These cases require readjusting your process and explaining why your findings are still reliable.

• Why is it important to identify limitations?

Identifying limitations adds credibility to research and provides a deeper understanding of how you arrived at your conclusions.

Constraints may have prevented you from collecting specific data or information you hoped would prove or disprove your hypothesis or provide a more comprehensive understanding of your research topic.

However, identifying the limitations contributing to your conclusions can inspire further research efforts that help gather more substantial information and data.

• Where to put limitations in a research paper

A research paper is broken up into different sections that appear in the following order:

Introduction

Methodology

The discussion portion of your paper explores your findings and puts them in the context of the overall research. Either place research limitations at the beginning of the discussion section before the analysis of your findings or at the end of the section to indicate that further research needs to be pursued.

What not to include in the limitations section

Evidence that doesn’t support your hypothesis is not a limitation, so you shouldn’t include it in the limitation section. Don’t just list limitations and their degree of severity without further explanation.

• How to present limitations

You’ll want to present the limitations of your study in a way that doesn’t diminish the validity of your research and leave the reader wondering if your results and conclusions have been compromised.

Include only the limitations that directly relate to and impact how you addressed your research questions. Following a specific format enables the reader to develop an understanding of the weaknesses within the context of your findings without doubting the quality and integrity of your research.

Identify the limitations specific to your study

You don’t have to identify every possible limitation that might have occurred during your research process. Only identify those that may have influenced the quality of your findings and your ability to answer your research question.

Explain study limitations in detail

This explanation should be the most significant portion of your limitation section.

Link each limitation with an interpretation and appraisal of their impact on the study. You’ll have to evaluate and explain whether the error, method, or validity issues influenced the study’s outcome and how.

Propose a direction for future studies and present alternatives

In this section, suggest how researchers can avoid the pitfalls you experienced during your research process.

If an issue with methodology was a limitation, propose alternate methods that may help with a smoother and more conclusive research project . Discuss the pros and cons of your alternate recommendation.

Describe steps taken to minimize each limitation

You probably took steps to try to address or mitigate limitations when you noticed them throughout the course of your research project. Describe these steps in the limitation section.

• Limitation example

“Approaches like stem cell transplantation and vaccination in AD [Alzheimer’s disease] work on a cellular or molecular level in the laboratory. However, translation into clinical settings will remain a challenge for the next decade.”

The authors are saying that even though these methods showed promise in helping people with memory loss when conducted in the lab (in other words, using animal studies), more studies are needed. These may be controlled clinical trials, for example. 

However, the short life span of stem cells outside the lab and the vaccination’s severe inflammatory side effects are limitations. Researchers won’t be able to conduct clinical trials until these issues are overcome.

• How to overcome limitations in research

You’ve already started on the road to overcoming limitations in research by acknowledging that they exist. However, you need to ensure readers don’t mistake weaknesses for errors within your research design.

To do this, you’ll need to justify and explain your rationale for the methods, research design, and analysis tools you chose and how you noticed they may have presented limitations.

Your readers need to know that even when limitations presented themselves, you followed best practices and the ethical standards of your field. You didn’t violate any rules and regulations during your research process.

You’ll also want to reinforce the validity of your conclusions and results with multiple sources, methods, and perspectives. This prevents readers from assuming your findings were derived from a single or biased source.

• Learning and improving starts with limitations in research

Dealing with limitations with transparency and integrity helps identify areas for future improvements and developments. It’s a learning process, providing valuable insights into how you can improve methodologies, expand sample sizes, or explore alternate approaches to further support the validity of your findings.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 22 August 2024

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 9 March 2023

Last updated: 30 April 2024

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next, log in or sign up.

Get started for free

Scientific Research and Methodology : An introduction to quantitative research and statistics

8 research design limitations.

So far, you have learnt to ask a RQ and designs research studies. In this chapter , you will learn to identify limitations to:

• internally valid.
• externally valid.
• ecologically valid.

8.1 Introduction

The type of study and the research design determine how the results of the study should be interpreted. Ideally, a study would be perfectly externally and internally valid; in practice this is very difficult to achieve. Practically every study has limitations. The results of a study should be interpreted in light of these limitations. Limitations are not necessarily problems .

Limitations generally can be discussed through three components:

• Internal validity (Sect.  3.1 ): Discuss any limitations to internal validity due to the research design (such as identifying possible confounding variables). This is related to the effectiveness of the study within the sample (Sect.  8.2 ).
• External validity (Sect.  6.1 ): Discuss how well the sample represents the intended population. This is related to the generalisability of the study to the intended population (Sect.  8.3 ).
• Ecological validity : Discuss how well the study methods, materials and context approximate the real situation of interest. This is related to the practicality of the results to real life (Sect.  8.4 ).

The type of study often introduces some of these limitations (Chap.  4 ). All these issues should be addressed when considering the study limitations.

Almost every study has limitations. Identifying potential limitations, and discussing the likely impact they have on the interpretation of the study results, is important and ethical.

Different types of research studies have limitations. Experimental studies, in general, have higher internal validity than observational studies, since more of the research design in under the control of the researchers; for example, random allocation of treatments is possible to minimise confounding.

Only well-conducted experimental studies can show cause-and-effect relationships.

However, experimental studies may suffer from poor ecological validity; for instance, laboratory experiments are often conducted under controlled temperature and humidity. Many experiments also require that people be told about being in a study (due to ethics), and so internal validity may be comprised (the Hawthorne effect).

Example 8.1 (Retrofitting) Giandomenico, Papineau, and Rivers ( 2022 ) studied retro-fitting houses with energy-saving devices, and found large discrepancies in savings for observational studies ( \(12.2\) %) and experimental studies ( \(6.2\) %). The authors say that 'this finding reinforces the importance of using study designs with high internal validity to evaluate program savings' (p. 692).

8.2 Limitations related to internal validity

Internal validity refers to the extent to which a cause-and-effect relationship can be established in a study, eliminating other possible explanations (Sect.  3.1 ); that is, the effectiveness of the study using the sample. A discussion of the limitations of internal validity should cover, as appropriate: possible confounding and lurking variables variables; the impact of the Hawthorne, observer, placebo and carry-over effects; the impact of any other design decisions.

If any of these issues are likely to compromise internal validity, the implications on the interpretation of the results should be discussed. For example, if the participants were not blinded, this should be clearly stated, and the conclusion should indicate that the individuals in the study may have behaved differently than usual.

Example 8.2 (Study limitations) Axmann et al. ( 2020 ) randomly allocated Ugandan farmers to receive, or not receive, hybrid maize seeds. One potential threat to internal validity was that farmers receiving the hybrid seeds could share their seeds with their neighbours.

Hence, the researchers contacted the \(75\) farmers allocated to receive the hybrid seeds; none of the contacted farmers reported selling or giving seeds to other farmers. This extra step increased the internal validity of the study.

Maximizing internal validity in observational studies is more difficult than in experimental studies (e.g., random allocation is not possible). The internal validity of experimental studies involving people is often compromised because people must be informed that they are participating in a study.

Example 8.3 (Internal validity) In a study of the hand-hygiene practices of paramedics ( Barr et al. 2017 ) , self -reported hand-hygiene practices were very different than what was reported by peers . That is, how people self-report their behaviours may not align with how they actually behave, which influenced the internal validity of the study.

A study evaluated using a new therapy on elderly men, and listed some limitations of their study:

... the researcher was not blinded and had prior knowledge of the research aims, disease status, and intervention. As such, these could all have influenced data recording [...] The potential of reporting bias and observer bias could be reduced by implementing blinding in future studies. --- Kabata-Piżuch et al. ( 2021 ) , p. 10

8.3 Limitations related to external validity

External validity refers to the ability to generalise the findings made from the sample to the entire intended population (Sect.  6.1 ). For a study to be externally valid, it must first be internally valid: that is, if the study of not effective in the sample studied (i.e., internally valid), the results may not apply to the intended population either.

External validity refers to how well the sample is likely to represent the intended population in the RQ.

If the population is Californians, then the study is externally valid if the sample is representative of Californians The results do not have to apply to people in the rest of the United States (though this can be commented on, too) to be externally valid. The intended population is Californians .

External validity depends on how the sample was obtained. Results from random samples (Sect.  6.5 ) are likely to generalise to the population and be externally valid. (The analyses in this book assume all samples are simple random samples .) Furthermore, results from approximately representative samples (Sect.  6.6 ) may generalise to the population and be externally valid if those in the study are not obviously different than those not in the study.

Any inclusion criteria, exclusion criteria or control variables may also limit the external validity of the study.

Example 8.4 (External validity) A New Zealand study ( Gammon et al. 2012 ) identified (for well-documented reasons) a population of interest: 'women of South Asian origin living in New Zealand' (p. 21). The women in the sample were 'women of South Asian origin [...] recruited using a convenience sample method throughout Auckland' (p. 21).

The results may not generalise to the intended population ( all women of South Asian origin living in New Zealand) because all the women in the sample came from Auckland, and the sample was not a random sample from this population anyway. The study was still useful however, since we have still learnt information about the population that the is represented by the sample, which may be similar to the intended population.

Example 8.5 (Using biochar) Farrar et al. ( 2018 ) studied growing ginger using biochar on one farm at Mt Mellum, Australia. The results may only generalise to growing ginger at Mt Mellum, but since ginger is usually grown in similar types of climates and soils, the results may apply to other ginger farms also.

8.4 Limitations related to ecological validity

The likely practicality of the study results in the real world should also be discussed. This is called ecological validity .

Definition 8.1 (Ecological validity) A study is ecologically valid if the study methods, materials and context closely approximate the real situation of interest.

Studies don't need to be ecologically valid to be useful; much can be learnt under special conditions, as long as the potential limitations are understood when applying the results to the real world. The ecological validity of experimental studies may be compromised because the experimental conditions are sometimes highly controlled (for good reason).

Example 8.6 (Ecological validity) Consider a study to determine the proportion of people that buy coffee in a reusable cup. People could be asked about their behaviour. This study may not be ecologically valid, as what people do may not align with what they say they will do.

An alternative study could watch people buy coffee at various coffee shops, and record what people do in practice. This second study is more likely to be ecologically valid , as real-world behaviour is observed.

A study observed the effect of using high-mounted rear brake lights ( Kahane and Hertz 1998 ) , which are now commonplace. The American study showed that such lights reduced rear-end collisions by about \(50\) %. However, after making these lights mandatory, rear-end collisions reduced by only \(5\) %. Why?

8.5 Chapter summary

The limitations in a study need to be identified, and may be related to:

• internal validity (effectiveness): how well the study is conducted within the sample, isolating the relationship of interest.
• external validity (generalisability): how well the sample results are likely to apply to the intended population.
• ecological validity (practicality): how well the results may apply to the real-world situation of interest.

8.6 Quick review questions

Are the following statements true or false ?

• When interpreting the results of a study, the steps taken to maximize internal validity should be evaluated. TRUE FALSE
• If studies are not externally valid, then they are not useful. TRUE FALSE
• When interpreting the results of a study, the steps taken to maximize external validity do not need to be evaluated. TRUE FALSE
• When interpreting the results of a study, ecological validity is about the impact of the study on the environment. TRUE FALSE

8.7 Exercises

Answers to odd-numbered exercises are available in App.  E .

Exercise 8.1 A research study examined how people can save energy through lighting choices ( Gentile 2022 ) . The study states (p. 9) that the results 'are limited to the specific study and cannot be easily projected to other similar settings'.

What type of validity is being discussed here?

Exercise 8.2 Fill the blanks with the correct word: internal , external or ecological .

When interpreting the results of studies, we consider the practicality ( internal external ecological validity), the generalizability ( internal external ecological validity) and the effectiveness ( internal external ecological validity).

Exercise 8.3 A student project asked if 'the percentage of word retention is higher in male students than female students'. When discussing external validity , the students stated:

We cannot say whether or not the general public have better or worse word retention compared to the students that we will be studying.

Why is the statement not relevant in a discussion of external validity?

Exercise 8.4 Yeh et al. ( 2018 ) conducted an experimental study to 'determine if using a parachute prevents death or major traumatic injury when jumping from an aircraft'.

The researchers randomised \(23\) volunteers into one of two groups: wearing a parachute, or wearing an empty backpack. The response variable was a measurement of death or major traumatic injury upon landing. From the study, death or major injury was the same in both groups (0% for each group). However, the study used 'small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps' (p. 1).

Discuss the internal, external and ecological validity based on this information.

Exercise 8.5 A study examined how well hospital patients sleep at night ( Delaney et al. 2018 ) . The researchers state that 'convenience sampling was used to recruit patients' (p. 2). Later, the researchers state (p. 7):

... while most healthy individuals sleep primarily or exclusively at night, it is important to consider that patients requiring hospitalization will likely require some daytime nap periods. This study looks at sleep only in the night-time period \(22\) : \(00\) -- \(07\) : \(00\) h, without the context of daytime sleep considered.

Exercise 8.6 Botelho et al. ( 2019 ) examined the food choices made when subjects were asked to shop for ingredients to make a last-minute meal. Half were told to prepare a 'healthy meal', and the other half told just to prepare a 'meal'. The authors stated (p. 436):

Another limitation is that results report findings from a simulated purchase. As participants did not have to pay for their selection, actual choices could be different. Participants may also have not behaved in their usual manner since they were taking part in a research study...

Exercise 8.7 D. Johnson et al. ( 2018 ) studied the use of over-the-counter menthol cough-drops in people with a cough. One conclusion from the observational study of \(548\) people was that, taking 'too many cough drops [...] may actually make coughs more severe', as one author explained in an interview about the study Critique this statement.

Exercise 8.8 Suppose a group of student group was studying this RQ:

Among Australians, is the average serum cholesterol concentration different for smokers and non-smokers?

The students gave the following information about their study. Explain why each of these statements is incorrect.

• The design is observational, as we cannot manipulate each person's serum cholesterol.
• The Outcome is 'the average serum cholesterol concentration for smokers and non-smokers'.
• The study is not externally valid, as the results may not apply to all people in the world.
• The response variable is serum cholesterol.
• In this experiment, the population is 'Australians'.
• The data file will have two columns: one for smokers, and one for non-smokers.
• 'Whether or not the person owns a cat' is likely to be a confounding variable.
• The observer effect is not relevant, as the participants will know they are involved in a study.

Exercise 8.9 Delarue et al. ( 2019 ) discuss studies where subjects rate the taste of new food products. They note that taste-testing studies should be externally and internally valid (p. 78):

However, even with good internal and external validity, these studies often result in a 'high rate of failures of new launched products'.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Crit Care Med
• v.23(Suppl 4); 2019 Dec

Understanding Research Study Designs

Priya ranganathan.

Department of Anesthesiology, Critical Care and Pain, Tata Memorial Hospital, Mumbai, Maharashtra, India

In this article, we will look at the important features of various types of research study designs used commonly in biomedical research.

How to cite this article

Ranganathan P. Understanding Research Study Designs. Indian J Crit Care Med 2019;23(Suppl 4):S305–S307.

We use a variety of research study designs in biomedical research. In this article, the main features of each of these designs are summarized.

TERMS USED IN RESEARCH DESIGNS

Exposure vs outcome.

Exposure refers to any factor that may be associated with the outcome of interest. It is also called the predictor variable or independent variable or risk factor. Outcome refers to the variable that is studied to assess the impact of the exposure on the population. It is also known as the predicted variable or the dependent variable. For example, in a study looking at nerve damage after organophosphate (OPC) poisoning, the exposure would be OPC and the outcome would be nerve damage.

Longitudinal vs Transversal Studies

In longitudinal studies, participants are followed over time to determine the association between exposure and outcome (or outcome and exposure). On the other hand, in transversal studies, observations about exposure and outcome are made at a single point in time.

Forward vs Backward Directed Studies

In forward-directed studies, the direction of enquiry moves from exposure to outcome. In backward-directed studies, the line of enquiry starts with outcome and then determines exposure.

Prospective vs Retrospective Studies

In prospective studies, the outcome has not occurred at the time of initiation of the study. The researcher determines exposure and follows participants into the future to assess outcomes. In retrospective studies, the outcome of interest has already occurred when the study commences.

CLASSIFICATION OF STUDY DESIGNS

Broadly, study designs can be classified as descriptive or analytical (inferential) studies.

Descriptive Studies

Descriptive studies describe the characteristics of interest in the study population (also referred to as sample, to differentiate it from the entire population in the universe). These studies do not have a comparison group. The simplest type of descriptive study is the case report. In a case report, the researcher describes his/her experience with symptoms, signs, diagnosis, or treatment of a patient. Sometimes, a group of patients having a similar experience may be grouped to form a case series.

Case reports and case series form the lowest level of evidence in biomedical research and, as such, are considered hypothesis-generating studies. However, they are easy to write and may be a good starting point for the budding researcher. The recognition of some important associations in the field of medicine—such as that of thalidomide with phocomelia and Kaposi's sarcoma with HIV infection—resulted from case reports and case series. The reader can look up several published case reports and case series related to complications after OPC poisoning. 1 , 2

Analytical (Inferential) Studies

Analytical or inferential studies try to prove a hypothesis and establish an association between an exposure and an outcome. These studies usually have a comparator group. Analytical studies are further classified as observational or interventional studies.

In observational studies, there is no intervention by the researcher. The researcher merely observes outcomes in different groups of participants who, for natural reasons, have or have not been exposed to a particular risk factor. Examples of observational studies include cross-sectional, case–control, and cohort studies.

Cross-sectional Studies

These are transversal studies where data are collected from the study population at a single point in time. Exposure and outcome are determined simultaneously. Cross-sectional studies are easy to conduct, involve no follow-up, and need limited resources. They offer useful information on prevalence of health conditions and possible associations between risk factors and outcomes. However, there are two major limitations of cross-sectional studies. First, it may not be possible to establish a clear cause–benefit relationship. For example, in a study of association between colon cancer and dietary fiber intake, it may be difficult to establish whether the low fiber intake preceded the symptoms of colon cancer or whether the symptoms of colon cancer resulted in a change in dietary fiber intake. Another important limitation of cross-sectional studies is survival bias. For example, in a study looking at alcohol intake vs mortality due to chronic liver disease, among the participants with the highest alcohol intake, several may have died of liver disease; this will not be picked up by the study and will give biased results. An example of a cross-sectional study is a survey on nurses’ knowledge and practices of initial management of acute poisoning. 3

Case–control Studies

Case–control studies are backward-directed studies. Here, the direction of enquiry begins with the outcome and then proceeds to exposure. Case–control studies are always retrospective, i.e., the outcome of interest has occurred when the study begins. The researcher identifies participants who have developed the outcome of interest (cases) and chooses matching participants who do not have the outcome (controls). Matching is done based on factors that are likely to influence the exposure or outcome (e.g., age, gender, socioeconomic status). The researcher then proceeds to determine exposure in cases and controls. If cases have a higher incidence of exposure than controls, it suggests an association between exposure and outcome. Case–control studies are relatively quick to conduct, need limited resources, and are useful when the outcome is rare. They also allow the researcher to study multiple exposures for a particular outcome. However, they have several limitations. First, matching of cases with controls may not be easy since many unknown confounders may affect exposure and outcome. Second, there may be biased in the way the history of exposure is determined in cases vs controls; one way to overcome this is to have a blinded assessor determining the exposure using a standard technique (e.g., a standardized questionnaire). However, despite this, it has been shown that cases are far more likely than controls to recall history of exposure—the “recall bias.” For example, mothers of babies born with congenital anomalies may provide a more detailed history of drugs ingested during their pregnancy than those with normal babies. Also, since case-control studies do not begin with a population at risk, it is not possible to determine the true risk of outcome. Instead, one can only calculate the odds of association between exposure and outcome.

Kendrick and colleagues designed a case–control study to look at the association between domestic poison prevention practices and medically attended poisoning in children. They identified children presenting with unintentional poisoning at home (cases with the outcome), matched them with community participants (controls without the outcome), and then elicited data from parents and caregivers on home safety practices (exposure). 4

Cohort Studies

Cohort studies resemble clinical trials except that the exposure is naturally determined instead of being decided by the investigator. Here, the direction of enquiry begins with the exposure and then proceeds to outcome. The researcher begins with a group of individuals who are free of outcome at baseline; of these, some have the exposure (study cohort) while others do not (control group). The groups are followed up over a period of time to determine occurrence of outcome. Cohort studies may be prospective (involving a period of follow-up after the start of the study) or retrospective (e.g., using medical records or registry data). Cohort studies are considered the strongest among the observational study designs. They provide proof of temporal relationship (exposure occurred before outcome), allow determination of risk, and permit multiple outcomes to be studied for a single exposure. However, they are expensive to conduct and time-consuming, there may be several losses to follow-up, and they are not suitable for studying rare outcomes. Also, there may be unknown confounders other than the exposure affecting the occurrence of the outcome.

Jayasinghe conducted a cohort study to look at the effect of acute organophosphorus poisoning on nerve function. They recruited 70 patients with OPC poisoning (exposed group) and 70 matched controls without history of pesticide exposure (unexposed controls). Participants were followed up or 6 weeks for neurophysiological assessments to determine nerve damage (outcome). Hung carried out a retrospective cohort study using a nationwide research database to look at the long-term effects of OPC poisoning on cardiovascular disease. From the database, he identified an OPC-exposed cohort and an unexposed control cohort (matched for gender and age) from several years back and then examined later records to look at the development of cardiovascular diseases in both groups. 5

Interventional Studies

In interventional studies (also known as experimental studies or clinical trials), the researcher deliberately allots participants to receive one of several interventions; of these, some may be experimental while others may be controls (either standard of care or placebo). Allotment of participants to a particular treatment arm is carried out through the process of randomization, which ensures that every participant has a similar chance of being in any of the arms, eliminating bias in selection. There are several other aspects crucial to the validity of the results of a clinical trial such as allocation concealment, blinding, choice of control, and statistical analysis plan. These will be discussed in a separate article.

The randomized controlled clinical trial is considered the gold standard for evaluating the efficacy of a treatment. Randomization leads to equal distribution of known and unknown confounders between treatment arms; therefore, we can be reasonably certain that any difference in outcome is a treatment effect and not due to other factors. The temporal sequence of cause and effect is established. It is possible to determine risk of the outcome in each treatment arm accurately. However, randomized controlled trials have their limitations and may not be possible in every situation. For example, it is unethical to randomize participants to an intervention that is likely to cause harm—e.g., smoking. In such cases, well-designed observational studies are the only option. Also, these trials are expensive to conduct and resource-intensive.

In a randomized controlled trial, Li et al. randomly allocated patients of paraquat poisoning to receive either conventional therapy (control group) or continuous veno-venous hemofiltration (intervention). Patients were followed up to look for mortality or other adverse events (outcome). 6

Researchers need to understand the features of different study designs, with their advantages and limitations so that the most appropriate design can be chosen for a particular research question. The Centre for Evidence Based Medicine offers an useful tool to determine the type of research design used in a particular study. 7

Source of support: Nil

Conflict of interest: None

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on September 5, 2024 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

You might have to write up a research design as a standalone assignment, or it might be part of a larger   research proposal or other project. In either case, you should carefully consider which methods are most appropriate and feasible for answering your question.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2024, September 05). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved September 25, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, what is your plagiarism score.

Writing Limitations of Research Study — 4 Reasons Why It Is Important!

It is not unusual for researchers to come across the term limitations of research during their academic paper writing. More often this is interpreted as something terrible. However, when it comes to research study, limitations can help structure the research study better. Therefore, do not underestimate significance of limitations of research study.

Allow us to take you through the context of how to evaluate the limits of your research and conclude an impactful relevance to your results.

Table of Contents

What Are the Limitations of a Research Study?

Every research has its limit and these limitations arise due to restrictions in methodology or research design.  This could impact your entire research or the research paper you wish to publish. Unfortunately, most researchers choose not to discuss their limitations of research fearing it will affect the value of their article in the eyes of readers.

However, it is very important to discuss your study limitations and show it to your target audience (other researchers, journal editors, peer reviewers etc.). It is very important that you provide an explanation of how your research limitations may affect the conclusions and opinions drawn from your research. Moreover, when as an author you state the limitations of research, it shows that you have investigated all the weaknesses of your study and have a deep understanding of the subject. Being honest could impress your readers and mark your study as a sincere effort in research.

Why and Where Should You Include the Research Limitations?

The main goal of your research is to address your research objectives. Conduct experiments, get results and explain those results, and finally justify your research question . It is best to mention the limitations of research in the discussion paragraph of your research article.

At the very beginning of this paragraph, immediately after highlighting the strengths of the research methodology, you should write down your limitations. You can discuss specific points from your research limitations as suggestions for further research in the conclusion of your thesis.

1. Common Limitations of the Researchers

Limitations that are related to the researcher must be mentioned. This will help you gain transparency with your readers. Furthermore, you could provide suggestions on decreasing these limitations in you and your future studies.

2. Limited Access to Information

Your work may involve some institutions and individuals in research, and sometimes you may have problems accessing these institutions. Therefore, you need to redesign and rewrite your work. You must explain your readers the reason for limited access.

3. Limited Time

All researchers are bound by their deadlines when it comes to completing their studies. Sometimes, time constraints can affect your research negatively. However, the best practice is to acknowledge it and mention a requirement for future study to solve the research problem in a better way.

4. Conflict over Biased Views and Personal Issues

Biased views can affect the research. In fact, researchers end up choosing only those results and data that support their main argument, keeping aside the other loose ends of the research.

Types of Limitations of Research

Before beginning your research study, know that there are certain limitations to what you are testing or possible research results. There are different types that researchers may encounter, and they all have unique characteristics, such as:

1. Research Design Limitations

Certain restrictions on your research or available procedures may affect your final results or research outputs. You may have formulated research goals and objectives too broadly. However, this can help you understand how you can narrow down the formulation of research goals and objectives, thereby increasing the focus of your study.

2. Impact Limitations

Even if your research has excellent statistics and a strong design, it can suffer from the influence of the following factors:

• Presence of increasing findings as researched
• Being population specific
• A strong regional focus.

3. Data or statistical limitations

In some cases, it is impossible to collect sufficient data for research or very difficult to get access to the data. This could lead to incomplete conclusion to your study. Moreover, this insufficiency in data could be the outcome of your study design. The unclear, shabby research outline could produce more problems in interpreting your findings.

How to Correctly Structure Your Research Limitations?

There are strict guidelines for narrowing down research questions, wherein you could justify and explain potential weaknesses of your academic paper. You could go through these basic steps to get a well-structured clarity of research limitations:

• Declare that you wish to identify your limitations of research and explain their importance,
• Provide the necessary depth, explain their nature, and justify your study choices.
• Write how you are suggesting that it is possible to overcome them in the future.

In this section, your readers will see that you are aware of the potential weaknesses in your business, understand them and offer effective solutions, and it will positively strengthen your article as you clarify all limitations of research to your target audience.

Know that you cannot be perfect and there is no individual without flaws. You could use the limitations of research as a great opportunity to take on a new challenge and improve the future of research. In a typical academic paper, research limitations may relate to:

1. Formulating your goals and objectives

If you formulate goals and objectives too broadly, your work will have some shortcomings. In this case, specify effective methods or ways to narrow down the formula of goals and aim to increase your level of study focus.

2. Application of your data collection methods in research

If you do not have experience in primary data collection, there is a risk that there will be flaws in the implementation of your methods. It is necessary to accept this, and learn and educate yourself to understand data collection methods.

3. Sample sizes

This depends on the nature of problem you choose. Sample size is of a greater importance in quantitative studies as opposed to qualitative ones. If your sample size is too small, statistical tests cannot identify significant relationships or connections within a given data set.

You could point out that other researchers should base the same study on a larger sample size to get more accurate results.

4. The absence of previous studies in the field you have chosen

Writing a literature review is an important step in any scientific study because it helps researchers determine the scope of current work in the chosen field. It is a major foundation for any researcher who must use them to achieve a set of specific goals or objectives.

However, if you are focused on the most current and evolving research problem or a very narrow research problem, there may be very little prior research on your topic. For example, if you chose to explore the role of Bitcoin as the currency of the future, you may not find tons of scientific papers addressing the research problem as Bitcoins are only a new phenomenon.

It is important that you learn to identify research limitations examples at each step. Whatever field you choose, feel free to add the shortcoming of your work. This is mainly because you do not have many years of experience writing scientific papers or completing complex work. Therefore, the depth and scope of your discussions may be compromised at different levels compared to academics with a lot of expertise. Include specific points from limitations of research. Use them as suggestions for the future.

Have you ever faced a challenge of writing the limitations of research study in your paper? How did you overcome it? What ways did you follow? Were they beneficial? Let us know in the comments below!

Frequently Asked Questions

Setting limitations in our study helps to clarify the outcomes drawn from our research and enhance understanding of the subject. Moreover, it shows that the author has investigated all the weaknesses in the study.

Scope is the range and limitations of a research project which are set to define the boundaries of a project. Limitations are the impacts on the overall study due to the constraints on the research design.

Limitation in research is an impact of a constraint on the research design in the overall study. They are the flaws or weaknesses in the study, which may influence the outcome of the research.

1. Limitations in research can be written as follows: Formulate your goals and objectives 2. Analyze the chosen data collection method and the sample sizes 3. Identify your limitations of research and explain their importance 4. Provide the necessary depth, explain their nature, and justify your study choices 5. Write how you are suggesting that it is possible to overcome them in the future

Excellent article ,,,it has helped me big

This is very helpful information. It has given me an insight on how to go about my study limitations.

Good comments and helpful

the topic is well covered

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Publishing Research
• Trending Now
• Understanding Ethics

Understanding the Impact of Retractions on Research Integrity – A global study

As we reach the midway point of 2024, ‘Research Integrity’ remains one of the hot…

• Diversity and Inclusion

The Silent Struggle: Confronting gender bias in science funding

In the 1990s, Dr. Katalin Kariko’s pioneering mRNA research seemed destined for obscurity, doomed by…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Addressing Barriers in Academia: Navigating unconscious biases in the Ph.D. journey

In the journey of academia, a Ph.D. marks a transitional phase, like that of a…

• Manuscripts & Grants
• Reporting Research

Unraveling Research Population and Sample: Understanding their role in statistical inference

Research population and sample serve as the cornerstones of any scientific inquiry. They hold the…

Research Problem Statement — Find out how to write an impactful one!

How to Develop a Good Research Question? — Types & Examples

5 Effective Ways to Avoid Ghostwriting for Busy Researchers

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

• Industry News
• AI in Academia
• Career Corner
• Infographics
• Expert Video Library
• Other Resources
• Enago Learn
• Upcoming & On-Demand Webinars
• Peer Review Week 2024
• Open Access Week 2023
• Conference Videos
• Enago Report
• Journal Finder
• Enago Plagiarism & AI Grammar Check
• Editing Services
• Publication Support Services
• Research Impact
• Translation Services
• Publication solutions
• AI-Based Solutions
• Thought Leadership
• Call for Articles
• Call for Speakers
• Author Training
• Edit Profile

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

Which among these features would you prefer the most in a peer review assistant?

Diving Deeper into Limitations and Delimitations

If you are working on a thesis, dissertation, or other formal research project, chances are your advisor or committee will ask you to address the delimitations of your study. When faced with this request, many students respond with a puzzled look and then go on to address what are actually the study’s limitations.

In a previous article , we covered what goes into the limitations, delimitations, and assumptions sections of your thesis or dissertation. Here, we will dive a bit deeper into the differences between limitations and delimitations and provide some helpful tips for addressing them in your research project—whether you are working on a quantitative or qualitative study.

Acknowledging Weaknesses vs. Defining Boundaries

These concepts are easy to get confused because both limitations and delimitations restrict (or limit) the questions you’ll be able to answer with your study, most notably in terms of generalizability.

However, the biggest difference between limitations and delimitations is the degree of control you have over them—that is, how much they are based in conscious, intentional choices you made in designing your study.

Limitations occur in all types of research and are, for the most part, outside the researcher’s control (given practical constraints, such as time, funding, and access to populations of interest). They are threats to the study’s internal or external validity.

Limitations may include things such as participant drop-out, a sample that isn’t entirely representative of the desired population, violations to the assumptions of parametric analysis (e.g., normality, homogeneity of variance), the limits of self-report, or the absence of reliability and validity data for some of your survey measures.

Limitations can get in the way of your being able to answer certain questions or draw certain types of inferences from your findings. Therefore, it’s important to acknowledge them upfront and make note of how they restrict the conclusions you’ll be able to draw from your study. Frequently, limitations can get in the way of our ability to generalize our findings to the larger populations or to draw causal conclusions, so be sure to consider these issues when you’re thinking about the potential limitations of your study.

Delimitations are also factors that can restrict the questions you can answer or the inferences you can draw from your findings. However, they are based on intentional choices you make a priori (i.e., as you’re designing the study) about where you’re going to draw the boundaries of your project. In other words, they define the project’s scope.

Like limitations, delimitations are a part of every research project, and this is not a bad thing. In fact, it’s very important! You can’t study everything at once. If you try to do so, your project is bound to get huge and unwieldy, and it will become a lot more difficult to interpret your results or come to meaningful conclusions with so many moving parts. You have to draw the line somewhere, and the delimitations are where you choose to draw these lines.

One of the clearest examples of a delimitation that applies to almost every research project is participant exclusion criteria. In conducting either a quantitative or a qualitative study, you will have to define your population of interest. Defining this population of interest means that you will need to articulate the boundaries of that population (i.e., who is not included). Those boundaries are delimitations.

For example, if you’re interested in understanding the experiences of elementary school teachers who have been implementing a new curriculum into their classrooms, you probably won’t be interviewing or sending a survey to any of the following people: non-teachers, high-school teachers, college professors, principals, parents of elementary school children, or the children themselves. Furthermore, you probably won’t be talking to elementary school teachers who have not yet had the experience of implementing the curriculum in question. You would probably only choose to gather data from elementary school teachers who have had this experience because that is who you’re interested in for the purposes of your study. Perhaps you’ll narrow your focus even more to elementary school teachers in a particular school district who have been teaching for a particular length of time. The possibilities can go on. These are choices you will need to make, both for practical reasons (i.e., the population you have access to) and for the questions you are trying to answer.

Of course, for this particular example, this does not mean that it wouldn’t be interesting to also know what principals think about the new curriculum. Or parents. Or elementary school children. It just means that, for the purposes of your project and your research questions, you’re interested in the experience of the teachers, so you’re excluding anyone who does not meet those criteria. Having delimitations to your population of interest also means that you won’t be able to answer any questions about the experiences of those other populations; this is ok because those populations are outside of the scope of your project . As interesting as their experiences might be, you can save these questions for another study. That is the part of the beauty of research: there will always be more studies to do, more questions to ask. You don’t have to (and can’t) do it all in one project.

Continuing with the previous example, for instance, let’s suppose that the problem you are most interested in addressing is the fact that we know relatively little about elementary school teachers’ experiences of implementing a new curriculum. Perhaps you believe that knowing more about teachers’ experiences could inform their training or help administrators know more about how to support their teachers. If the identified problem is our lack of knowledge about teachers’ experiences, and your research questions focus on better understanding these experiences, that means that you are choosing not to focus on other problems or questions, even those that may seem closely related. For instance, you are not asking how effective the new curriculum is in improving student test scores or graduation rates. You might think that would be a very interesting question, but it will have to wait for another study. In narrowing the focus of your research questions, you limit your ability to answer other questions, and again, that’s ok. These other questions may be interesting and important, but, again, they are beyond the scope of your project .

Common Examples of Limitations

While each study will have its own unique set of limitations, some limitations are more common in quantitative research, and others are more common in qualitative research.

In quantitative research, common limitations include the following:

– Participant dropout

– Small sample size, low power

– Non-representative sample

– Violations of statistical assumptions

– Non-experimental design, lack of manipulation of variables, lack of controls

– Potential confounding variables

– Measures with low (or unknown) reliability or validity

– Limits of an instrument to measure the construct of interest

– Data collection methods (e.g., self-report)

– Anything else that might limit the study’s internal or external validity

In qualitative research, common limitations include the following:

– Lack of generalizability of findings (not the goal of qualitative research, but still worth mentioning as a limitation)

– Inability to draw causal conclusions (again, not the goal of qualitative research, but still worth mentioning)

– Researcher bias/subjectivity (especially if there is only one coder)

– Limitations in participants’ ability/willingness to share or describe their experiences

– Any factors that might limit the rigor of data collection or analysis procedures

Common Examples of Delimitations

As noted above, the two most common sources of delimitations in both quantitative and qualitative research include the following:

– Inclusion/exclusion criteria (or how you define your population of interest)

– Research questions or problems you’ve chosen to examine

Several other common sources of delimitations include the following:

– Theoretical framework or perspective adopted

– Methodological framework or paradigm chosen (e.g., quantitative, qualitative, or mixed-methods)

– In quantitative research, the variables you’ve chosen to measure or manipulate (as opposed to others)

Whether you’re conducting a quantitative or qualitative study, you will (hopefully!) have chosen your research design because it is well suited to the questions you’re hoping to answer. Because these questions define the boundaries or scope of your project and thus point to its delimitations, your research design itself will also be related to these delimitations.

Questions to Ask Yourself

As you are considering the limitations and delimitations of your project, it can be helpful to ask yourself a few different questions.

Questions to help point out your study’s limitations :

1. If I had an unlimited budget, unlimited amounts of time, access to all possible populations, and the ability to manipulate as many variables as I wanted, how would I design my study differently to be better able to answer the questions I want to answer? (The ways in which your study falls short of this will point to its limitations.)

2. Are there design issues that get in the way of my being able to draw causal conclusions?

3. Are there sampling issues that get in the way of my being able to generalize my findings?

4. Are there issues related to the measures I’m using or the methods I’m using to collect data? Do I have concerns about participants telling the truth or being able to provide accurate responses to my questions?

5. Are there any other factors that might limit my study’s internal or external validity?

Questions that help point out your study’s delimitations :

1. What are my exclusion criteria? Who did I not include in my study, and why did I make this choice?

2. What questions did I choose not to address in my study? (Of course, the possibilities are endless here, but consider related questions that you chose not to address.)

3. In what ways did I narrow the scope of my study in order to hone in on a particular issue or question?

4. What other methodologies did I not use that might have allowed me to answer slightly different questions about the same topic?

How to Write About Limitations and Delimitations

Remember, having limitations and delimitations is not a bad thing. They’re present in even the most rigorous research. The important thing is to be aware of them and to acknowledge how they may impact your findings or the conclusions you can draw.

In fact, writing about them and acknowledging them gives you an opportunity to demonstrate that you can think critically about these aspects of your study and how they impact your findings, even if they were out of your control.

Keep in mind that your study’s limitations will likely point to important directions for future research. Therefore, when you’re getting ready to write about your recommendations for future research in your discussion, remember to refer back to your limitations section!

As you write about your delimitations in particular, remember that they are not weaknesses, and you don’t have to apologize for them. Good, strong research projects have clear boundaries. Also, keep in mind that you are the researcher and you can choose whatever delimitations you want for your study. You’re in control of the delimitations. You just have to be prepared—both in your discussion section and in your dissertation defense itself—to justify the choices you make and acknowledge how these choices impact your findings.

Browse More on PhDStudent

Everything You Need to Know About References and Citations: Part 1

When you conduct your research, it is important to record the details of all the information you find to provide accurate references, …

How to Write a Proposal: For a Master’s Thesis or Dissertation

Note: Many thanks to fellow PhDStudent blogger Ryan Krone for his contributions and insight to this post. Your thesis/dissertation proposal provides an …

How to Find Free Money for Graduate School Part 2

Getting into graduate school is already a challenge on its own, and funding the program once admitted is even harder. Graduate studies …

How to Find Free Money for Graduate School

You’ve finally earned your Bachelor’s degree and have made it into graduate school. Whether you already have massive student loans from undergrad …

Part 3 of How to Pick Your Defense Committee

  What strategies can a doctoral student employ to maneuver the trials and tribulations of a dissertation committee? In Part 2, we …

Best Dissertation Proofreading and Editing Tips to Make Your Work Spotless

You’ve heard this statement many times before: “the dissertation is the most important project you’ve ever worked on.” That may sound like …

Part 2 of How to Pick Your Defense Committee

Choosing a committee can be a daunting task for a doctoral student.  We’ve already covered two strategies that can help you through this …

Part 1 of How to Pick Your Defense Committee

So you’re ready to pick your committee members; there are a few things to keep in mind first—after all, it is a …

Intro To Series on How to Pick Your Defense Committee

Choosing the right defense committee can potentially be the difference between a smooth transition of receiving your doctoral degree or dodging bullets …

On Babies and Dissertations: Part 3

I recently had the experience of expecting my first baby a month before I graduated. Throughout the process, I accidentally learned several …

On Babies and Dissertations: Part 2

On Babies and Dissertations: Part 1

I recently had the experience of expecting my first baby a month before I graduated and accidentally learned tips on graduating on time with a …

Click here to cancel reply.

You must be logged in to post a comment.

Copyright © 2024 PhDStudent.com. All rights reserved. Designed by Divergent Web Solutions, LLC .

Research Limitations

It is for sure that your research will have some limitations and it is normal. However, it is critically important for you to be striving to minimize the range of scope of limitations throughout the research process.  Also, you need to provide the acknowledgement of your research limitations in conclusions chapter honestly.

It is always better to identify and acknowledge shortcomings of your work, rather than to leave them pointed out to your by your dissertation assessor. While discussing your research limitations, don’t just provide the list and description of shortcomings of your work. It is also important for you to explain how these limitations have impacted your research findings.

Your research may have multiple limitations, but you need to discuss only those limitations that directly relate to your research problems. For example, if conducting a meta-analysis of the secondary data has not been stated as your research objective, no need to mention it as your research limitation.

Research limitations in a typical dissertation may relate to the following points:

1. Formulation of research aims and objectives . You might have formulated research aims and objectives too broadly. You can specify in which ways the formulation of research aims and objectives could be narrowed so that the level of focus of the study could be increased.

2. Implementation of data collection method . Because you do not have an extensive experience in primary data collection (otherwise you would not be reading this book), there is a great chance that the nature of implementation of data collection method is flawed.

3. Sample size. Sample size depends on the nature of the research problem. If sample size is too small, statistical tests would not be able to identify significant relationships within data set. You can state that basing your study in larger sample size could have generated more accurate results. The importance of sample size is greater in quantitative studies compared to qualitative studies.

4. Lack of previous studies in the research area . Literature review is an important part of any research, because it helps to identify the scope of works that have been done so far in research area. Literature review findings are used as the foundation for the researcher to be built upon to achieve her research objectives.

However, there may be little, if any, prior research on your topic if you have focused on the most contemporary and evolving research problem or too narrow research problem. For example, if you have chosen to explore the role of Bitcoins as the future currency, you may not be able to find tons of scholarly paper addressing the research problem, because Bitcoins are only a recent phenomenon.

5. Scope of discussions . You can include this point as a limitation of your research regardless of the choice of the research area. Because (most likely) you don’t have many years of experience of conducing researches and producing academic papers of such a large size individually, the scope and depth of discussions in your paper is compromised in many levels compared to the works of experienced scholars.

You can discuss certain points from your research limitations as the suggestion for further research at conclusions chapter of your dissertation.

My e-book,  The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance  offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker

APA Citation Generator

MLA Citation Generator

Chicago Citation Generator

Vancouver Citation Generator

• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

Limitations of the Study – How to Write & Examples

What are the limitations of a study?

The limitations of a study are the elements of methodology or study design that impact the interpretation of your research results. The limitations essentially detail any flaws or shortcomings in your study. Study limitations can exist due to constraints on research design, methodology, materials, etc., and these factors may impact the findings of your study. However, researchers are often reluctant to discuss the limitations of their study in their papers, feeling that bringing up limitations may undermine its research value in the eyes of readers and reviewers.

In spite of the impact it might have (and perhaps because of it) you should clearly acknowledge any limitations in your research paper in order to show readers—whether journal editors, other researchers, or the general public—that you are aware of these limitations and to explain how they affect the conclusions that can be drawn from the research.

In this article, we provide some guidelines for writing about research limitations, show examples of some frequently seen study limitations, and recommend techniques for presenting this information. And after you have finished drafting and have received manuscript editing for your work, you still might want to follow this up with academic editing before submitting your work to your target journal.

Why do I need to include limitations of research in my paper?

Although limitations address the potential weaknesses of a study, writing about them toward the end of your paper actually strengthens your study by identifying any problems before other researchers or reviewers find them.

Furthermore, pointing out study limitations shows that you’ve considered the impact of research weakness thoroughly and have an in-depth understanding of your research topic. Since all studies face limitations, being honest and detailing these limitations will impress researchers and reviewers more than ignoring them.

Where should I put the limitations of the study in my paper?

Some limitations might be evident to researchers before the start of the study, while others might become clear while you are conducting the research. Whether these limitations are anticipated or not, and whether they are due to research design or to methodology, they should be clearly identified and discussed in the discussion section —the final section of your paper. Most journals now require you to include a discussion of potential limitations of your work, and many journals now ask you to place this “limitations section” at the very end of your article. 

Some journals ask you to also discuss the strengths of your work in this section, and some allow you to freely choose where to include that information in your discussion section—make sure to always check the author instructions of your target journal before you finalize a manuscript and submit it for peer review .

Limitations of the Study Examples

There are several reasons why limitations of research might exist. The two main categories of limitations are those that result from the methodology and those that result from issues with the researcher(s).

Common Methodological Limitations of Studies

Limitations of research due to methodological problems can be addressed by clearly and directly identifying the potential problem and suggesting ways in which this could have been addressed—and SHOULD be addressed in future studies. The following are some major potential methodological issues that can impact the conclusions researchers can draw from the research.

Issues with research samples and selection

Sampling errors occur when a probability sampling method is used to select a sample, but that sample does not reflect the general population or appropriate population concerned. This results in limitations of your study known as “sample bias” or “selection bias.”

For example, if you conducted a survey to obtain your research results, your samples (participants) were asked to respond to the survey questions. However, you might have had limited ability to gain access to the appropriate type or geographic scope of participants. In this case, the people who responded to your survey questions may not truly be a random sample.

Insufficient sample size for statistical measurements

When conducting a study, it is important to have a sufficient sample size in order to draw valid conclusions. The larger the sample, the more precise your results will be. If your sample size is too small, it will be difficult to identify significant relationships in the data.

Normally, statistical tests require a larger sample size to ensure that the sample is considered representative of a population and that the statistical result can be generalized to a larger population. It is a good idea to understand how to choose an appropriate sample size before you conduct your research by using scientific calculation tools—in fact, many journals now require such estimation to be included in every manuscript that is sent out for review.

Lack of previous research studies on the topic

Citing and referencing prior research studies constitutes the basis of the literature review for your thesis or study, and these prior studies provide the theoretical foundations for the research question you are investigating. However, depending on the scope of your research topic, prior research studies that are relevant to your thesis might be limited.

When there is very little or no prior research on a specific topic, you may need to develop an entirely new research typology. In this case, discovering a limitation can be considered an important opportunity to identify literature gaps and to present the need for further development in the area of study.

Methods/instruments/techniques used to collect the data

After you complete your analysis of the research findings (in the discussion section), you might realize that the manner in which you have collected the data or the ways in which you have measured variables has limited your ability to conduct a thorough analysis of the results.

For example, you might realize that you should have addressed your survey questions from another viable perspective, or that you were not able to include an important question in the survey. In these cases, you should acknowledge the deficiency or deficiencies by stating a need for future researchers to revise their specific methods for collecting data that includes these missing elements.

Common Limitations of the Researcher(s)

Study limitations that arise from situations relating to the researcher or researchers (whether the direct fault of the individuals or not) should also be addressed and dealt with, and remedies to decrease these limitations—both hypothetically in your study, and practically in future studies—should be proposed.

Limited access to data

If your research involved surveying certain people or organizations, you might have faced the problem of having limited access to these respondents. Due to this limited access, you might need to redesign or restructure your research in a different way. In this case, explain the reasons for limited access and be sure that your finding is still reliable and valid despite this limitation.

Time constraints

Just as students have deadlines to turn in their class papers, academic researchers might also have to meet deadlines for submitting a manuscript to a journal or face other time constraints related to their research (e.g., participants are only available during a certain period; funding runs out; collaborators move to a new institution). The time available to study a research problem and to measure change over time might be constrained by such practical issues. If time constraints negatively impacted your study in any way, acknowledge this impact by mentioning a need for a future study (e.g., a longitudinal study) to answer this research problem.

Conflicts arising from cultural bias and other personal issues

Researchers might hold biased views due to their cultural backgrounds or perspectives of certain phenomena, and this can affect a study’s legitimacy. Also, it is possible that researchers will have biases toward data and results that only support their hypotheses or arguments. In order to avoid these problems, the author(s) of a study should examine whether the way the research problem was stated and the data-gathering process was carried out appropriately.

Steps for Organizing Your Study Limitations Section

When you discuss the limitations of your study, don’t simply list and describe your limitations—explain how these limitations have influenced your research findings. There might be multiple limitations in your study, but you only need to point out and explain those that directly relate to and impact how you address your research questions.

We suggest that you divide your limitations section into three steps: (1) identify the study limitations; (2) explain how they impact your study in detail; and (3) propose a direction for future studies and present alternatives. By following this sequence when discussing your study’s limitations, you will be able to clearly demonstrate your study’s weakness without undermining the quality and integrity of your research.

Step 1. Identify the limitation(s) of the study

• This part should comprise around 10%-20% of your discussion of study limitations.

The first step is to identify the particular limitation(s) that affected your study. There are many possible limitations of research that can affect your study, but you don’t need to write a long review of all possible study limitations. A 200-500 word critique is an appropriate length for a research limitations section. In the beginning of this section, identify what limitations your study has faced and how important these limitations are.

You only need to identify limitations that had the greatest potential impact on: (1) the quality of your findings, and (2) your ability to answer your research question.

Step 2. Explain these study limitations in detail

• This part should comprise around 60-70% of your discussion of limitations.

After identifying your research limitations, it’s time to explain the nature of the limitations and how they potentially impacted your study. For example, when you conduct quantitative research, a lack of probability sampling is an important issue that you should mention. On the other hand, when you conduct qualitative research, the inability to generalize the research findings could be an issue that deserves mention.

Explain the role these limitations played on the results and implications of the research and justify the choice you made in using this “limiting” methodology or other action in your research. Also, make sure that these limitations didn’t undermine the quality of your dissertation .

Step 3. Propose a direction for future studies and present alternatives (optional)

• This part should comprise around 10-20% of your discussion of limitations.

After acknowledging the limitations of the research, you need to discuss some possible ways to overcome these limitations in future studies. One way to do this is to present alternative methodologies and ways to avoid issues with, or “fill in the gaps of” the limitations of this study you have presented.  Discuss both the pros and cons of these alternatives and clearly explain why researchers should choose these approaches.

Make sure you are current on approaches used by prior studies and the impacts they have had on their findings. Cite review articles or scientific bodies that have recommended these approaches and why. This might be evidence in support of the approach you chose, or it might be the reason you consider your choices to be included as limitations. This process can act as a justification for your approach and a defense of your decision to take it while acknowledging the feasibility of other approaches.

P hrases and Tips for Introducing Your Study Limitations in the Discussion Section

The following phrases are frequently used to introduce the limitations of the study:

• “There may be some possible limitations in this study.”
• “The findings of this study have to be seen in light of some limitations.”
•  “The first is the…The second limitation concerns the…”
•  “The empirical results reported herein should be considered in the light of some limitations.”
• “This research, however, is subject to several limitations.”
• “The primary limitation to the generalization of these results is…”
• “Nonetheless, these results must be interpreted with caution and a number of limitations should be borne in mind.”
• “As with the majority of studies, the design of the current study is subject to limitations.”
• “There are two major limitations in this study that could be addressed in future research. First, the study focused on …. Second ….”

For more articles on research writing and the journal submissions and publication process, visit Wordvice’s Academic Resources page.

And be sure to receive professional English editing and proofreading services , including paper editing services , for your journal manuscript before submitting it to journal editors.

Wordvice Resources

Proofreading & Editing Guide

Writing the Results Section for a Research Paper

How to Write a Literature Review

Research Writing Tips: How to Draft a Powerful Discussion Section

How to Captivate Journal Readers with a Strong Introduction

Tips That Will Make Your Abstract a Success!

APA In-Text Citation Guide for Research Writing

Additional Resources

• Diving Deeper into Limitations and Delimitations (PhD student)
• Organizing Your Social Sciences Research Paper: Limitations of the Study (USC Library)
• Research Limitations (Research Methodology)
• How to Present Limitations and Alternatives (UMASS)

Article References

Pearson-Stuttard, J., Kypridemos, C., Collins, B., Mozaffarian, D., Huang, Y., Bandosz, P.,…Micha, R. (2018). Estimating the health and economic effects of the proposed US Food and Drug Administration voluntary sodium reformulation: Microsimulation cost-effectiveness analysis. PLOS. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002551

Xu, W.L, Pedersen, N.L., Keller, L., Kalpouzos, G., Wang, H.X., Graff, C,. Fratiglioni, L. (2015). HHEX_23 AA Genotype Exacerbates Effect of Diabetes on Dementia and Alzheimer Disease: A Population-Based Longitudinal Study. PLOS. Retrieved from https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001853

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

Design flaws to avoid.

• Purpose of Guide
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The research design establishes the decision-making processes, conceptual structure of investigation, and methods of analysis used to address the study's research problem. Taking the time to develop a thorough research design helps to organize your thoughts, sets the boundaries of your study, maximizes the reliability of your findings, and avoids misleading or incomplete conclusions. Therefore, if any aspect of your research design is flawed or under-developed, the quality and reliability of your final results, as well as the overall value of your study, will be diminished.

In no particular order, here are some common problems to avoid when designing a research study. Some are general issues you should think about as your organize your thoughts [e.g., developing a good research problem] while other issues must be explicitly addressed in your paper [e.g., describe study limitations].

• Lack of Specificity -- do not describe the investigative aspects of your study in overly-broad generalities. Avoid using vague qualifiers, such as, extremely, very, entirely, completely, etc. It's important that you design a study that describes the process of investigation in clear and concise terms. Otherwise, the reader cannot be certain about what you intend to do.
• Poorly Defined Research Problem -- the starting point of most new research in the social and behavioral sciences is to formulate a problem problem and begin the process of developing questions that address the problem. Your paper should outline and explicitly delimit the problem and state what you intend to investigate because this will determine what research design you will use [identifying the research problem always precedes choice of design].
• Lack of Theoretical Framework -- the theoretical framework represents the conceptual foundation of your study. Therefore, your research design should include an explicit set of logically derived hypotheses, basic postulates, or assumptions that can be tested in relation to the research problem. More information about developing a theoretical framework can be found here .
• Significance -- this refers to describing what value your study has in contributing to understanding the research problem. In the social and behavioral sciences, arguing why a study is significant is framed in the context of clearly answering the "So What?" question [e.g., "This study compares key areas of economic relations among three Central American countries." So what?]. In describing the research design, state why your study is important and how it contributes to the larger body of studies about the topic being investigated.
• Relationship between Past Research and Your Paper -- do not simply offer a summary description of prior research. Your literature review should include an explicit statement linking the results of prior research to the research you are about to undertake. This can be done, for example, by identifying basic weaknesses in previous studies, filling specific gaps in knowledge, or describing how your study contributes a unique or different perspective or approach to the problem.
• Provincialism -- this refers to designing a narrowly applied scope, geographical area, sampling, or method of analysis that restricts your ability to create meaningful outcomes and, by extension, obtaining results that are relevant and possibly transferable to understanding phenomena in other settings. The scope of your research should be clearly defined, but not to the point of where you cannot extrapolate findings in a meaningful way applied to better understanding the research problem.
• Objectives, Hypotheses, or Questions -- your research design should include one or more questions or hypotheses that you are attempting to answer about the research problem. These should be clearly articulated and closely tied to the overall aims of your paper. Although there is no rule regarding the number of questions or hypotheses associated with a research problem, most studies in the social and behavioral sciences address between two and five key questions.
• Poor Methodological Approach -- the design must include a well-developed and transparent plan for how you intend to collect or generate data and how it will be analyzed. Ensure that the method used to gather information for analysis is aligned with the topic of inquiry and the underlying research questions to be addressed.
• Proximity Sampling -- this refers to using a sample that is based not on the purpose of your study, but rather, is based on the proximity of a particular group of subjects. The units of analysis, whether they be persons, places, events, or things, should not be based solely on ease of access and convenience. Note that this does not mean you should not use units of analysis that are easy to access. The point is that this closeness to data or information cannot be the sole factor that determines the purpose of your study.
• Techniques or Instruments -- be clear in describing the techniques [e.g., semi-structured interviews; Linear Regression Analysis] or instruments [e.g., questionnaire; online survey] used to gather data. Your research design should note how the technique or instrument will provide reasonably reliable data to answer the questions associated with the research problem.
• Statistical Treatment -- in quantitative studies, you must give a complete description of how you will organize the raw data for analysis. In most cases, this involves describing the data through the measures of central tendencies like mean, median, and mode that help the researcher explain how the data are concentrated and, thus, lead to meaningful interpretations of key trends or patterns found within that data.
• Vocabulary -- research often contains jargon and specialized language that the reader is presumably familiar with. However, avoid overuse of technical or pseudo-technical terminology as part of describing your research design. Problems with vocabulary also can refer to the use of popular terms, cliche's, or culture-specific language that is inappropriate for academic writing. More information about academic writing can be found here .
• Ethical Dilemmas -- in the methods section of qualitative research studies, your design must document how you intend to minimize risk for participants [a.k.a., "respondents", "human subjects"] during stages of data gathering while, at the same time, still being able to adequately address the research problem. Failure to do so can lead the reader to question the validity and objectivity of your entire study.
• Limitations of Study -- all studies have limitations. Your research design should anticipate and explain the reasons why these limitations exist and clearly describe the extent of missing data. It is important to include a statement concerning what impact these limitations may have on the validity of your results and how you helped to ameliorate the significance of these limitations. For more details about study limitations, go here .

Butin, Dan W. The Education Dissertation A Guide for Practitioner Scholars . Thousand Oaks, CA: Corwin, 2010; Carter, Susan. Structuring Your Research Thesis . New York: Palgrave Macmillan, 2012; Creswell, John W. and J. David Creswell. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . 5th edition. Thousand Oaks, CA: Sage, 2018;Gorard, Stephen. Research Design: Creating Robust Approaches for the Social Sciences . Thousand Oaks, CA: Sage, 2013; Lunenburg, Frederick C. Writing a Successful Thesis or Dissertation: Tips and Strategies for Students in the Social and Behavioral Sciences . Thousand Oaks, CA: Corwin Press, 2008. Lunsford, Andrea and Robert Connors. The St. Martin’s Handbook . 3rd ed. New York: St. Martin’s Press, 1995.

• << Previous: Types of Research Designs
• Next: Independent and Dependent Variables >>
• Last Updated: Sep 17, 2024 10:59 AM
• URL: https://libguides.usc.edu/writingguide

Organizing Academic Research Papers: Limitations of the Study

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The limitations of the study are those characteristics of design or methodology that impacted or influenced the application or interpretation of the results of your study. They are the constraints on generalizability and utility of findings that are the result of the ways in which you chose to design the study and/or the method used to establish internal and external validity.

Importance of...

Always acknowledge a study's limitations. It is far better for you to identify and acknowledge your study’s limitations than to have them pointed out by your professor and be graded down because you appear to have ignored them.

Keep in mind that acknowledgement of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgement of a study's limitations also provides you with an opportunity to demonstrate to your professor that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitiations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the findings and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in your paper.

Here are examples of limitations you may need to describe and to discuss how they possibly impacted your findings. Descriptions of limitations should be stated in the past tense.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but to offer reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe the need for future research.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, consult with a librarian! In cases when a librarian has confirmed that there is a lack of prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design]. Note that this limitation can serve as an important opportunity to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need in future research to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing self-reported data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to take what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data contain several potential sources of bias that should be noted as limitations: (1) selective memory (remembering or not remembering experiences or events that occurred at some point in the past); (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, or documents and, for whatever reason, access is denied or otherwise limited, the reasons for this need to be described.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single research problem, the time available to investigate a research problem and to measure change or stability within a sample is constrained by the due date of your assignment. Be sure to choose a topic that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, or thing is viewed or shown in a consistently inaccurate way. It is usually negative, though one can have a positive bias as well. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places and how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. Note that if you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating bias.
• Fluency in a language -- if your research focuses on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students, for example, and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic. This deficiency should be acknowledged.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods . Powerpoint Presentation. Regent University of Science and Technology.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as a pilot study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in later studies.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study  is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to reframe your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to  the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't ask a particular question in a survey that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in any future study. A underlying goal of scholarly research is not only to prove what works, but to demonstrate what doesn't work or what needs further clarification.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. Limitations are not Properly Acknowledged in the Scientific Literature. Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed . January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings! After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitiations of your study. Inflating of the importance of your study's findings in an attempt hide its flaws is a big turn off to your readers. A measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated, or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Yet Another Writing Tip

A Note about Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgement about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Huberman, A. Michael and Matthew B. Miles. Data Management and Analysis Methods. In Handbook of Qualitative Research. Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

What Is Research Design?

A Plain-Language Explainer (With Examples)

By: Derek Jansen (MBA) | Reviewers: Eunice Rautenbach (DTech) & Kerryn Warren (PhD) | April 2023

Overview: Research Design 101

What is research design.

• Research design types for quantitative studies
• Video explainer : quantitative research design
• Research design types for qualitative studies
• Video explainer : qualitative research design
• How to choose a research design
• Key takeaways

Research design refers to the overall plan, structure or strategy that guides a research project , from its conception to the final data analysis. A good research design serves as the blueprint for how you, as the researcher, will collect and analyse data while ensuring consistency, reliability and validity throughout your study.

Understanding different types of research designs is essential as helps ensure that your approach is suitable  given your research aims, objectives and questions , as well as the resources you have available to you. Without a clear big-picture view of how you’ll design your research, you run the risk of potentially making misaligned choices in terms of your methodology – especially your sampling , data collection and data analysis decisions.

The problem with defining research design…

One of the reasons students struggle with a clear definition of research design is because the term is used very loosely across the internet, and even within academia.

Some sources claim that the three research design types are qualitative, quantitative and mixed methods , which isn’t quite accurate (these just refer to the type of data that you’ll collect and analyse). Other sources state that research design refers to the sum of all your design choices, suggesting it’s more like a research methodology . Others run off on other less common tangents. No wonder there’s confusion!

In this article, we’ll clear up the confusion. We’ll explain the most common research design types for both qualitative and quantitative research projects, whether that is for a full dissertation or thesis, or a smaller research paper or article.

Research Design: Quantitative Studies

Quantitative research involves collecting and analysing data in a numerical form. Broadly speaking, there are four types of quantitative research designs: descriptive , correlational , experimental , and quasi-experimental .

As the name suggests, descriptive research design focuses on describing existing conditions, behaviours, or characteristics by systematically gathering information without manipulating any variables. In other words, there is no intervention on the researcher’s part – only data collection.

For example, if you’re studying smartphone addiction among adolescents in your community, you could deploy a survey to a sample of teens asking them to rate their agreement with certain statements that relate to smartphone addiction. The collected data would then provide insight regarding how widespread the issue may be – in other words, it would describe the situation.

The key defining attribute of this type of research design is that it purely describes the situation . In other words, descriptive research design does not explore potential relationships between different variables or the causes that may underlie those relationships. Therefore, descriptive research is useful for generating insight into a research problem by describing its characteristics . By doing so, it can provide valuable insights and is often used as a precursor to other research design types.

Correlational Research Design

Correlational design is a popular choice for researchers aiming to identify and measure the relationship between two or more variables without manipulating them . In other words, this type of research design is useful when you want to know whether a change in one thing tends to be accompanied by a change in another thing.

For example, if you wanted to explore the relationship between exercise frequency and overall health, you could use a correlational design to help you achieve this. In this case, you might gather data on participants’ exercise habits, as well as records of their health indicators like blood pressure, heart rate, or body mass index. Thereafter, you’d use a statistical test to assess whether there’s a relationship between the two variables (exercise frequency and health).

As you can see, correlational research design is useful when you want to explore potential relationships between variables that cannot be manipulated or controlled for ethical, practical, or logistical reasons. It is particularly helpful in terms of developing predictions , and given that it doesn’t involve the manipulation of variables, it can be implemented at a large scale more easily than experimental designs (which will look at next).

Need a helping hand?

Experimental research design is used to determine if there is a causal relationship between two or more variables . With this type of research design, you, as the researcher, manipulate one variable (the independent variable) while controlling others (dependent variables). Doing so allows you to observe the effect of the former on the latter and draw conclusions about potential causality.

For example, if you wanted to measure if/how different types of fertiliser affect plant growth, you could set up several groups of plants, with each group receiving a different type of fertiliser, as well as one with no fertiliser at all. You could then measure how much each plant group grew (on average) over time and compare the results from the different groups to see which fertiliser was most effective.

Overall, experimental research design provides researchers with a powerful way to identify and measure causal relationships (and the direction of causality) between variables. However, developing a rigorous experimental design can be challenging as it’s not always easy to control all the variables in a study. This often results in smaller sample sizes , which can reduce the statistical power and generalisability of the results.

Moreover, experimental research design requires random assignment . This means that the researcher needs to assign participants to different groups or conditions in a way that each participant has an equal chance of being assigned to any group (note that this is not the same as random sampling ). Doing so helps reduce the potential for bias and confounding variables . This need for random assignment can lead to ethics-related issues . For example, withholding a potentially beneficial medical treatment from a control group may be considered unethical in certain situations.

Quasi-Experimental Research Design

Quasi-experimental research design is used when the research aims involve identifying causal relations , but one cannot (or doesn’t want to) randomly assign participants to different groups (for practical or ethical reasons). Instead, with a quasi-experimental research design, the researcher relies on existing groups or pre-existing conditions to form groups for comparison.

For example, if you were studying the effects of a new teaching method on student achievement in a particular school district, you may be unable to randomly assign students to either group and instead have to choose classes or schools that already use different teaching methods. This way, you still achieve separate groups, without having to assign participants to specific groups yourself.

Naturally, quasi-experimental research designs have limitations when compared to experimental designs. Given that participant assignment is not random, it’s more difficult to confidently establish causality between variables, and, as a researcher, you have less control over other variables that may impact findings.

Research Design: Qualitative Studies

There are many different research design types when it comes to qualitative studies, but here we’ll narrow our focus to explore the “Big 4”. Specifically, we’ll look at phenomenological design, grounded theory design, ethnographic design, and case study design.

Phenomenological design involves exploring the meaning of lived experiences and how they are perceived by individuals. This type of research design seeks to understand people’s perspectives , emotions, and behaviours in specific situations. Here, the aim for researchers is to uncover the essence of human experience without making any assumptions or imposing preconceived ideas on their subjects.

For example, you could adopt a phenomenological design to study why cancer survivors have such varied perceptions of their lives after overcoming their disease. This could be achieved by interviewing survivors and then analysing the data using a qualitative analysis method such as thematic analysis to identify commonalities and differences.

Phenomenological research design typically involves in-depth interviews or open-ended questionnaires to collect rich, detailed data about participants’ subjective experiences. This richness is one of the key strengths of phenomenological research design but, naturally, it also has limitations. These include potential biases in data collection and interpretation and the lack of generalisability of findings to broader populations.

Grounded Theory Research Design

Grounded theory (also referred to as “GT”) aims to develop theories by continuously and iteratively analysing and comparing data collected from a relatively large number of participants in a study. It takes an inductive (bottom-up) approach, with a focus on letting the data “speak for itself”, without being influenced by preexisting theories or the researcher’s preconceptions.

As an example, let’s assume your research aims involved understanding how people cope with chronic pain from a specific medical condition, with a view to developing a theory around this. In this case, grounded theory design would allow you to explore this concept thoroughly without preconceptions about what coping mechanisms might exist. You may find that some patients prefer cognitive-behavioural therapy (CBT) while others prefer to rely on herbal remedies. Based on multiple, iterative rounds of analysis, you could then develop a theory in this regard, derived directly from the data (as opposed to other preexisting theories and models).

Grounded theory typically involves collecting data through interviews or observations and then analysing it to identify patterns and themes that emerge from the data. These emerging ideas are then validated by collecting more data until a saturation point is reached (i.e., no new information can be squeezed from the data). From that base, a theory can then be developed .

Ethnographic design involves observing and studying a culture-sharing group of people in their natural setting to gain insight into their behaviours, beliefs, and values. The focus here is on observing participants in their natural environment (as opposed to a controlled environment). This typically involves the researcher spending an extended period of time with the participants in their environment, carefully observing and taking field notes .

All of this is not to say that ethnographic research design relies purely on observation. On the contrary, this design typically also involves in-depth interviews to explore participants’ views, beliefs, etc. However, unobtrusive observation is a core component of the ethnographic approach.

As an example, an ethnographer may study how different communities celebrate traditional festivals or how individuals from different generations interact with technology differently. This may involve a lengthy period of observation, combined with in-depth interviews to further explore specific areas of interest that emerge as a result of the observations that the researcher has made.

As you can probably imagine, ethnographic research design has the ability to provide rich, contextually embedded insights into the socio-cultural dynamics of human behaviour within a natural, uncontrived setting. Naturally, however, it does come with its own set of challenges, including researcher bias (since the researcher can become quite immersed in the group), participant confidentiality and, predictably, ethical complexities . All of these need to be carefully managed if you choose to adopt this type of research design.

Case Study Design

With case study research design, you, as the researcher, investigate a single individual (or a single group of individuals) to gain an in-depth understanding of their experiences, behaviours or outcomes. Unlike other research designs that are aimed at larger sample sizes, case studies offer a deep dive into the specific circumstances surrounding a person, group of people, event or phenomenon, generally within a bounded setting or context .

As an example, a case study design could be used to explore the factors influencing the success of a specific small business. This would involve diving deeply into the organisation to explore and understand what makes it tick – from marketing to HR to finance. In terms of data collection, this could include interviews with staff and management, review of policy documents and financial statements, surveying customers, etc.

While the above example is focused squarely on one organisation, it’s worth noting that case study research designs can have different variation s, including single-case, multiple-case and longitudinal designs. As you can see in the example, a single-case design involves intensely examining a single entity to understand its unique characteristics and complexities. Conversely, in a multiple-case design , multiple cases are compared and contrasted to identify patterns and commonalities. Lastly, in a longitudinal case design , a single case or multiple cases are studied over an extended period of time to understand how factors develop over time.

How To Choose A Research Design

Having worked through all of these potential research designs, you’d be forgiven for feeling a little overwhelmed and wondering, “ But how do I decide which research design to use? ”. While we could write an entire post covering that alone, here are a few factors to consider that will help you choose a suitable research design for your study.

Data type: The first determining factor is naturally the type of data you plan to be collecting – i.e., qualitative or quantitative. This may sound obvious, but we have to be clear about this – don’t try to use a quantitative research design on qualitative data (or vice versa)!

Research aim(s) and question(s): As with all methodological decisions, your research aim and research questions will heavily influence your research design. For example, if your research aims involve developing a theory from qualitative data, grounded theory would be a strong option. Similarly, if your research aims involve identifying and measuring relationships between variables, one of the experimental designs would likely be a better option.

Time: It’s essential that you consider any time constraints you have, as this will impact the type of research design you can choose. For example, if you’ve only got a month to complete your project, a lengthy design such as ethnography wouldn’t be a good fit.

Resources: Take into account the resources realistically available to you, as these need to factor into your research design choice. For example, if you require highly specialised lab equipment to execute an experimental design, you need to be sure that you’ll have access to that before you make a decision.

Keep in mind that when it comes to research, it’s important to manage your risks and play as conservatively as possible. If your entire project relies on you achieving a huge sample, having access to niche equipment or holding interviews with very difficult-to-reach participants, you’re creating risks that could kill your project. So, be sure to think through your choices carefully and make sure that you have backup plans for any existential risks. Remember that a relatively simple methodology executed well generally will typically earn better marks than a highly-complex methodology executed poorly.

Recap: Key Takeaways

We’ve covered a lot of ground here. Let’s recap by looking at the key takeaways:

• Research design refers to the overall plan, structure or strategy that guides a research project, from its conception to the final analysis of data.
• Research designs for quantitative studies include descriptive , correlational , experimental and quasi-experimenta l designs.
• Research designs for qualitative studies include phenomenological , grounded theory , ethnographic and case study designs.
• When choosing a research design, you need to consider a variety of factors, including the type of data you’ll be working with, your research aims and questions, your time and the resources available to you.

If you need a helping hand with your research design (or any other aspect of your research), check out our private coaching services .

Learn More About Methodology

Triangulation: The Ultimate Credibility Enhancer

Triangulation is one of the best ways to enhance the credibility of your research. Learn about the different options here.

Research Limitations 101: What You Need To Know

Learn everything you need to know about research limitations (AKA limitations of the study). Includes practical examples from real studies.

In Vivo Coding 101: Full Explainer With Examples

Learn about in vivo coding, a popular qualitative coding technique ideal for studies where the nuances of language are central to the aims.

Process Coding 101: Full Explainer With Examples

Learn about process coding, a popular qualitative coding technique ideal for studies exploring processes, actions and changes over time.

Qualitative Coding 101: Inductive, Deductive & Hybrid Coding

Inductive, Deductive & Abductive Coding Qualitative Coding Approaches Explained...

📄 FREE TEMPLATES

Research Topic Ideation

Proposal Writing

Literature Review

Methodology & Analysis

Academic Writing

Referencing & Citing

Apps, Tools & Tricks

The Grad Coach Podcast

15 Comments

Is there any blog article explaining more on Case study research design? Is there a Case study write-up template? Thank you.

Thanks this was quite valuable to clarify such an important concept.

Thanks for this simplified explanations. it is quite very helpful.

This was really helpful. thanks

Thank you for your explanation. I think case study research design and the use of secondary data in researches needs to be talked about more in your videos and articles because there a lot of case studies research design tailored projects out there.

Please is there any template for a case study research design whose data type is a secondary data on your repository?

This post is very clear, comprehensive and has been very helpful to me. It has cleared the confusion I had in regard to research design and methodology.

This post is helpful, easy to understand, and deconstructs what a research design is. Thanks

This post is really helpful.

how to cite this page

Thank you very much for the post. It is wonderful and has cleared many worries in my mind regarding research designs. I really appreciate .

how can I put this blog as my reference(APA style) in bibliography part?

This post has been very useful to me. Confusing areas have been cleared

This is very helpful and very useful!

Wow! This post has an awful explanation. Appreciated.

Thanks This has been helpful

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

• Print Friendly

• Privacy Policy

Home » Research Design – Types, Methods and Examples

Research Design – Types, Methods and Examples

Table of Contents

Research Design

Definition:

Research design refers to the overall strategy or plan for conducting a research study. It outlines the methods and procedures that will be used to collect and analyze data, as well as the goals and objectives of the study. Research design is important because it guides the entire research process and ensures that the study is conducted in a systematic and rigorous manner.

Types of Research Design

Types of Research Design are as follows:

Descriptive Research Design

This type of research design is used to describe a phenomenon or situation. It involves collecting data through surveys, questionnaires, interviews, and observations. The aim of descriptive research is to provide an accurate and detailed portrayal of a particular group, event, or situation. It can be useful in identifying patterns, trends, and relationships in the data.

Correlational Research Design

Correlational research design is used to determine if there is a relationship between two or more variables. This type of research design involves collecting data from participants and analyzing the relationship between the variables using statistical methods. The aim of correlational research is to identify the strength and direction of the relationship between the variables.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This type of research design involves manipulating one variable and measuring the effect on another variable. It usually involves randomly assigning participants to groups and manipulating an independent variable to determine its effect on a dependent variable. The aim of experimental research is to establish causality.

Quasi-experimental Research Design

Quasi-experimental research design is similar to experimental research design, but it lacks one or more of the features of a true experiment. For example, there may not be random assignment to groups or a control group. This type of research design is used when it is not feasible or ethical to conduct a true experiment.

Case Study Research Design

Case study research design is used to investigate a single case or a small number of cases in depth. It involves collecting data through various methods, such as interviews, observations, and document analysis. The aim of case study research is to provide an in-depth understanding of a particular case or situation.

Longitudinal Research Design

Longitudinal research design is used to study changes in a particular phenomenon over time. It involves collecting data at multiple time points and analyzing the changes that occur. The aim of longitudinal research is to provide insights into the development, growth, or decline of a particular phenomenon over time.

Structure of Research Design

The format of a research design typically includes the following sections:

• Introduction : This section provides an overview of the research problem, the research questions, and the importance of the study. It also includes a brief literature review that summarizes previous research on the topic and identifies gaps in the existing knowledge.
• Research Questions or Hypotheses: This section identifies the specific research questions or hypotheses that the study will address. These questions should be clear, specific, and testable.
• Research Methods : This section describes the methods that will be used to collect and analyze data. It includes details about the study design, the sampling strategy, the data collection instruments, and the data analysis techniques.
• Data Collection: This section describes how the data will be collected, including the sample size, data collection procedures, and any ethical considerations.
• Data Analysis: This section describes how the data will be analyzed, including the statistical techniques that will be used to test the research questions or hypotheses.
• Results : This section presents the findings of the study, including descriptive statistics and statistical tests.
• Discussion and Conclusion : This section summarizes the key findings of the study, interprets the results, and discusses the implications of the findings. It also includes recommendations for future research.
• References : This section lists the sources cited in the research design.

Example of Research Design

An Example of Research Design could be:

Research question: Does the use of social media affect the academic performance of high school students?

Research design:

• Research approach : The research approach will be quantitative as it involves collecting numerical data to test the hypothesis.
• Research design : The research design will be a quasi-experimental design, with a pretest-posttest control group design.
• Sample : The sample will be 200 high school students from two schools, with 100 students in the experimental group and 100 students in the control group.
• Data collection : The data will be collected through surveys administered to the students at the beginning and end of the academic year. The surveys will include questions about their social media usage and academic performance.
• Data analysis : The data collected will be analyzed using statistical software. The mean scores of the experimental and control groups will be compared to determine whether there is a significant difference in academic performance between the two groups.
• Limitations : The limitations of the study will be acknowledged, including the fact that social media usage can vary greatly among individuals, and the study only focuses on two schools, which may not be representative of the entire population.
• Ethical considerations: Ethical considerations will be taken into account, such as obtaining informed consent from the participants and ensuring their anonymity and confidentiality.

How to Write Research Design

Writing a research design involves planning and outlining the methodology and approach that will be used to answer a research question or hypothesis. Here are some steps to help you write a research design:

• Define the research question or hypothesis : Before beginning your research design, you should clearly define your research question or hypothesis. This will guide your research design and help you select appropriate methods.
• Select a research design: There are many different research designs to choose from, including experimental, survey, case study, and qualitative designs. Choose a design that best fits your research question and objectives.
• Develop a sampling plan : If your research involves collecting data from a sample, you will need to develop a sampling plan. This should outline how you will select participants and how many participants you will include.
• Define variables: Clearly define the variables you will be measuring or manipulating in your study. This will help ensure that your results are meaningful and relevant to your research question.
• Choose data collection methods : Decide on the data collection methods you will use to gather information. This may include surveys, interviews, observations, experiments, or secondary data sources.
• Create a data analysis plan: Develop a plan for analyzing your data, including the statistical or qualitative techniques you will use.
• Consider ethical concerns : Finally, be sure to consider any ethical concerns related to your research, such as participant confidentiality or potential harm.

When to Write Research Design

Research design should be written before conducting any research study. It is an important planning phase that outlines the research methodology, data collection methods, and data analysis techniques that will be used to investigate a research question or problem. The research design helps to ensure that the research is conducted in a systematic and logical manner, and that the data collected is relevant and reliable.

Ideally, the research design should be developed as early as possible in the research process, before any data is collected. This allows the researcher to carefully consider the research question, identify the most appropriate research methodology, and plan the data collection and analysis procedures in advance. By doing so, the research can be conducted in a more efficient and effective manner, and the results are more likely to be valid and reliable.

Purpose of Research Design

The purpose of research design is to plan and structure a research study in a way that enables the researcher to achieve the desired research goals with accuracy, validity, and reliability. Research design is the blueprint or the framework for conducting a study that outlines the methods, procedures, techniques, and tools for data collection and analysis.

Some of the key purposes of research design include:

• Providing a clear and concise plan of action for the research study.
• Ensuring that the research is conducted ethically and with rigor.
• Maximizing the accuracy and reliability of the research findings.
• Minimizing the possibility of errors, biases, or confounding variables.
• Ensuring that the research is feasible, practical, and cost-effective.
• Determining the appropriate research methodology to answer the research question(s).
• Identifying the sample size, sampling method, and data collection techniques.
• Determining the data analysis method and statistical tests to be used.
• Facilitating the replication of the study by other researchers.
• Enhancing the validity and generalizability of the research findings.

Applications of Research Design

There are numerous applications of research design in various fields, some of which are:

• Social sciences: In fields such as psychology, sociology, and anthropology, research design is used to investigate human behavior and social phenomena. Researchers use various research designs, such as experimental, quasi-experimental, and correlational designs, to study different aspects of social behavior.
• Education : Research design is essential in the field of education to investigate the effectiveness of different teaching methods and learning strategies. Researchers use various designs such as experimental, quasi-experimental, and case study designs to understand how students learn and how to improve teaching practices.
• Health sciences : In the health sciences, research design is used to investigate the causes, prevention, and treatment of diseases. Researchers use various designs, such as randomized controlled trials, cohort studies, and case-control studies, to study different aspects of health and healthcare.
• Business : Research design is used in the field of business to investigate consumer behavior, marketing strategies, and the impact of different business practices. Researchers use various designs, such as survey research, experimental research, and case studies, to study different aspects of the business world.
• Engineering : In the field of engineering, research design is used to investigate the development and implementation of new technologies. Researchers use various designs, such as experimental research and case studies, to study the effectiveness of new technologies and to identify areas for improvement.

Advantages of Research Design

Here are some advantages of research design:

• Systematic and organized approach : A well-designed research plan ensures that the research is conducted in a systematic and organized manner, which makes it easier to manage and analyze the data.
• Clear objectives: The research design helps to clarify the objectives of the study, which makes it easier to identify the variables that need to be measured, and the methods that need to be used to collect and analyze data.
• Minimizes bias: A well-designed research plan minimizes the chances of bias, by ensuring that the data is collected and analyzed objectively, and that the results are not influenced by the researcher’s personal biases or preferences.
• Efficient use of resources: A well-designed research plan helps to ensure that the resources (time, money, and personnel) are used efficiently and effectively, by focusing on the most important variables and methods.
• Replicability: A well-designed research plan makes it easier for other researchers to replicate the study, which enhances the credibility and reliability of the findings.
• Validity: A well-designed research plan helps to ensure that the findings are valid, by ensuring that the methods used to collect and analyze data are appropriate for the research question.
• Generalizability : A well-designed research plan helps to ensure that the findings can be generalized to other populations, settings, or situations, which increases the external validity of the study.

Research Design Vs Research Methodology

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Thesis Format – Templates and Samples

Research Problem – Examples, Types and Guide

Research Objectives – Types, Examples and...

Research Paper Conclusion – Writing Guide and...

Data Interpretation – Process, Methods and...

How to Cite Research Paper – All Formats and...

Leave a comment x.

Save my name, email, and website in this browser for the next time I comment.

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Table of Contents
• Chapter 1: Introduction
• Chapter 2: Creating Trustworthy Guidelines
• Chapter 3: Overview of the Guideline Development Process
• Chapter 4: Formulating PICO Questions
• Chapter 5: Choosing and Ranking Outcomes
• Chapter 6: Systematic Review Overview
• Chapter 7: GRADE Criteria Determining Certainty of Evidence
• Chapter 8: Domains Decreasing Certainty in the Evidence
• Chapter 9: Domains Increasing One's Certainty in the Evidence
• Chapter 10: Overall Certainty of Evidence
• Chapter 11: Communicating findings from the GRADE certainty assessment
• Chapter 12: Integrating Randomized and Non-randomized Studies in Evidence Synthesis

Related Topics:

• Advisory Committee on Immunization Practices (ACIP)
• Vaccine-Specific Recommendations
• Evidence-Based Recommendations—GRADE

Chapter 8: Domains Decreasing Certainty in the Evidence

• This ACIP GRADE handbook provides guidance to the ACIP workgroups on how to use the GRADE approach for assessing the certainty of evidence.

8.1 Risk of bias (study limitations)

Study limitations may bias the estimates of the effect of an intervention on health outcomes. 1 The factors considered for evaluating study limitations or risk of bias (also referred to as internal validity) will depend on the study design. The number of studies is not a determining factor in determining risk of bias, as a single well-conducted study may result in high confidence in the estimated effect of vaccination on health outcomes. Risk of bias can differ amongst outcomes within an individual study, therefore, limitations for each outcome of interest in a study should be assessed separately.

Randomized Controlled Trials

For randomized controlled trials, Cochrane's revised risk of bias (RoB 2) tool can be used to assess study limitations. 2 3 The tool considers bias that may arise from the randomization process, deviations from the intended interventions, missing outcome data, measurement of the outcome and the selection of the reported result. Signaling questions are used to highlight concerns in each RoB domain. Judgements can express "High", "Low" or "Some concerns" with risk of bias. Details on how to use the tool and the various assessment questions can be found on the Risk of bias website 2 . Studies in which participants are allocated to intervention or control groups through quasi-randomization techniques (e.g., allocation by odd or even date of birth, date or day of admission, case record number, alternation/rotation) will automatically be at risk of selection bias due to inadequate generation of a randomized sequence, in addition to the ability of participants, or investigators enrolling participants, to foresee allocation. 4 Blinding outcome assessors is less important for the assessment of objective outcomes such as all-cause mortality, but is crucial for subjective outcomes such as quality of life. Risk of bias can differ across outcomes (e.g., higher risk of bias for subjective outcomes compared to objective outcomes when outcome assessors are not blinded; different subsets of studies for safety vs. efficacy studies). For adverse events or non-inferiority studies, intention-to-treat analyses may not be appropriate. If any information for assessing risk of bias is not reported in a publication, study investigators may be contacted. It may be possible to assess risk of bias from other reported information. For example, if information on allocation sequence concealment is not reported, data showing that the intervention and control groups are balanced at baseline may assuage concern regarding risk of bias. When assessing the risk of bias due to missing outcome data, reasons for the missing data and the quantity of missing data should both be taken into consideration. Table 5 provides a summary of the domains used in the RoB 2 assessment.

Table 5. Domains of RoB 2 tool

The Cochrane group has also developed risk of bias assessment tools to use for cluster-randomized trials and crossover trials . 2

Non-randomized Studies

The criteria for assessing non-randomized studies like cohort studies, case-control studies, controlled before-after studies, interrupted time series, and case series differs from risk of bias assessments for randomized trials. 1 The Cochrane group recommends using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool to assess the risk of bias for non-randomized studies, specifically for comparative cohort studies. 5 Similar to the RoB 2 tool recommended for RCTs, ROBINS-I assessments are done for specific results; each reported outcome study should be considered separately rather than judging the study as a whole. Confounding and co-interventions are major concerns that could lead to bias in non-randomized studies. Other domains such as selection bias, information bias, and reporting bias are also evaluated using the ROBINS-I tool; details on the signaling questions and domains used in the tool can be found on the Risk of bias website .

Table 6 provides an overview of the domains used in the ROBINS-I tool. Each domain is judged to have "Low", Moderate", or "Critical" risk of bias. "No information (NI)" is used when there is insufficient information to make a judgment on a domain. When using this tool, NRS start off with high certainty and can be graded down for study limitations after the ROBINS-I tool is used and concerns with risk of bias are identified. 27 The ROBINS-I tool uses an absolute metric rather than comparing non-randomized studies to a standard ideal NRS, thus making it easier to compare RCTs and non-randomized studies, as both are assessed using a similar metric for risk of bias.

Table 6. Domains of the ROBINS-I tool for NRS

The Newcastle-Ottawa Scale (NOS) is another tool that has been developed to assess the risk of bias of nonrandomized studies. 6

After using a tool to assess the risk of bias for each outcome in an individual study, the extent of study limitations for the body of evidence is categorized into one of the following groups: 1

• No serious limitations (do not downgrade evidence type): most of the studies comprising the body of evidence have low risk of bias for all key criteria for evaluating study limitations.
• Serious limitations (downgrade one level): most of the studies have crucial limitations for one criterion or some limitations for multiple criteria that lower confidence in the estimated effect of vaccination on the outcome of interest.
• Very serious limitations (downgrade two levels): most of the studies have crucial limitations for one or more criteria that substantially lower confidence in the estimated effect.
• Extremely serious limitations (downgrade three levels): 7 most of the studies have crucial limitations for multiple criteria that substantially lower confidence in the estimated effect. This option exists only for studies which are evaluated using ROBINS-I tool. The use of ROBINS-I here starts the evidence at high certainty.

When considering a body of evidence in which some studies have no serious limitations, some have serious limitations, and some have very serious limitations, it is not appropriate to automatically assign an average rating of serious limitations for the group of studies. When the risk of bias varies across studies, principles for determining whether to downgrade the evidence type for a group of studies include: 1

• Consider the extent to which each study contributes to the overall or pooled estimate of effect. Larger studies with many outcome events will contribute more.
• Assess whether the results differ for studies with low risk of bias and those with high risk of bias. Consider focusing on studies with lower risk of bias if the results differ by risk of bias.
• Downgrade when there is substantial risk of bias across most of the studies.
• Consider limitations pertaining to the other GRADE criteria (if there are close calls regarding risk of bias with another GRADE criterion, consider downgrading the evidence level for at least one of the two GRADE criteria)

When close-call situations occur, this should be made explicit, and the reason for the ultimate classification should be stated. Table 7a provides an example of when results from NRS may not have serious concerns with risk of bias, while the body of evidence consisting of randomized trials has concerns with study limitations. Since the trials used subjective reporting of the outcome and lacked blinding, the body of evidence was downgraded due to serious concerns with risk of bias.

Table 7b presents a situation in which the certainty of the evidence from RCTs for the outcomes of serious adverse events and myo- /pericarditis were judged as very low; therefore, the work group considered the evidence from NRS. For both of these outcomes, the RCTs had concerns due to the small number of events and total patients. The NRS provided complementary evidence with a larger number of participants and results consistent with those from RCTs.

Table 7a. Evidence profile for outcome of incidence of arthritis (5–56 days)

References in this table: 8

Note: Non-randomized studies without comparators are not included in evidence table, but would be considered to offer very low certainty (evidence type 4)

Explanations

a. Studies used variable definitions and methods for diagnosing and reporting arthritis. In addition, participants, healthcare personnel, and outcome assessors were not blinded in Huttner 2015 or Samai 2018 potentially influencing events reported for this subjective outcome.

b. The 95% CI includes the potential for possible harms, as well as possible benefit.

c. Few events reported do not meet optimal information size and suggest fragility in the estimate.

d. RR calculated using the standard continuity correction of 0.5 and the overall effect uses a random effects model.

Table 7b. Evidence profile for Use of JYNNEOS (orthopoxvirus) vaccine heterologous for those who received ACAM2000 primary series

References in this table: 9 10 11 12 13 14 15 16 17

RR: risk ratio; CI: confidence interval

a. Risk of bias due to lack of comparison data.

b. Seroconversion rate is an indirect measure of prevention.

c. Small sample size, no comparison.

d. Attrition rate was variable across study groups. One group lost 17% of participants.

e. Small sample size, fragility of estimate.

f. In the protocol it is unclear how serious adverse events were assessed.

g. Sample size is small, too small to detect rare adverse events.

h. Observational data was included in the evidence profile for this outcome because the effect estimate for the randomized trials was not estimable.

i. Single-arm studies contribute data to the intervention, but no available data for the comparison from the systematic review. Downgraded for indirectness because historical data was used for comparison.

j. Intervention data was drawn from 3 observational studies included in the systematic review. 0/349 (0.00 %) participants from 3 studies developed vaccine related serious adverse events.

k. Comparison data was drawn from historical data. In a phase III clinical trial for ACAM2000 enrolling participants with previous smallpox vaccination 3/1371 (0.22%) developed vaccine related serious adverse events after ACAM2000 administration. No smallpox vaccine-specific serious adverse event was recorded.

l. Assessment of myo-/pericarditis was initiated late in the study at the request of FDA. Very few subjects could be evaluated at that point. It was unclear how many subjects were evaluated.

m. Sample size is small, too small to detect rare events of myopericarditis after JYNNEOS®.

n. Intervention data was drawn from 3 observational studies included in the systematic review. 0/349 (0.00 %) participants developed myo-/pericarditis.

o. Comparison data was drawn from historical data. In a phase III clinical trial for ACAM2000 enrolling participants with previous smallpox vaccination, 0/1371 (0.00%) developed myo-/pericarditis after ACAM2000 administration.

8.2 Inconsistency

Inconsistency refers to an unexplained heterogeneity in the effect estimates across studies contributing to a summary estimate (e.g., relative risk or odds ratio for binary outcomes; mean difference for continuous outcomes). 18 Inconsistency can be assessed by examining the following indicators of heterogeneity: 1) visual examination of the forest plot (point estimates and confidence intervals); 2) calculating statistical test of heterogeneity])- Chi-squared (Chi 2 or X 2 ) statistic; 3) calculating the (I-squared[I 2 ]; 4) contextualizing the findings with the target for our certainty rating.

Heterogeneity occurs when there is large variability between the studies pooled in a meta-analysis. Visual inspection can show effects that differ from the rest and should include an examination of the point estimates and overlap of confidence intervals. 19 A forest plot suggesting heterogeneity would show confidence intervals from individual studies that have limited or no overlap with the summary estimate. The studies contributing to the summary estimate may have point estimates that widely differ. However, difference may not only be detected by visualization; therefore, complementing this with numerical estimates of heterogeneity may be helpful. The I² statistic describes the percentage of variation across studies that is due to heterogeneity rather than chance. The higher the I 2 statistic, the more likely the variability seen is due to more than just change ( I2 >30% is low, ~50% is moderate, and >75% is substantial and requires further exploration). The Chi 2 tests the null hypothesis that the included studies are not different (homogenous); however, the results are susceptible to studies with small samples or if there are few studies in the meta-analysis. If the Chi 2 is small and the p-value large (>0.10 or >0.05; i.e., not significant) heterogeneity may not be suspected. Lastly, if the point estimate of the pooled estimate visually falls within the 95% CI of the studies included in the analysis, heterogeneity is less of a concern.

When making decisions about the extent to which heterogeneity contributes to our certainty rating (i.e., should we rate down for inconsistency and by how much), the target (threshold or range) of our certainty rating must be identified. 20 This could be the null, a minimally important difference, a range of magnitudes of trivial, small, moderate or large. Inconsistency is a concern when it crosses possible thresholds of meaning. Inconsistency may not be a concern when all of the point estimates (and CIs) of included studies lie above a given threshold even if they are disparate (e.g., visually confidence intervals don't overlap or I 2 is high, etc.).

In addition to noting the presence of inconsistency, it is desirable to determine potential reasons for the inconsistency. Differences in the following may result in inconsistency:

• Populations (e.g., vaccines may have different relative effects in sicker populations);
• Interventions (e.g., different effects with different number of doses or comparators);
• Outcomes (e.g., duration of follow-up);
• Study methods (e.g., studies with higher and lower risk of bias

When heterogeneity is large and a plausible explanation cannot be identified, the evidence level should be downgraded by one or two levels, depending on heterogeneity in the magnitude of effect. While there are not specific guidelines for this; see "GRADE guidelines: 7. Rating the quality of evidence—inconsistency" for examples of downgrading. 18 If inconsistency can be explained, estimates of effect should be presented separately for the stratification that explains the observed heterogeneity. If results differ by study methods, preference may be given to results of studies with a lower risk of bias. If results differ by population groups, different recommendations may be made for different groups. If only one study is available, there are by default no concerns with inconsistency (i.e., select "Not serious" when grading).

Inconsistency is assessed more strictly in binary/dichotomous outcomes (relative values) than continuous outcomes (absolute values). For binary outcomes, inconsistency should be assessed using risk ratio or odds ratio which are measures of relative effect, where a value of 1 indicates the estimated effect is similar for both the intervention and comparison group. 21 Conversely, the risk difference is a measure of absolute effect that represents the difference in the observed risk and should not be used to assess inconsistency because it is very sensitive to the baseline risk (i.e., risk in control group) and baseline risk can differ substantially between studies. 18 The forest plot below (Figure 6) shows four studies included in the analysis for the binary outcome of severe (grade 3) arthralgia. Here, two studies contribute to the effect estimate (risk ratio), as they contain events. Visually, the pooled estimate (6.40) falls within the 95% CIs of the included studies; the Chi2 is small (0.08) and the p-value is large (i.e., not significant at 0.10), and the I 2 = 0%. 8 Based on all three steps, heterogeneity is not serious for this outcome.

To recap, any of the following factors may result in rating down for inconsistency:

• I 2 is large (I2 >30% is low, ~ 50% is moderate, and >75% is substantial and requires further exploration).
• Statistical test for heterogeneity (Chi 2 ) shows a low P-value (i.e., < 0.05).
• Confidence intervals of the point estimates of included studies do not overlap or show minimal overlap.

Figure 6. Estimates of effect for RCTs included in analysis for outcome of incidence of severe (grade 3) arthralgia (0-42 days)

References in this figure: 8

Effect estimates from continuous outcomes can be presented in a number of ways. If the primary studies included have assessed an outcome using the same scale, then it can be presented as a Mean Difference (MD). However, when pooling studies which measure the same continuous outcome using different instruments or varying scales, researchers might choose to present this as a Standardized Mean Difference (SMD). The MD can be easily interpreted and assessed for heterogeneity and inconsistency. However, SMD might pose more of a difficulty and reviewers might need to use a different approach to further present and interpret the effect estimate. 22 Tables 8 and 9 present the options available to reviewers dealing with studies with these challenges.

Table 8: Five approaches to presenting results of continuous variables when primary studies have used different instruments to measure the same construct

References in this table: 22

Abbreviations: SD, standard deviation; MID, minimally important difference.

Table 9: Application of approaches to dexamethasone for pain after laparoscopic cholecystectomy example

Abbreviations: CI, confidence interval; SD, standard deviation; SMD, standardized mean difference.

a. Evidence limited by heterogeneity between studies

b. Evidence limited by imprecise data

c. The 20% comes from the proportion in the control group requiring rescue analgesia

d. Crude (arithmetic) means of the postoperative pain mean responses across all five trials when transformed to a 100-point scale

Table 10 provides an example of how inconsistency is explained in an evidence profile. The footnotes highlight the large I 2 value and, while some of the heterogeneity may be explained by study limitations, there is enough concern to warrant downgrading the body of evidence. As a result, the table shows serious concerns with inconsistency.

Table 10. Evidence profile for outcome of incidence of arthralgia (0–42 days)

Note: Non-randomized studies without comparators are not included in evidence table, but would be considered of very low certainty (evidence type 4); CI: Confidence interval; RR: Relative risk

a. Participants, healthcare personnel, and outcome assessors were not blinded in Huttner 2015 or Samai 2018 potentially influencing events reported for this subjective outcome. Concern for possible underreporting in Kennedy because arthralgia was only solicited at one week and at one month for most participants; Huttner only solicited arthralgia for low dose participants

b. Rated down once due to concerns with heterogeneity (I2=70%). Some may be explained by concerns with risk of bias (poor randomization or outcome definition)

c. The 95% confidence interval of the mean pooled estimate includes potential for possible harms as well as benefits

d. Few events reported do not meet optimal information size and suggest fragility in the estimate

e. RR calculated using the standard continuity correction of 0.5 and uses a random effects mode

8.3 Indirectness

Research that answers the PICO question most appropriately is considered direct evidence; therefore, studies that address the target population, compare the interventions specified in the question and measure the outcomes of interest can be classified as direct evidence. 23 Indirectness can be introduced when any of the four situations below occur:

• The population that participated in studies may differ from the population of interest;
• The intervention that was evaluated may differ from the intervention of interest;
• The primary interest is head-to-head comparisons of vaccine A to vaccine B, but A was compared with C and B was compared with C (i.e., the comparator is different from the comparator of interest)
• The outcome that was assessed may differ from that of primary interest. This may occur when there is either an intermediate outcome or a surrogate outcome used to inform the outcome of interest. For example, a panel may decide that vaccine efficacy is a critical outcome; however, the underlying evidence does not report directly on the measure of efficacy. This may occur when there is a low baseline risk of developing the outcome of interest. When assessing the evidence for vaccines, immunogenicity may serve as an appropriate surrogate for vaccine efficacy if vaccine efficacy data are not available; however, unless there is an established immune correlate of protection, this should result in downgrading for indirectness.

Table 11. Examples of indirect evidence

Both systematic reviews and guidelines may require the use of evidence that is indirect with respect to the comparator and outcomes of interest. Guidelines also commonly deal with evidence that is indirectly related to the population and intervention specified in the PICO question; these are sometimes described as concerns with applicability. When limited evidence is available, it is often necessary to turn to indirect evidence to help inform judgements. For the purpose of guidelines, it is important to consider all four potential causes of indirectness when rating down the domain; when there are multiple concerns with indirectness, it may be appropriate to rate down twice for indirectness. The use of surrogate outcomes typically results in rating down unless evidence of a strong association between the surrogate and the long- or short-term outcome of interest is established. The rating down process is not always additive, thus it is important to consider the evidence from all angles.

When developing recommendations, guidelines may need to use surrogate outcomes and/or indirect evidence. Although direct evidence is ideal, recommendations may be supported by indirect evidence as long as the indirectness is acknowledged in the certainty assessment.

To decide whether JYNNEOS® (orthopoxvirus) vaccine primary series or ACAM2000 vaccine primary series should be recommended for persons who are at risk for occupational exposure to orthopoxviruses, the guideline panel prioritized the outcome of "Prevention of Disease". However, cases of orthopoxvirus were not reported by the trials. Instead, the surrogate measures of geometric mean titer (GMT) and seroconversion rate were used to inform the outcome of "Prevention of Disease". The work group decided to rate down for indirectness for both of these measures, as there was some uncertainty in how directly findings about the GMT or seroconversion rate would predict prevention of disease. Table 12a presents a truncated GRADE Evidence Profile showing the use of a surrogate outcome to inform the critical outcome of Prevention of Disease. The second outcome presented, Severity of Disease, was informed by one trial reporting on the proportion of study participants with an attenuated take lesion. The ideal measure of disease severity is taking maximum lesion area. However, the work group recognized that the clinical difference between categorical (proportion of participants with attenuated take) and the continuous measurement (take maximum lesion area) was minimal and therefore did not rate down for indirectness for the outcome of Severity of Disease.

In a second example, the ACIP recently provided recommendations for the following policy question: Should pre-exposure vaccination with the rVSVΔG-ZEBOV-GP vaccine be recommended for adults 18 years of age or older in the U.S. population who are at potential occupational risk of exposure to Ebola virus (species Zaire ebolavirus) for prevention of Ebola virus infection. 24 Due to the limited literature available for certain outcomes like the development of Ebola-related symptomatic illness, a randomized cluster study was used in the evidence profile that focused on contacts of recently confirmed Ebola cases in Guinea, west Africa. 25 Since the PICO question was specific to the U.S. population, the evidence was downgraded for indirectness but was still used to support the guideline recommendations. As a result, in table 12b, the cluster study is downgraded, and an explanation is provided in the footnotes regarding why there are serious concerns for indirectness.

Table 12a. GRADE Evidence Profile for Use of JYNNEOS (orthopoxvirus) vaccine primary series for research, clinical laboratory, response team, and healthcare personnel

References in this table: 9 18 10 11 12 13 14 15

a. Geometric mean titer is an indirect measure of efficacy.

b. Frey study used Dryvax in the comparison group. For the immunogenicity outcomes we do not feel there would be a significant difference between the two live vaccines.

c. In order to calculate a mean difference and 95% CI, geometric mean data were transformed to arithmetic mean. The effect estimate was then transformed to geometric mean difference, which you see here.

d. Seroconversion rate is an indirect measure of efficacy.

e. 95% CI includes the potential for both meaningful benefit as well as meaningful harm.

f. Concerns for risk of bias due to attrition. The two groups that contributed data to the intervention and comparison for this outcome lost between 11 and 21% of participants at the time this outcome was assessed.

g. The ideal measure of disease severity is to take maximum lesion area. This study reports the proportion of participants with an attenuated take lesion. Clinical difference between categorical (proportion of participants with attenuated take) vs. continuous measurement (take maximum lesion area) is minimal. We feel this won't affect indirectness. See Parrino et al. 2007 for a description of lesion attenuation criteria.

Table 12b. Evidence profile for outcome of development of Ebola-related symptomatic illness

Note: Outcome assessed with laboratory confirmed case of EVD

a. Henao-Restrepo 2017 was a cluster randomized trial (i.e., units of randomization were clusters); cluster-level data presented here.

b. Concern for indirectness to U.S. population: population consists of contacts and contacts of contacts of EVD case, ring vaccination strategy which may include post-exposure vaccination.

c. Because this study was done at a time when the 2014—2015 West Africa outbreak was waning in Guinea and there are few events reported, it does not meet optimal information size and suggests fragility in the estimate; 95% CI contains the potential for desirable as well as undesirable effects.

d. Henao-Restrepo 2017 was a cluster randomized trial (i.e., units of randomization were clusters); participant-level data presented here

e. The concerns with indirectness pose no inflationary effect; therefore, the evidence was rated up based on a very large magnitude of effect from the 96% reduction in risk and overall certainty was upgraded two levels.

f. Denominator represents participants from the clusters randomized to receive immediate vaccination.

g. RR calculated using the standard continuity correction of 0.5.

8.4 Imprecision

Imprecision refers to the risk of random error in the evidence. It is rated as either not serious, serious or very serious, similar to the other GRADE domains discussed above. 26 The estimated effect is considered imprecise when studies have a wide confidence interval (CI). This usually occurs when few events and few patients are included in studies. Concerns with imprecision can lead to uncertainty in the results presented in the evidence. For systematic reviews, the following indicate imprecision for an outcome:

• Total sample size across all studies for an outcome is lower than the calculated sample size for a single adequately powered study ( online calculators are available for sample size calculations; or
• The 95% confidence interval (CI) of the pooled or best estimate of effect size includes both no effect AND appreciable benefit or appreciable harm (even if sample size is adequate). When an outcome is rare, 95% CIs of relative effects may be very wide, but 95% CIs of absolute effects may be narrow; in such situations, the evidence level may not be downgraded. For continuous outcomes, the threshold for appreciable benefit or appreciable harm refers to the difference in score in the outcome that is perceived as important.

For guidelines, additional considerations like clinical decision thresholds for optimal sample size and the event rate must be accounted for. 27 The evidence level may be downgraded because of imprecision in the following situations:

• The 95% CI includes both no effect AND an effect that represent a benefit that would outweigh potential harms.
• The 95% CI excludes no effect, but the lower confidence limit crosses a threshold below which, given potential harms, one would not recommend the intervention
• The 95% CI includes no effect AND an effect that represent a harm that despite the benefits, would still be unacceptable.
• The 95% CI excludes no effect, but the upper confidence limit crosses a threshold above which, given the benefits, one would recommend the intervention.

When assessing the risk for rare events (e.g., GBS, myocarditis, etc.) caused by a vaccine, the number of events needed may not be large enough to detect such rare events. The suspected rate of such events should be assessed in relation to the number of subjects tested to determine if the evidence should be downgraded for concerns about fragility with imprecision. An alternative approach would be to calculate the optimal information size (OIS) based on the total population instead of relying on the number of events that typically inform a judgment for imprecision. The OIS has been defined as the minimum amount of cumulative information required for reliable conclusions about an intervention, i.e., a calculation similar to calculating the sample size of patient in an individual trial, the difference being that the OIS considers the potential for heterogeneity between studies. 28 Therefore, if the number of participants in the meta-analyses is less than what is generated from a conventional sample-size calculation, there may be serious or very serious concerns about imprecision.

Table 11 provides an example of how imprecision assessments are justified. For example, the results from the randomized controlled trials are informed by a large sample size, however, the confidence interval is wide and cannot exclude the potential for both harm and benefit. Thus, concerns with imprecision are serious. In contrast, the results from the NRS have a wide confidence interval that cannot exclude the potential for harm and benefit; they are informed by few events that do not meet the optimal information size. Therefore, the concerns with imprecision are classified as "very serious" rather than "serious".

More information on assessing imprecision is available in the "Grade Guidelines 6. Rating the quality of evidence—imprecision" 2011 26 29

Table 13. Evidence profile for outcome of incidence of arthritis (5-56 days)

Note: Non-randomized studies without comparators are not included in evidence table, but would be considered of very low certainty (evidence type 4)

8.5 Publication bias

Publication bias is a type of reporting bias that leads to a systematic underestimation or an overestimation of the underlying effect (beneficial or harmful) due to the selective publication of studies. 30 Publication bias arises when investigators fail to publish studies, typically those that show no effect. Publication bias might be suspected if the available studies are uniformly small and funded by industry; a thorough review of clinical trial registries should be performed to identify if any trials were registered but not published. A funnel plot of studies with the magnitude of the effect size (e.g., relative risk or odds ratio for a binary outcome) on the X-axis, and variance (proxy for sample size) on the Y-axis can help assess publication bias. A funnel plot with asymmetrical distribution suggests publication bias. For meta-analyses with fewer than 10 studies, performing a funnel plot may be skewed; therefore, it is recommended to only perform when more than 10 studies are available. In situations with fewer than 10 studies, authors can consider additional factors when assessing publication bias: size and direction of identified studies, records of unpublished trials, availability of intervention under investigation (i.e., proprietary or specialty vaccines may be more regulated or documented, therefore, increased confidence that all available studies have been identified).

Due to the challenges in determining publication bias, publication bias is either described as "undetected" or "strongly suspected" in an evidence profile. Figure 7 provides an example of a funnel plot that has a symmetrical distribution and there is not suspicion of undetected publication bias. Conversely, figure 8 presents an example in which the forest plot is asymmetrical and therefore suggests there may be concerns with publication bias, requiring further investigation.

Figure 7. Example of funnel plot with no strong suspicion of publication bias

References in this figure: 31

Figure 8. Example of a funnel plot with suspicion of publication bias

References in this figure: 30

• Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol. 2011/04// 2011;64(4):407-415. doi:10.1016/j.jclinepi.2010.07.017
• Risk of bias tools - RoB 2 tool.
• Sterne JA, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366
• Higgins J, Savović J, Page M, Elbers R, Sterne J. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022). Cochrane; 2022. www.training.cochrane.org/handbook .
• Sterne J, Hernán M, McAleenan A, Reeves B, Higgins J. Chapter 25: Assessing risk of bias in a non-randomized study. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022) Cochrane; 2022. www.training.cochrane.org/handbook .
• GA Wells BS, D O'Connell, J Peterson, V Welch, M Losos, P Tugwell. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute. https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
• Thomas Piggott RLM, Carlos A Cuello-Garcia, Nancy Santesso, Reem A Mustafa, Joerg J Meerpohl, Holger J Schünemann; GRADE Working Group. Grading of Recommendations Assessment, Development, and Evaluations (GRADE) notes: extremely serious, GRADE's terminology for rating down by three levels. J Clin Epidemiol. 2020;120:116-120. doi:10.1016/j.jclinepi.2019.11.019
• Choi MJ, Cossaboom CM, Whitesell AN, et al. Use of ebola vaccine: recommendations of the Advisory Committee on Immunization Practices, United States, 2020. MMWR Recommendations and Reports. 2021;70(1):1.
• (ACIP) ACoIP. Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Use of JYNNEOS® (orthopoxvirus) vaccine heterologous for those who received ACAM2000 primary series. Centers for Disease Control and Prevention. https://www.cdc.gov/vaccines/acip/recs/grade/JYNNEOS-orthopoxvirus-heterologous.html
• Ahmed F, Temte JL, Campos-Outcalt D, Schünemann HJ, Group AEBRW. Methods for developing evidence-based recommendations by the Advisory Committee on Immunization Practices (ACIP) of the US Centers for Disease Control and Prevention (CDC). Vaccine. 2011;29(49):9171-9176.
• Committee on Standards for Developing Trustworthy Clinical Practice Guidelines BoHCS, Institute of Medicine. Clinical Practice Guidelines We Can Trust. National Academies Press; 2011.
• Schünemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014/02/18/ 2014;186(3):E123-E142. doi:10.1503/cmaj.131237
• World Health O. WHO handbook for guideline development. World Health Organization; 2014:167.
• Thomas J, Kneale D, McKenzie J, Brennan S, Bhaumik S. Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022). Cochrane; 2022. www.training.cochrane.org/handbook .
• Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011/04// 2011;64(4):395-400. doi:10.1016/j.jclinepi.2010.09.012
• Fitch K, Bernstein SJ, Aguilar MD, et al. The RAND/UCLA Appropriateness Method User's Manual. 2001. 2001/01/01/. Accessed 2022/03/06/21:27:33. https://www.rand.org/pubs/monograph_reports/MR1269.html
• (ACIP) ACoIP. GRADE: Use of Smallpox Vaccine in Laboratory and Health-Care Personnel at Risk for Occupational Exposure to Orthopoxviruses. Centers for Disease Control and Prevention.
• Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 7. Rating the quality of evidence-- inconsistency. J Clin Epidemiol. 2011/12// 2011;64(12):1294-1302. doi:10.1016/j.jclinepi.2011.03.017
• Cynthia P Cordero ALD. Key concepts in clinical epidemiology: detecting and dealing with heterogeneity in meta-analyses. J Clin Epidemiol. 2021;130:149-151. doi:10.1016/j.jclinepi.2020.09.045
• Gordon Guyatt YZ, Martin Mayer, Matthias Briel, Reem Mustafa, Ariel Izcovich, Monica Hultcrantz, Alfonso Iorio, Ana Carolina Alba, Farid Foroutan, Xin Sun, Holger Schunemann, Hans DeBeer, Elie A Akl, Robin Christensen, Stefan Schandelmaier. GRADE guidance 36: updates to GRADE's approach to addressing inconsistency. J Clin Epidemiol. 2023;158:70-83. doi:10.1016/j.jclinepi.2023.03.003
• Higgins J, Li T, Deeks J. Chapter 6: Choosing effect measures and computing estimates of effect. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022). Cochrane; 2022. www.training.cochrane.org/handbook .
• Guyatt GH, Thorlund K, Oxman AD, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol. Feb 2013;66(2):173-83. doi:10.1016/j.jclinepi.2012.08.001
• Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 8. Rating the quality of evidence-- indirectness. J Clin Epidemiol. 2011/12// 2011;64(12):1303-1310.
• ACIP Grading for Ebola Vaccine | CDC. 2021/01/07/T05:56:55Z 2021
• Henao-Restrepo AM, Camacho A, Longini IM, et al. Efficacy and effectiveness of an rVSV vectored vaccine in preventing Ebola virus disease: final results from the Guinea ring vaccination, open-label, cluster-randomised trial (Ebola Ça Suffit!). The Lancet. 2017/02/04/ 2017;389(10068):505-518. doi:10.1016/S0140-6736(16)32621-6
• Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence-- imprecision. J Clin Epidemiol. 2011/12// 2011;64(12):1283-1293. doi:10.1016/j.jclinepi.2011.01.012
• Zeng L, Brignardello-Petersen R, Hultcrantz M, et al. GRADE guidelines 32: GRADE offers guidance on choosing targets of GRADE certainty of evidence ratings. J Clin Epidemiol. Sep 2021;137:163-175. doi:10.1016/j.jclinepi.2021.03.026
• Pogue JM, & Yusuf, S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Controlled clinical trials. 1997;18(6):580- 593.
• Gordon H Guyatt ADO, Regina Kunz, Jan Brozek, Pablo Alonso-Coello, David Rind, P J Devereaux, Victor M Montori, Bo Freyschuss, Gunn Vist, Roman Jaeschke, John W Williams Jr, Mohammad Hassan Murad, David Sinclair, Yngve Falck-Ytter, Joerg Meerpohl, Craig Whittington, Kristian Thorlund, Jeff Andrews, Holger J Schünemann. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol. 2011;64(12):1283-93. doi:10.1016/j.jclinepi.2011.01.012
• Guyatt GH, Oxman AD, Montori V, et al. GRADE guidelines: 5. Rating the quality of evidence-- publication bias. J Clin Epidemiol. 2011/12// 2011;64(12):1277-1282. doi:10.1016/j.jclinepi.2011.01.011
• Yong PJ, Matwani S, Brace C, et al. Endometriosis and Ectopic Pregnancy: A Meta-analysis. J Minim Invasive Gynecol. 2020/02// 2020;27(2):352-361.e2.

ACIP GRADE Handbook

This handbook provides guidance to the ACIP workgroups on how to use the GRADE approach for assessing the certainty of evidence.