(%)
Of the 329 identified as feasibility studies:
Therefore, from the 160 records with known feasibility outcomes, 133 (83%) concluded that the RCT was feasible and 27 (17%) that the RCT was not feasible. The interrater reliability across the three rating points was 84%.
The breakdown of the results by topic and setting are presented in Tables Tables2 2 and and3, 3 , albeit based on small numbers in Table Table2, 2 , where there were at least 10 studies reported for a specific research topic, the feasibility rate ranged between 79% and 100% for individual research topics, and in Table Table3, 3 , the feasibility rate ranged between 73% and 93% for research setting.
Breakdown of research topic area and whether RCT was feasible or not feasible for studies with known outcomes
Research topic | Total ( = 160) (% of total) | RCT feasible ( = 133) (% of topic) | RCT not feasible ( = 27) (% of topic) |
---|---|---|---|
Cancer | 19 (12) | 15 (79) | 4 (21) |
Circulatory System | 19 (12) | 19 (100) | 0 (0) |
Digestive System | 4 (3) | 3 (75) | 1 (25) |
Ear, Nose and Throat | 0 (0) | 0 (0) | 0 (0) |
Eye Diseases | 2 (1) | 2 (100) | 0 (0) |
Genetic Diseases | 0 (0) | 0 (0) | 0 (0) |
Infections and Infestations | 4 (3) | 1 (25) | 3 (75) |
Injury, Occupational Diseases, Poisoning | 2 (1) | 0 (0) | 2 (100) |
Mental and Behavioural Disorders | 39 (24) | 33 (85) | 6 (15) |
Musculoskeletal Diseases | 7 (4) | 5 (71) | 2 (29) |
Neonatal Diseases | 1 (1) | 1 (100) | 0 (0) |
Nervous System Diseases | 9 (6) | 8 (89) | 1 (11) |
Not Applicable | 8 (5) | 5 (63) | 3 (38) |
Nutritional, Metabolic, Endocrine | 15 (9) | 13 (87) | 2 (13) |
Oral Health | 1 (1) | 1 (100) | 0 (0) |
Pregnancy and Childbirth | 5 (3) | 5 (100) | 0 (0) |
Respiratory | 4 (3) | 3 (75) | 1 (25) |
Signs and Symptoms | 10 (6) | 9 (90) | 1 (10) |
Skin and Connective Tissue Diseases | 1 (1) | 1 (100) | 0 (0) |
Surgery | 6 (4) | 5 (83) | 1 (17) |
Urological and Genital Diseases | 4 (3) | 4 (100) | 0 (0) |
Breakdown of research setting and whether RCT was feasible or not feasible for studies with known outcomes
Research Setting | Total ( = 160) (% of total) | RCT feasible ( = 133) (% of setting) | RCT not feasible ( = 27) (% of setting) |
---|---|---|---|
Community | 3 (2) | 2 (67) | 1 (33) |
GP practices | 15 (9) | 11 (73) | 4 (27) |
Home | 1 (1) | 1 (100) | 0 (0) |
Hospitals | 80 (50) | 69 (86) | 11 (14) |
Internet | 0 (0) | 0 (0) | 0 (0) |
Not specified | 14 (9) | 13 (93) | 1 (7) |
Other | 37 (23) | 30 (81) | 7 (19) |
Schools | 10 (6) | 7 (70) | 3 (30) |
Figure Figure1 1 shows the specific reasons RCTs were deemed not to be feasible, irrespective of research topic area or research setting, with patient recruitment clearly being the most common reason and reported for 18 (60%) of the 27 feasibility studies reporting the RCT to be not feasible.
Reasons why RCTs were not feasible
The largest outcome group was ‘unknown if RCT feasible’ ( n = 160) which was where the feasibility study had completed but the published results could not be found. The trial end date of these studies ranged from 1999 to 2019. Sixty three of the 160 studies had trial end dates at least 2 years before our data access and therefore would have had ample time to publish. This suggests that there is a large amount of under-reporting of results of feasibility studies which is consistent with the general under-reporting of trials [ 14 – 18 ].
It was not possible to determine whether certain research topics and/or research settings are more associated with feasibility because we found too few studies reporting that the RCT was not feasible. Irrespective of research topic or research setting, there were consistent reasons why RCTs were considered not feasible: patient recruitment, trial design/methods, intervention and outcome measures. Of these, patient recruitment was the most commonly reported reason in the published results paper. This will not be a controversial finding for the research and funding community.
There are several plausible reasons why studies showing that RCTs are not feasible do not often appear in the literature. First, whilst feasibility studies are typically smaller than RCTs, it is clear from RfPB data [ 12 ] that feasibility studies are still a large investment of resource costing on average £219,048 and taking 31 months. Research teams may be confident (and persuasive) enough to apply for the full RCT, perhaps including an internal pilot. Second, it is becoming more routine to include feasibility progression criteria in feasibility designs with ‘red, amber, green’ pre-specified assessments to indicate feasibility. Many of the studies we reviewed did not use progression criteria or used unclear criteria. Therefore, it is very plausible that some studies were optimistic in reporting the RCT to be feasible. We also noted examples where researchers had reported the RCT feasible subject to such substantial changes that it was debatable what relation the feasibility study could claim to any eventual RCT. Third, the bar to demonstrate feasibility may be artificially low if studies are not identifying and addressing the specific challenges in the clinical field of interest. This may be accentuated if feasibility studies rely on a formulaic design [ 8 ]. Fourth, the team may be reluctant to publish the conclusion that a trial is not feasible. Historically, there have been challenges publishing ‘negative results’, but recent advances such as better reporting guidance (CONSORT) and specialist journals such as Pilot and Feasibility Studies have helped to address these challenges. Looking at the 160 studies that apparently had no results published, 63 had finished more than 2 years before our data access and thus would have had ample time to publish. The remaining 97 studies may well go on to publish non-feasible conclusions in future. Therefore, future reviews of feasibility studies might show a more complete publication picture.
There were some challenges in reviewing the ISRCTN records. Firstly, although we used a clear definition of what constitutes a feasibility study, to some extent, we were retro-fitting a definition to previously funded research. Nevertheless, we took the view that any study that explicitly stated it was preparing for a RCT, and had typical feasibility outcomes, was eligible for inclusion. One particular challenge was the number of studies that identified as pilot studies yet had no explicit plan to conduct a further RCT. Some included a mix of typical feasibility study outcomes and typical pilot study outcomes. These were typically excluded if they were primarily exploratory studies or even underpowered trials. Another challenge from the ISRCTN records was that we found self-identified feasibility studies which were examining the feasibility of implementation of a particular treatment/service rather than examining whether a future RCT was feasible. Studies that were not planning to progress to a RCT were excluded but were often challenging to identify.
It is clear from the data that more feasibility studies, or at least more studies which resemble current definitions of feasibility studies, are being conducted, with over 97 feasibility studies completing in 2018 compared with 21 10 years earlier and two 10 years before that. This follows the trend of more focus on feasibility studies with activity such as more research, specialised journals and better guidance on designing, conducting and reporting feasibility studies. It would be interesting to understand why more feasibility studies are being conducted. Could it be that funders of RCTs require more evidence that expensive RCTs will be successfully delivered? Perhaps there are also more funding opportunities for feasibility studies with NIHR programmes like RfPB becoming well known for supporting such studies. It is likely that more feasibility studies are now included on trial registries and were therefore discoverable.
Whilst the majority of the feasibility studies with reported outcomes are feasible, it would have been useful to have known how many studies progressed to RCT and how long they took to progress to have a better-informed view of the utility of feasibility studies. Whilst we noticed that some ISRCTN records contained both the feasibility study and subsequent RCT in a single ISRCTN record, it was not clear how systematically these records were updated. It would be useful if trial registries collected more data including links between feasibility studies and RCTs and ultimately outcomes from RCTs. Knowing the full picture from feasibility study progression to RCT competition and study outcome would allow a more informed view on how feasibility studies contribute to the trials pathway.
It is reasonable to assume that if the feasibility rate is too high, then researchers and research funders are contributing to the waste in the system by conducting studies which will often show the RCT is feasible. In such cases, it may be more appropriate to go straight to the RCT and accept that a relatively small percent will inevitably fail, whilst noting that feasibility studies do help address other uncertainties which may not have been foreseen. Conversely, if the feasibility rate is too low, it may indicate that researchers and funders are too optimistic and commit to studies which are not likely to progress down the trials pathway, although based on the current analysis, there is insufficient evidence to suggest that this scenario is occurring. However, it is acknowledged that many useful aspects and insights are gained via feasibility studies and that those demonstrating the RCT to be feasible often do once addressing challenges encountered via the feasibility study.
There must be a point where if the feasibility rate is too high it would be more cost and time effective to fund studies as RCTs and accept that a certain percentage will inevitably fail. However, for those that do fail, a greater amount of time and/or funding will have been saved by successfully completed RCTs which avoided the feasibility and/or pilot step. The cost of the ISRCTN feasibility studies was not reported, but based on the previous RfPB review [ 12 ], the average feasibility study cost £219,048 and took 31 months. If the 83% feasibility rate of ISRCTN studies is correct, then that is a substantial amount of funding and time taken up by the feasibility studies when only approximately 1 in 5 will show the RCT to not be feasible. Using the average cost of feasibility studies (£219,048) and RCTs (£1,163,996) from our previous review of RfPB studies [ 12 ], we can begin to estimate what an appropriate feasibility rate may be. RfPB feasibility studies were shown to be feasible in approximately two out of three studies, and therefore approximately £657,144 was spent on feasibility studies to save up to approximately £1,163,996 for the RCT which was not feasible. If we apply the same estimates based on the ISRCTN feasibility rate of 83% (approximately four out of five studies), then approximately £1,095,240 needs to be spent on feasibility studies to save up to approximately £1,163,996 for the RCT which was not feasible. In addition to cost, there is also a time addition as each feasibility study takes an average 31 months to complete. A feasibility rate of 83% appears too high and would suggest feasibility studies are wasteful, whereas 64% may be considered more reasonable.
How many of these RCTs genuinely needed a feasibility study and then focussed on the actual uncertainties instead of adopting a generic design? Perhaps even more important is how many of those 83% which demonstrate feasibility actually progress to RCT? Feasibility studies which show the RCT feasible but which do not progress to RCT, for various reasons, could also be considered to be adding to research waste, especially if these feasibility studies are not really needed in the first place. The answer to this question is made challenging by historic poor reporting and publication rates of feasibility studies.
This study raises, but cannot address, the question of what would be an acceptable ‘success’ rate for feasibility studies: should most demonstrate feasibility of the RCT or should more demonstrate the RCT not feasible? How much risk do funders and researchers want to take? Might shorter and more cost effective feasibility studies be more informative? Perhaps the view that feasibility studies are essential before conducting a RCT is leading to the design of studies which are likely to ‘succeed’ and therefore lack equipoise or do not focus on the most important uncertainties which need to be addressed in relation to the specific trial. As shown by the existing literature, it is often the case that feasibility studies in certain topic areas do not maximise their potential benefit and focus on the key uncertainties [ 8 , 11 ] and instead adopt a generic design.
Although we were unable to answer our initial question, a potentially more important and interesting question is what the rate of feasibility studies demonstrating the RCT is feasible should be. The previous review of the RfPB portfolio [ 12 ] showed that 64% of studies demonstrated the RCT to be feasible and this review of ISRCTN registered studies showed that 83% of studies with known/published outcomes demonstrated the RCT feasible. Are these feasibility rates appropriate and what do they mean for the wider trajectory along the trials pathway?
It is likely that there are insufficient published studies demonstrating that the RCT is not feasible to be able to assess whether some studies may or may not be more feasible than average based on research topic and/or research setting. More discussion is required between researchers, methodologists and research funders on exactly what feasibility studies are aiming to achieve and what proportion of studies should demonstrate feasibility or not and how this relates to the wider research, funder and patient benefit pathway. This will help ensure that feasibility studies maximise the potential to reduce waste in research instead of potentially adding to it.
The sample included a range of studies spanning over 20 years. During this time, research design and conduct has changed including definitions of ‘feasibility study’. To that end, we applied a relatively recent definition of ‘feasibility study’ to historical studies which made reviewing studies challenging. The potential under-reporting of non-feasible studies possibly biased the sample leading to an artificially high feasibility rate.
SZ is an NIHR Senior Investigator Emerita.
BM, JH, DA and SZ conceived the study. The work was undertaken by BM, JH and KK and supported by DA and SZ. Analysis and quality assurance were conducted by BM, JH and KK. All authors read and approved the final manuscript.
This study was funded by the NIHR Central Commissioning Facility (CCF).
Declarations.
Not applicable
BM, JH and KK are employed through NIHR to manage RfPB and its databases. SZ is the current Programme Director of RfPB, and DA was the Programme Director of RfPB until May 2017. The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and social Care.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Definition of human subjects research.
Are you planning on conducting human subjects research? Learn more about research that meets the definition human subjects research, Federal regulation requirements, and whether your project may be considered exempt. Also, learn about NIH specific considerations and become more familiar with NIH policies, and other regulations as it relates to human subjects research protections.
The questionnaire is a tool to assist you with determining whether your project involves non-exempt human subjects research, meets the criteria for exempt human subjects research, or does not involve human subjects research.
This resource summarizes the definition of human subjects research and provides examples of human subjects research projects. It also describes what you will need when you are preparing your NIH application and what is required if you are funded.
This resource is a guide to simplify the understanding of the exemptions from the federal regulations for the protection of human subjects research. It summarizes Exemptions 1, 2, 3, 4, 5, 6, 7 and 8, providing basic definitions, examples of studies that meet and do not meet the criteria of the exemption, and aspects one must consider when engaged in exempt or non-exempt human subjects research.
Studies involving the use of human specimens or data may or may not be considered to be research involving human subjects, depending on the details of the materials to be used. Use this flowchart to help determine if studies involving private information or biospecimens may meet the definition of human subjects research.
Learn about research activities which may qualify for a public health surveillance exclusion. Find useful information, key resources, and instructions for NIH applicants and offerors.
Aims Structured reporting in pathology is not universally adopted and extracting elements essential to research often requires expensive and time-intensive manual curation. The accuracy and feasibility of using large language models (LLMs) to extract essential pathology elements, for cancer research is examined here.
Methods Retrospective study of patients who underwent pathology sampling for suspected hepatocellular carcinoma and underwent Ytrrium-90 embolisation. Five pathology report elements of interest were included for evaluation. LLMs (Generative Pre-trained Transformer (GPT) 3.5 turbo and GPT-4) were used to extract elements of interest. For comparison, a rules-based, regular expressions (REGEX) approach was devised for extraction. Accuracy for each approach was calculated.
Results 88 pathology reports were identified. LLMs and REGEX were both able to extract research elements with high accuracy (average 84.1%–94.8%).
Conclusions LLMs have significant potential to simplify the extraction of research elements from pathology reporting, and therefore, accelerate the pace of cancer research.
https://doi.org/10.1136/jcp-2024-209669
Request permissions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
IMAGES
VIDEO
COMMENTS
A feasibility study is an in-depth assessment conducted to determine the practicality and viability of a proposed project or idea. It involves evaluating various factors such as technical, economic, legal, operational, and scheduling aspects to ascertain whether the project can be successfully implemented.
Feasibility Study - An overview. A Feasibility Study is an initial investigation into the potential benefits and viability of a project or endeavour. An impartial appraisal that looks at a project's technical, financial, legal, and environmental elements is what this study provides. Decision-makers can use this information to assess if the ...
A feasibility study starts with a preliminary analysis. Stakeholders are interviewed, market research is conducted, and a business plan is prepared. All of this information is analyzed to make an ...
Rigorous assessment of feasibility for studies being put forward for funding is one way of increasing the likelihood that the studies run in the NHS are well designed and likely to deliver to time and to target. Equally, a record of running successful studies that answer the key research question is a strong metric of success for funders.
A feasibility study is an essential analytical tool that evaluates the viability of a proposed project on multiple fronts, such as financials, technical requirements, and market demand. Conducted during the project initiation phase, this type of study serves as an early checkpoint to identify potential roadblocks and assess risks.
Design Options for Feasibility Studies. The choice of an optimal research design depends upon the selected area of focus. This premise holds equally for feasibility studies and for other kinds of research. As the knowledge base and needs for an intervention progress, different questions come to the fore.
Implementation trials aim to test the effects of implementation strategies on the adoption, integration or uptake of an evidence-based intervention within organisations or settings. Feasibility and pilot studies can assist with building and testing effective implementation strategies by helping to address uncertainties around design and methods, assessing potential implementation strategy ...
A feasibility study is a systematic and comprehensive analysis of a proposed project or business idea to assess its viability and potential for success. It involves evaluating various aspects such as market demand, technical feasibility, financial viability, and operational capabilities.
A feasibility study consists of research conducted before the approval of a project. It is essential to the project life cycle development as it helps determine the likelihood of success before you've spent your resources on a potential lost cause. The study helps determine a project's viability by looking at cost, resource requirements ...
Feasibility Studies are pieces of research done before a main study. They are used to estimate important parameters that are needed to design the main study. For instance: standard deviation of the outcome measure, which is needed in some cases to estimate sample size,
These definitions propose that any type of study relating to the preparation for a main study may be classified as a "feasibility study", and that the term "pilot" study represents a subset of feasibility studies that specifically look at a design feature proposed for the main trial, whether in part of in full, that is being conducted ...
A feasibility study is undertaken to determine whether there is a sufficiently high (acceptable) likelihood that a research study being considered can be successfully executed. Description. When planning a large and complex research study (particularly an experimental study such as a randomized clinical trial of a behavioral intervention), it ...
Distinguishing Feasibility and Pilot Studies. One of the clearer definitions of feasibility study and its differentiation from pilot study comes from the United Kingdom's National Institute for Health Research Evaluation, Trials and Studies Coordination Centre (NETSCC; 2012), which states, "Feasibility studies are pieces of research done before a main study in order to answer the question ...
To conduct a feasibility study, hire a trained consultant or, if you have an in-house project management office (PMO), ask if they take on this type of work. In general, here are the steps they'll take to complete this work: 1. Run a preliminary analysis. Creating a feasibility study is a time-intensive process.
In this article, we highlight the distinctive features of a feasibility study, identify the main objectives and guiding questions of a feasibility study, and illustrate the use of these objectives. We synthesized the research methods literature related to feasibility studies to identify five overarching objectives of feasibility studies that ...
These central issues of limited and asymmetric knowledge characterise feasibility studies as applied research projects. It is thus critical that feasibility studies are competently planned and managed by the owning organisation. Therefore, this paper seeks to address the research question: "How should organisations design feasibility studies ...
The word ' feasibility ' means the degree or state of being easily, conveniently, or reasonably done. If something is ' feasible,' it means that we can do it, make it, or achieve it. In other words, it is 'doable' and also 'viable.'. A viable business, for example, is one we expect will make a profit every year for a long time.
Pilot studies are a necessary first step to assess the feasibility of methods and procedures to be used in a larger study. Some consider pilot studies to be a subset of feasibility studies (), while others regard feasibility studies as a subset of pilot studies.As a result, the terms have been used interchangeably ().Pilot studies have been used to estimate effect sizes to determine the sample ...
A feasibility study provides a blueprint to determine feasibility of a business endeavor or a planned project. A feasibility study is a systematic plan and analysis of the sustainability of a ...
Definitions. A feasibility study asks whether something can be done, should we proceed with it, and if so, how. A pilot study asks the same questions but also has a specific design feature: in a pilot study a future study, or part of a future study, is conducted on a smaller scale.. As a subset of feasibility research, pilot studies may or may not be randomised.
3. Conduct a Market Survey or Perform Market Research. This step is key to the success of your feasibility study, so make your market analysis as thorough as possible. It's so important that if your organization doesn't have the resources to do a proper one, then it is advantageous to hire an outside firm to do so.
A feasibility study is an assessment of the practicality of a project or system. ... marketing research and policies, financial data, legal requirements and tax obligations. [1] Generally, feasibility studies precede technical development and project implementation. A feasibility study evaluates the project's potential for success; therefore ...
Anne M. Kolenic. feasibility studies, clinical research, nursing science, evidence-based practice. ONF 2018, 45 (5), 572-574. DOI: 10.1188/18.ONF.572-574. Preview. Nursing clinical research is a growing field, and as more nurses become engaged in conducting clinical research, feasibility studies may be their first encounter. Understanding what ...
However, feasibility studies typically do not meet the definition of research involving human subjects and therefore would not require IRB review. In order for the IRB to determine whether your activities constitute a feasibility study or pilot testing, and subsequently, whether they require IRB review, please complete and submit the initial ...
A feasibility study is a preliminary exploration of a proposed project or undertaking to determine its merits and viability. A feasibility study aims to provide an independent assessment that examines all aspects of a proposed project, including technical, economic, financial, legal, and environmental considerations.
That's why feasibility studies matter. Combine sales forecasting data with the insight from a feasibility report, and you'll be able to gauge the success rate of your proposed plan before you start. Other feasibility benefits include: Determining if the project is appropriate for your team. Making sound decisions for your team.
The study team's findings highlighted the beneficial role of greenness and access to green or blue spaces in reducing socioeconomic-related inequalities in mental health. They also discovered every additional 360m to the nearest green or blue space was associated with higher odds of anxiety and depression.
Feasibility studies are often conducted before committing to a randomised controlled trial (RCT), yet there is little published evidence to inform how useful feasibility studies are, especially in terms of adding or reducing waste in research. This study ...
Studies involving the use of human specimens or data may or may not be considered to be research involving human subjects, depending on the details of the materials to be used. Use this flowchart to help determine if studies involving private information or biospecimens may meet the definition of human subjects research.
Aims Structured reporting in pathology is not universally adopted and extracting elements essential to research often requires expensive and time-intensive manual curation. The accuracy and feasibility of using large language models (LLMs) to extract essential pathology elements, for cancer research is examined here. Methods Retrospective study of patients who underwent pathology sampling for ...