Skip to main content
Download PDF

Recommendations for optimising pilot and feasibility work in surgery

Pilot and Feasibility Studies volume 10, Article number: 64 (2024) Cite this article

Abstract

Background

Surgical trials are recognised as inherently challenging. Pilot and feasibility studies (PAFS) are increasingly acknowledged as a key method to optimise the design and conduct of randomised trials but remain limited in surgery. We used a mixed methods approach to develop recommendations for how surgical PAFS could be optimised.

Methods

The findings from a quantitative analysis of funded surgical PAFS over a 10-year period and in-depth qualitative interviews with surgeons, methodologists and funders were triangulated and synthesised with available methodological guidance on PAFS.

Results

The synthesis informed the development of an explanatory model describing root causes and compounding challenges that contribute to how and why surgical PAFS is not currently optimised. The four root causes identified include issues relating to (i) understanding the full scope of PAFS; (ii) design and conduct of PAFS; (iii) reporting of PAFS; and (iv) lack of appreciation of the value of PAFS by all stakeholder groups. Compounding challenges relate to both cultural issues and access to and interpretation of available methodological PAFS guidance. The study findings and explanatory model were used to inform the development of a practical guidance tool for surgeons and study teams to improve research practice.

Conclusions

Optimisation of PAFS in surgery requires a cultural shift in research practice amongst funders, academic institutions, regulatory bodies and journal editors, as well as amongst surgeons. Our ‘Top Tips’ guidance tool offers an accessible framework for surgeons designing PAFS. Adoption and utilisation of these recommendations will optimise surgical PAFS, facilitating successful and efficient future surgical trials.

Peer Review reports

Background

Surgical research is fundamentally challenging because surgery is a complex intervention. Unlike in pharmacological studies, surgical interventions are more complicated to standardise, define and compare with other interventions [1]. The challenges of designing, conducting and evaluating surgical trials have been well documented [2,3,4,5,6,7] and may be considered largely related to methodological or cultural issues. Key methodological challenges of surgical trials include recruitment, intervention stability and standardisation, and outcome selection and/or measurement. Cultural issues often compound methodological challenges and include problems with equipoise amongst surgeons, inexperience recruiting to Randomised Controlled Trials (RCTs) and lack of understanding of the multidisciplinary (non-clinical) expertise needed for definitive trial success. Although many of the practical and methodological challenges of evaluating surgical interventions are common to the assessment of all complex, non-pharmacological interventions, surgery is unique in that multiple challenges often coincide [2].

Pilot and feasibility studies are increasingly recognised as a key component for the success of subsequent definitive trials (studies appropriately powered to achieve an effect size and consequently definitively answer a research question). Definitions of the terms ‘pilot’ and ‘feasibility’ have evolved in the literature over time, with terms often used interchangeably and without universally accepted definitions [8,9,10]. Several major funders in the UK such as the National Institute for Health and Care Research (NIHR) and the Medical Research Council (MRC), have now adopted the conceptual framework to classify pilot and feasibility studies published by Eldridge et al. in 2016 [8]. However, for the purposes of this work, pilot and feasibility studies (PAFS) are defined broadly as ‘Any research undertaken before a main study that is explicitly intended to inform the design and/or conduct of a future main study.’

PAFS have value in informing the design and conduct of surgical trials because they face unique complexities, and often interacting uncertainties surrounding the design, conduct and completion of trials, meaning there is an even greater need to consider if and how surgical trials can be improved. Indeed, funders encourage consideration of feasibility before an agreement to fund a definitive trial is reached [11]. PAFS may help avoid poor research design, conduct and analysis, all of which are known to contribute to significant research waste [12,13,14,15,16]. By avoiding common problems such as the inability to recruit and a corresponding reduction in statistical power, excessive attrition due to intolerable procedures and cross-over between treatment groups, it has been suggested that PAFS may reduce the proportion of failed trials [17].

Whilst published guidance from the MRC [1, 18] and the IDEAL (Idea, Development, Evaluation, Assessment, Long-term follow-up) collaboration [19, 20] emphasises the importance of PAFS for trials of complex interventions such as surgery, accessible practical guidance tailored specifically to optimally design and undertake PAFS for surgical trials is lacking. The conduct, reporting and publication of PAFS in surgery remain rare [21, 22], and guidance such as the IDEAL recommendations, a framework for evaluating and reporting surgical innovation, has yet to lead to a demonstrable improvement in the performance and publication of surgical PAFS [23]. Furthermore, there is no surgery-specific guidance endorsed by funding bodies or professional membership organisations such as the Royal College of Surgeons (RCS) tailored to surgeons participating in or contemplating designing PAFS.

Accessible guidance specifically for surgeons, which distils the important methodological messages for designing and conducting PAFS into a practical useable framework, is needed. The aim of this work was, to generate a detailed understanding of the challenges of PAFS in surgery and to develop clear and practical recommendations for surgeons for how to optimise the design and conduct of surgical PAFS in the future. Such guidance will improve the understanding of the true purpose of PAFS in the context of surgical research, drive up the quality of research applications, optimise reporting and ultimately improve the quality and value of surgical RCTs, thereby significantly reducing research waste.

Methods

Our recently published 10-year review of NIHR-funded surgical PAFS [24] established that the full potential of PAFS to address the uncertainties and challenges specific to undertaking surgical trials is yet to be achieved. Findings from this review and from in-depth qualitative interviews with key stakeholders exploring the challenges and barriers to undertaking PAFS in surgery were synthesised to identify key factors contributing to sub-optimal surgical PAFS and develop practical recommendations for the design and conduct of PAFS in surgery. The synthesis triangulated data from both the quantitative review and the qualitative interviews, with available guidance on the importance of PAFS for trials of complex interventions [1, 19, 20] and wider methodological literature on PAFS more generally [8, 25,26,27,28,29]. Figure 1 illustrates the study process.

Fig. 1
figure 1

Study process of data collection, synthesis and analysis

Full size image

Interview study participant sampling

A list of potential participants was created by considering a long list of trial team members of studies included in our published 10-year review of NIHR-funded surgical PAFS [24]; Senior members of clinical trial units (CTUs) and RCS Trials Centres in the UK; Senior authors of published literature on PAFS work methodology; Senior editors of surgical journals and those publishing PAFS specifically and; Senior funding panel members of major funding bodies in the UK including NIHR, Cancer Research UK (CRUK), Chief Scientist Office (CSO) and Arthritis Research UK (ARUK). Senior participants were deliberately targeted as it was hypothesised they would have the greatest breadth and range of experience in the area of PAFS design and conduct, to consequently allow the extraction of information about the challenges and barriers to actually conducting pre-trial research from those with the most extensive experience. Participants were purposively sampled to achieve maximum variation based on the geographical place of work, clinical vs non-clinical expertise; clinical speciality (if applicable), areas of expertise and research roles. Snowballing sampling was also used, by asking participants during the interviews, if they had suggestions for other participants who may have alternative, relevant or important perspectives for this work.

Interview data collection and analysis

Interviews were conducted either face-to-face or by telephone at times and locations convenient to the participants, using a topic guide. This semi-structured approach provided a tool to frame the interviews and offer prompts for the interviewer whilst also allowing key topics of importance to participants to emerge naturally and be further explored (Appendix 1). Data analysis used an inductive thematic approach underpinned by the principles of grounded theory [30] using NVivo 10 software [31]. Sampling, data collection and analysis were undertaken concurrently and iteratively until no new themes emerged and data saturation was achieved.

Results

Demographics of Interview study participants

A total of 33 participants were invited to participate in an interview of whom 28 (85%) expressed an interest and 27 (81%) consented and were interviewed. These included 18 (67%) males and 9 (33%) females. Of the interviewed participants, 11 (41%) were surgeons, 16 (59%) methodologists and 20 (74%) funders. Surgeon participants included representatives of a broad spectrum of surgical specialties from centres across the UK. All the surgeon participants were currently involved in surgical research, but experience of involvement in PAFS varied widely. Trial methodologist participants included CTU directors and trial statisticians, all of whom reported experience in designing and conducting trials of complex interventions and PAFS, with a smaller proportion having had specific experience in surgical trials (3/16).

More than two-thirds (20/27) of participants interviewed had current or recent experience of membership on a UK research funding body panel including as a panel chair (n = 6), deputy chair (n = 3) or panel member (n = 12). More than half (15/27) were members of journal editorial boards and most (24/27) also currently held a professorial position at a UK university. Table 1 shows further interview participant demographics.

Table 1 Interview participant (n = 27) summary of demographic characteristics and surgical research/trials experience
Full size table

As shown in Fig. 1 and described in the methods, phase 3 of this work synthesised the findings from our published 10-year review of NIHR-funded surgical PAFS [24], and the n = 27 qualitative interviews with key stakeholders, as well as available methodological guidance on PAFS [1, 8, 19, 20, 25,26,27,28,29]. This synthesis resulted in four root causes for why PAFS in surgery are  not currently optimised as shown in Fig. 2. These root causes emerged from the data as issues relating to (1) understanding the full scope of PAFS; (2) the design and conduct of PAFS; (3) the reporting of PAFS; and (4) the undervaluation of PAFS by surgeons, journal editors, academic institutions and sometimes funders. In addition to the root causes identified for why PAFS in surgery are not currently optimised, compounding factors were identified which are linked to both the root causes and to each other. These factors relate to the challenges of (1) current guidance and (2) cultural issues surrounding both surgical research in general and PAFS more specifically. These challenges can impact at different and multiple points in the cycle, and thereby act as barriers to improving research practice (see Fig. 2). Each of the root causes and compounding factors identified by this work are described in detail below, supported by participant quotes (see Table 2) and references to the other data sources synthesised to produce these results.

Fig. 2
figure 2

Cyclical model of sub-optimisation of PAFS illustrating the linked and co-influential root causes and compounding challenges underpinning why surgical PAFS are not currently optimised

Full size image
Table 2 Quotes from interview participants
Full size table

Root cause 1: surgical PAFS are not optimally understood

A key finding from all data sources was that the purpose and scope of PAFS were not well understood by the surgical research community. The analysis of NIHR-funded PAFS protocols [24], for example, demonstrated that nearly a quarter of PAFS studies planned to conduct formal hypothesis testing (8/35, 23%). Interview data confirmed that many surgeons perceived PAFS to be small underpowered RCTs, designed and reported with the opportunity to test certain outcomes including safety and effectiveness [see Table 2 Quotes 1 and 2]. Whilst methodologists understood the full scope of PAFS to explore the breadth of areas of uncertainty about a potential main trial [see Table 2 Quote 3], surgeons were generally less specific about the multiple areas of uncertainty that could be explored and tended to focus on recruitment as the primary area of uncertainty that could be addressed [see Table 2 Quote 4].

Root cause 2: surgical PAFS are not optimally conducted

Misunderstanding of what PAFS are and why they should be done impacts the range of areas of uncertainty that surgeons will seek to explore in PAFS. Consequently, PAFS in surgery are not optimally conducted [see Table 2 Quote 5]. In particular, there is a tendency for surgical PAFS [24] to focus on issues that are generic to all trials, such as recruitment, rather than exploring and addressing key uncertainties of specific relevance to surgical trials, such as intervention stability and the learning curve. Interviews with methodologists highlighted that surgeons applying for funding to undertake PAFS tended to follow ‘example’ lists provided by funders of what to consider investigating in PAFS, rather than thinking about the specific needs of their trial [see Table 2 Quote 6]. The importance of exploring the intervention protocol (which also includes careful consideration of co-interventions) in surgical PAFS was particularly underappreciated [see Table 1 Quote 7].

Root cause 3: surgical PAFS are not optimally reported

Both the quantitative and qualitative work provided evidence that surgical PAFS are currently not well reported. Data from the review of NIHR-funded surgical PAFS illustrated that PAFS in surgery is under-reported, with only two-thirds of surgical PAFS studies funded by the NIHR between 2005 and 2015 publishing study findings [24]. In addition, interview findings demonstrated that PAFS are still masqueraded as full RCTs when submitted to journals and that underpowered RCTs are badged as PAFS a posteriori [see Table 2 Quote 8].

Root cause 4: surgical PAFS are undervalued

The interview study provided an explanation for why PAFS might be sub-optimally conducted and reported [24], by illustrating that PAFS were undervalued by all key stakeholder groups. Funders, for example, perceived that many PAFS had historically been conducted as standalone pieces of work with no intention of the study team or funders to progress to a main trial. PAFS were consequently undervalued as being ineffectual and not worth investment [see Table 2 Quotes 9 and 10]. Similarly, academic institutions were perceived to undervalue PAFS, considering them low-impact studies, which do not contribute significantly to the Research Excellence Framework (REF) as high-impact papers, and often have no outputs at all [see Table 2 Quote 11]. This is perpetuated by journal editors and peer reviewers, some of whom undervalue PAFS and consider them of limited interest as they do not offer definitive practice-changing results [see Table 2 Quote 12]. Such editorial practice may perpetuate the cycle of misunderstanding; if definitive results are requested, authors may feel compelled to produce them to achieve publication, thus small underpowered RCTs veiled as PAFS will continue to litter the literature, which further perpetuates misunderstanding [see Table 2 Quote 13]. Whilst there is now this journal dedicated to the reporting and publication of PAFS (The Journal of Pilot and Feasibility Studies [32]), without investment from journal editors, academic institutions and funders to drive the importance of accurately publishing pre-trial work, the cycle of sub-optimisation of PAFS will continue.

Compounding factor 1: challenges with current guidance

The interviews identified challenges with both guidance provided by funders, and information in the methodological literature conceptualising the types and purposes of PAFS and describing methods for reporting PAFS [8, 25,26,27,28,29]. The funder guidance was perceived as being limited, variable and sometimes contradictory to the definitions given in the methodological guidance [see Table 2 Quote 14]. Whilst most methodologists recognised the extensive methodological work already undertaken [8, 25,26,27,28,29], many felt this work to be inaccessible and poorly disseminated to surgeons [see Table 2 Quote 15].

There was limited awareness of the existence of the current methodological work amongst surgeons. The few surgeons who did mention this perceived the methodological literature as largely theoretical, generic and difficult to operationalise, thereby making it mostly unhelpful [see Table 2 Quotes 16]. This finding indicates that the methodological work is poorly understood and not widely acknowledged beyond the methodological community [see Table 2 Quote 17]. Guidance from the IDEAL collaboration [19, 33], widely considered as the conceptual work most aligned with surgeons and surgical trials, was not perceived in the interview study to be widely accepted or utilised amongst surgeons [see Table 2 Quote 18]. It is encouraging that the newest NIHR guidance first published in 2019 [34] does now signpost and reference the underpinning methodological work in this area.

If available methodological guidance is not effectively operationalised to be of practical use to surgeons [see Table 2 Quote 17], its inaccessibility is compounded, further adding to confusion so that it is consequently misunderstood or ignored. In addition, surgeons recognised significant barriers to being able to access Clinical Trials Units (CTUs) and methodological expertise, which was often perceived as not possible within PAFS funding envelopes [see Table 2 Quotes 19 and 20].

Compounding factor 2: cultural challenges

Many of the existing challenges for surgical trials have already reported [2,3,4,5,6,7] impact on PAFS and, in doing so, make PAFS potentially even more relevant in surgery. There has been a blossoming culture of surgical research partnerships and cross-specialty collaboration in recent years through, for example, the formation of nationwide surgical trainee research collaboratives and the RCS of England Surgical Trials Initiative [35]. Such cultural changes have undoubtedly contributed significantly to raising the profile of surgical research and, more specifically collaboratively conducting surgical trials [36,37,38]. However, of all clinical research funded by the NIHR Health Technology Assessment (HTA) and Research for Patient Benefit (RfPB) programmes from 2005 to 2015, only 10.4% (140/1341) were studies where surgery was the main intervention [24]. Most funders felt that the reason surgical research was less frequently funded was because the questions being asked were not important enough to the National Health Service (NHS) or to patients [see Table 2 Quote 21]. In contrast, surgeons perceived a lack of surgical representation on funding panels, and competition with translational science and experienced research teams for funding, as significant barriers to fair funding opportunities [see Table 2 Quotes 22 and 23].

Surgeons perceived other barriers to funding related to the relative lack of regulation for the formal evaluation of new surgical procedures and surgical devices. This was observed to have led to a lack of research infrastructure within the industry, resulting in fewer avenues for funding surgical research when compared, for example, to pharmaceutical research [see Table 2 Quotes 24 and 25].

Most funders still offer ‘uncoupled’ funding, where a PAFS is funded without a firm promise of funding for a subsequent main trial. This system was perceived by most interview participants as inefficient, due to the additional time and resources needed to perform standalone pre-trial work, and therefore a further barrier to completing PAFS [see Table 2 Quote 26].

Finally, both surgeons and methodologists perceived undertaking PAFS to lengthen the process of trial research, meaning answers to important questions took longer to attain. Consequently, it was considered that the research question may become obsolete before pre-trial work is completed, particularly in fast-moving clinical areas such as surgery [see Table 2 Quote 27 and 28].

Discussion

This is the first published work to specifically consider current research practice for PAFS in surgery and to explore the explicit challenges and barriers preventing optimal conduct of PAFS in surgery. This work has identified four key areas for improvement to research practice necessary to optimise future PAFS in surgery. These have informed the development of broad recommendations, summarised in Table 3, which require a wider cultural shift in research practice amongst funders, academic institutions, regulatory bodies and journal editors, as well as amongst surgeons. Whilst the recommendations from this work are focused on PAFS in surgery, many may be relevant to the wider context of complex interventions as a whole. As part of a future consensus process, this question could perhaps be addressed, especially in light of similar work in other areas [39,40,41].

Table 3 Recommendations for change to improve wider research practice around P/FS
Full size table

Examining the literature, it seems PAFS may be less commonly done in countries outside the UK. A systematic review by the methodology group which produced the conceptual framework of the definitions of PAFS, looked at the quality of reporting of 18 pilot and feasibility cluster randomised trials conducted and published between 2011 and 2014 [42]. This study found that half (56%) were set in the UK, with all other countries represented only once, apart from Canada (three studies) and the United States of America (USA) (two studies). In addition, it was noted in our study, that the UK-produced methodological guidance [8, 26] was perceived to have not been incorporated into practice by authors from overseas yet.

Whilst the focus of this work was entirely on research and funding practice in the UK, it is perhaps reasonable to suggest that the UK is leading the way in developing a methodology for the design and conduct of PAFS and that with further exploration and collaboration, the findings of this research could well be relevant to researchers in other countries. A limitation of this work is that only senior surgeons and methodologists were sampled. Surgeons leading the trainee surgical research collaboratives and research-naïve surgeons for example may have provided differing perspectives and potential solutions to the challenges encountered. However, the well-documented issues with inappropriate reporting of both underpowered RCTs as PAFS and vice versa [43,44,45] indicate a widespread misunderstanding of the value and purpose of PAFS. It was therefore deemed important to concentrate on extracting data from the most experienced and data-rich sources, hence focusing on experienced participants for the interviews, and systematic analysis of NIHR-funded PAFS (as opposed to performing a traditional systematic or narrative review of the literature). In addition, whilst patients were not involved in this work as the aim was to specifically explore the methodological and cultural barriers and challenges of completing PAFS from a professional perspective, involving patients in the design and delivery of PAFS is vitally important, and future work in the area will need to include the patient perspective.

Conclusion

This work identified the need for accessible, operationalised guidance for surgeons designing and conducting surgical PAFS. Our ‘Top Tips’ guidance tool for surgeons (Fig. 3) offers a practical framework for surgeons designing and writing funding applications for PAFS. The guidance operationalises and bridges the current gap between the available methodological guidance and the broader recommendations for improving research practice made here (see Table 3). These top tips include defining the purpose of PAFS, identifying uncertainties of specific relevance to surgery to be considered and engaging with methodologist support early and systematic reporting of PAFS, with references to key methodological resources. Both the recommendations for cultural changes (Table 3) and the practical guidance tips for surgeons (Fig. 3) are intended to optimise future best research practice around the design and conduct of surgical PAFS. Adoption of these recommendations will, therefore, facilitate successful and efficient surgical trials in the future and, ultimately, improve the evidence base for surgeons and patients.

Fig. 3
figure 3

Top Tips for surgeons designing and conducting pilot and feasibility studies

Full size image

Availability of data and materials

The datasets generated and analysed during the current study are stored under the provisions of the Data Protection Act 2018 and the University of Bristol requirements but are not publicly available due to the potentially sensitive nature of the data, and the risk of re-identification of participants. Data may be made available to bona fida researchers only, on reasonable request to the corresponding author after their host institution has signed a Data Access Agreement.

Abbreviations

PAFS:

Pilot and feasibility studies

RCTs:

Randomised Controlled Trials

UK:

United Kingdom

NIHR:

National Institute for Health and Care Research

MRC:

Medical Research Council

RCS:

Royal College of Surgeons

CRUK:

Cancer Research UK

CSO:

Chief Scientist Office

ARUK:

Arthritis Research UK

CTUs:

Clinical Trial Units

IDEAL collaboration:

Idea, Development, Evaluation, Assessment, Long-term follow-up

HTA:

Health Technology Assessment

RfPB:

Research for Patient Benefit

USA:

United States of America

REF:

Research Excellence Framework

References

  1. Medical Research Council (MRC). Developing and evaluating complex interventions: new guidance. London: Medical Research Council; 2008.

    Google Scholar 

  2. Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, et al. Challenges in evaluating surgical innovation. Lancet. 2009;374(9695):1097–104.

    Article  PubMed  PubMed Central  Google Scholar 

  3. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324(7351):1448–51.

    Article  PubMed  PubMed Central  Google Scholar 

  4. National Cancer Research Institute (NCRI). Challenges and opportunities in surgical cancer research in the UK. London: National Cancer Research Institute; 2012.

    Google Scholar 

  5. Cook JA. The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials. 2009;10:9.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Byrne BE, Rooshenas L, Lambert HS, Blazeby JM. A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice. BMC Med Res Methodol. 2020;20(1):112.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Farrokhyar F, Karanicolas PJ, Thoma A, Simunovic M, Bhandari M, Devereaux PJ, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251(3):409–16.

    Article  PubMed  Google Scholar 

  8. Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. Plos One. 2016;11(3):e0150205.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

    Article  PubMed  Google Scholar 

  10. O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Goode J, Hewison J. Maximising the value of combining qualitative research and randomised controlled trials in health research: the QUAlitative Research in Trials (QUART) study–a mixed methods study. Health Technol Asses. 2014;18(38):1–197 (v-vi).

    Google Scholar 

  11. National Institute for Health and Care Research (NIHR). Clinical Trials Toolkit. 2023. [Available from: https://www.ct-toolkit.ac.uk/routemap/feasibility-and-investigator-selection/]. Accessed 22 Aug 23.

  12. Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85.

    Article  PubMed  Google Scholar 

  13. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

    Article  PubMed  Google Scholar 

  14. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.

    Article  PubMed  Google Scholar 

  16. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9.

    Article  PubMed  Google Scholar 

  18. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update. Health Technol Assess. 2021;25(57):1–132.

    Article  PubMed  PubMed Central  Google Scholar 

  19. McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374(9695):1105–12.

    Article  PubMed  Google Scholar 

  20. Hirst A, Philippou Y, Blazeby J, Campbell B, Campbell M, Feinberg J, et al. No surgical innovation without evaluation: evolution and further development of the IDEAL framework and recommendations. Ann Surg. 2019;269:211–20.

    Article  PubMed  Google Scholar 

  21. Fairhurst KAK, O’Connell Francischetto E, Metcalfe C, Blazeby J. How can pilot work optimally inform surgical RCTs? A review of current evidence. Trials. 2015;16(Suppl 2):17.

    Article  Google Scholar 

  22. O’Connell Francischetto EAK, Metcalfe C, Williamson P, Gamble C, Blazeby J. Optimising the design and evaluation of pilot work to inform the main trial: a review of current evidence and consideration of future practices. Trials. 2013;14(Suppl 1):O17.

    Article  PubMed Central  Google Scholar 

  23. McCulloch P, Feinberg J, Philippou Y, Kolias A, Kehoe S, Lancaster G, et al. Progress in clinical research in surgery and IDEAL. Lancet. 2018;392(10141):88–94.

    Article  PubMed  Google Scholar 

  24. Fairhurst K, Blazeby JM, Potter S, Gamble C, Rowlands C, Avery KNL. Value of surgical pilot and feasibility study protocols. Br J Surg. 2019;106(8):968–78.

    Article  CAS  PubMed  Google Scholar 

  25. Eldridge SBC, Campbell M, Lancaster G, Thabane L, Hopewell S. Definition and reporting of pilot and feasibility studies. Trials. 2013;14(Supplement 1):O18.

    Article  PubMed Central  Google Scholar 

  26. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibil Stud. 2016;2:64.

    Article  Google Scholar 

  28. Eldridge SBC, Campbell M, Hopewell S, Thabane L, Lancaster G, Coleman C. Defining feasibility and pilot studies in preparation for randomised controlled trials: using consensus methods and validation to develop a conceptual framework. Trials. 2015;16(Suppl 2):087.

    Article  Google Scholar 

  29. Thabane L, Hopewell S, Lancaster GA, Bond CM, Coleman CL, Campbell MJ, et al. Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot Feasibil Stud. 2016;2(1):1–13.

    Google Scholar 

  30. Braun VCV. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.

    Article  Google Scholar 

  31. http://www.qsrinternational.com/nvivo [Accessed 22.08.2023]

  32. Lancaster GA. Pilot and feasibility studies come of age! Pilot Feasibil Stud. 2015;1(1):1–4.

    Article  Google Scholar 

  33. IDEAL Collaboration. http://www.ideal-collaboration.net/ [Accessed 09.11.2022].

  34. National Institute for Health and Care Research (NIHR). RfPB programme guidance on applying for feasibility studies [Available from: https://www.nihr.ac.uk/documents/nihr-research-for-patient-benefit-rfpb-programme-guidance-on-applying-for-feasibility-studies/20474?pr]. Accessed 2 Sept 2019.

  35. Royal College of Surgeons of England (RCSEng). https://www.rcseng.ac.uk/standards-and-research/research/surgical-trials-initiative/. Accessed 28 June 2017.

  36. Royal College of Surgeons of England (RCSEng). Surgical Research Report 2017–2018. 2017 [Available from: https://www.rcseng.ac.uk/library-and-publications/rcs-publications/docs/rcs-surgical-research-report-2017/]. Accessed 9 Nov 2022.

  37. Pinkney TD, Calvert M, Bartlett DC, Gheorghe A, Redman V, Dowswell G, et al. Impact of wound edge protection devices on surgical site infection after laparotomy: multicentre randomised controlled trial (ROSSINI Trial). BMJ. 2013;347:f4305.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Jamjoom AA, Phan PN, Hutchinson PJ, Kolias AG. Surgical trainee research collaboratives in the UK: an observational study of research activity and publication productivity. BMJ Open. 2016;6(2):e010374.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Peter Craig AM, Browne S, Simpson SA, Wight D, Robling M, Moore G, Hallingberg B, Segrott J, Turley R, Murphy S, Moore L. Development of guidance for feasibility studies to decide whether and how to proceed to full-scale evaluation of complex public health interventions: a systematic review. Lancet. 2018;392:7.

    Article  Google Scholar 

  40. Moore L, Hallingberg B, Wight D, Turley R, Segrott J, Craig P, et al. Exploratory studies to inform full-scale evaluations of complex public health interventions: the need for guidance. J Epidemiol Community Health. 2018;72(10):865–6.

    Article  PubMed  Google Scholar 

  41. Hallingberg B, Turley R, Segrott J, Wight D, Craig P, Moore L, et al. Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance. Pilot Feasibil Stud. 2018;4:104.

    Article  Google Scholar 

  42. Chan CL, Leyrat C, Eldridge SM. Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review. BMJ Open. 2017;7(11):e016970.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10:67.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Loscalzo J. Pilot trials in clinical research: of what value are they? Circulation. 2009;119(13):1694–6.

    Article  PubMed  Google Scholar 

  45. Shanyinde M, Pickering RM, Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol. 2011;11:117.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the interview participants, though they remain anonymous, for without them this work would not have happened. Exceptionally busy people gave up their valuable time to talk to us, and we are extremely grateful for their generosity and engagement with this work.

Funding

This work was supported by an MRC Hub for Trials Methodology Research (HTMR) PhD studentship for Katherine Fairhurst. JMB and KA are supported by the NIHR Biomedical Research Centre (BRC) at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol and the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomised controlled Trials In Invasive procedures) for Trials Methodology Research (MR/K0256/43/1) (https://www.bristol.ac.uk/population-health-sciences/centres/surgical-research).

Author information

Authors and Affiliations

  1. Centre for Surgical Research, Medical Research Council ConDuCT-II Hub for Trials Methodology Research and Biomedical Research Centre, Department of Population Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Clifton, Bristol, BS8 2PS, UK

    K. Fairhurst, S. Potter, J. M. Blazeby & K. N. L. Avery

Authors
  1. K. Fairhurst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Potter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. Blazeby
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. N. L. Avery
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KF collected the data, performed the data analysis and interpretation and drafted the manuscript under the guidance and supervision of SP, JB and KA. All authors helped conceive the ideas for the work, contributed to the methodological design, revised the manuscript and have agreed the final version for submission.

Authors’ information

The views expressed are those of the authors and not necessarily those of the UK NHS, NIHR, Department of Health or MRC. SP is an NIHR Clinician Scientist. JMB is an NIHR Senior Investigator.

Corresponding author

Correspondence to K. Fairhurst.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for this work (Application 41001) was granted by the University of Bristol Faculty of Health Sciences Research Ethics Committee (FREC) on 13th October 2016. All participants provided written consent to be interviewed.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. Appendix 1.

Semi-structured interview topic guide.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fairhurst, K., Potter, S., Blazeby, J.M. et al. Recommendations for optimising pilot and feasibility work in surgery. Pilot Feasibility Stud 10, 64 (2024). https://0-doi-org.brum.beds.ac.uk/10.1186/s40814-024-01489-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s40814-024-01489-1

Keywords

Pilot and Feasibility Studies

ISSN: 2055-5784