Article Text
Abstract
Introduction Patients with long-term conditions consistently report a lack of information around services and support available to them. This unmet need for information is significant among people with dementia and family carers. A quality improvement intervention is being carried out to tackle this issue as part of a co-creation initiative in the North East of England (UK). The intervention consists of the dissemination (via the local Community Mental Health Services for Older People) of a leaflet about services available to people with dementia and their family carers in the study site. This protocol is reported in accordance with the Standards for Reporting Implementation Studies.
Methods and analysis This effectiveness–implementation hybrid type 2 study aims at understanding (1) the unfolding and outcomes of the implementation strategy, (2) the outcomes of the intervention (for people with dementia and family carers, staff implementing the intervention and local service providers) and (3) the contribution of co-creation to the design and implementation of the intervention and its outcomes. The prospective theory of change of the intervention articulated by local stakeholders is used as a reference framework against which to assess the implementation and outcomes of the intervention. Evaluation data will be collected through in-depth interviews with people with dementia and family carers receiving the intervention, staff implementing the intervention and managers from local service providers. Referral data from local service providers will be collected to triangulate the interview data. A focus group with key stakeholders will support the sense-making of findings. The realist configuration of mechanism–context–outcome, operationalised using an information behaviour model, will inform data analysis and interpretation.
Ethics and dissemination Ethical and research governance approvals have been obtained from the West Midlands—South Birmingham Research Ethics Committee. The results of the study will be submitted for publication in peer-reviewed journals and disseminated through conferences.
- dementia
- quality in health care
- qualitative research
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The study is reported in accordance with the Standards for Reporting Implementation Studies framework complemented by additional relevant reporting standards, such as the Template for Intervention Description and Replication checklist and Proctor et al’s framework for specifying and reporting implementation strategies.
The evaluation framework integrates the evaluation of the implementation strategy and of the implemented intervention both in terms of process and of outcomes.
The mixed-method evaluation uses data collected through in-depth interviews with three sets of stakeholders, services’ referral data and a focus group.
In-depth, longitudinal interviews with people with dementia are planned to explore their experiences of receiving and using the leaflet; however, recruitment can be difficult and dropout rate can be high.
The evaluation will provide insights regarding the process and impact of delivering and receiving the intervention but will have no comparison condition.
Introduction
Patient activation, self-management, ageing in place, enablement and shared decision-making are recurrent themes in policy discourses across health systems addressing ageing populations, chronic diseases, multimorbidity and complex needs.
Underpinning, although often implicit, assumptions of such policy discourses is the idea that by providing information, patients and family carers are supported to make better choices, have more agency on managing their own health (or the health of the person they care for), may feel empowered and ultimately may report better outcomes and satisfaction with the care and support they receive.
However, such a view fails to acknowledge the ‘information work’, that is, the process of seeking, using, evaluating and sharing of information1 that is required from patients and family carers in order for them to be able to make better choices or to play a more active role in their own or others’ care.2 Also, it assumes that patients and family carers are physically and cognitively able to carry out information work, and it overlooks the emotional processes that influence adaptation to long-term conditions, which may influence whether information work is successfully undertaken.3
Extant research both documents and describes the unmet information need of patients and family carers and also offers possible explanations for it.4–6 This issue has been recognised as highly significant among people with dementia and family carers,7–11 whose information needs span beyond the topics usually addressed by information materials available, leaving them with some information gaps.12 13 Moreover, people living with dementia identify a preference to receive information directly from health and social care professionals rather than over the internet, which is the primary source of information to which they are often signposted.14 They highlight that the timeliness of their information needs, which vary along the dementia trajectory,15 is seldom taken into account by information providers and resources. They also expect up-to-date information to be provided in a culturally acceptable way and in easy to understand language.9 16 17 Several user-centred models in the field of information science have been developed to unpack users’ information behaviour and explore how the actual process of information need, seeking and use occurs, in order to explain the observed variations in information need.18 Notwithstanding the specificities of each, these models consistently acknowledge that information is context sensitive and uniquely dependent on the individual user, who actively constructs its subjective meaning.19
Building on this research, an implementation study is being carried out as part of a co-creation initiative in dementia care taking place in the North East of England (UK). The study aims at designing, implementing and evaluating a leaflet about local services providing care and support to people with dementia and their family carers living in the study site.
This article reports the protocol for the evaluation of the implementation study in accordance with the Standards for Reporting Implementation Studies (StaRI) framework and adopts the recommended dual strands for reporting, one about the intervention being implemented and the other about its implementation strategy (ie, the ‘bundle’ of techniques used to enhance the adoption, integration into routine practice and sustainability of the intervention).20 21
The evaluation has three aims. First, it intends to review the unfolding and outcomes of the implementation strategy. Second, it aims to assess the outcomes of the intervention as perceived by people living with dementia and their family carers, staff implementing the intervention and local organisations providing care and support to people with dementia and family carers and to explain how and why the intervention contributed to achieve these outcomes. Third, it will reflect on the extent to which, how and why the co-creation initiative has influenced the design, implementation and impact of the intervention.
Intervention development
The implementation study herein described has been developed as part of a co-creation initiative in dementia care taking place in the North East of England (UK). The initiative, started in 2015 and with expected completion at the end of 2020, involves a constellation of local stakeholders, including people living with dementia and their family carers, health and care professionals, service managers and commissioners alongside a research team, encompassing university-based researchers and a local research facilitator.
Grounded in the principles of action research, co-creation activities were organised in three broad phases (figure 1).22 The diagnostic phase (phase 1, 2015–2018) mapped the local dementia care system and identified its strengths and weaknesses. The improvement phase (phase 2, 2017 and 2018) articulated and designed three interventions to address the priority issues identified in the diagnostic phase. Each improvement area was addressed via a subcycle of action research. Local stakeholders with an investment in the respective improvement areas were invited to work with the research team to identify different available options to address each area, select a course of action and accordingly design an intervention and define its implementation.
The planned evaluation (phase 3, 2020) will focus on both the improvement interventions and the co-creation initiative. With respect to the first, the evaluation aims to assess the outcomes of the implementation strategy and the outcomes of each intervention. With respect to the latter, the evaluation aims to unpack how the co-creation initiative has unfolded in practice and its impact on local policy-making around care and support for people with dementia.
The intervention and the implementation strategy
A fundamental unmet need for information was reported by people with dementia and family carers who took part in the diagnostic phase. Local stakeholders agreed to tackle this issue by providing up-to-date, accessible and timely information about local services available to those living with dementia and family carers. Figure 2 provides a timeline of the process underpinning the development of the agreed intervention.
A local working group (including commissioners, statutory service providers, third sector organisations providing support and care to people with dementia and family carers, a professional designer, alongside the research team) was set up with the remit of developing an information leaflet and planning its implementation. Efforts to involve family carers and people with dementia by asking local third sector organisations to identify people keen to be involved in the working group resulted in one family carer joining the group. Another turned down the offer because of competing commitments and no one living with dementia was identified. To mitigate their absence, inputs from front-line staff providing daily direct support and care to people with dementia and to family carers were used (one was also a family carer). In a similar vein, we relied also on the professional designer who had extensive experience and expertise of designing information material intended to be used by people with cognitive impairment.
The group produced three prototypes which were shared and discussed in a consultation exercise involving about 20 among people with dementia and family carers. The consultation exercise was planned around the schedule of activities or groups organised locally for people with dementia and/or family carers. This solution overcame some of the practical problems that might have limited their participation in the working group.
The preferred prototype was iteratively refined using best practice guidance on Patient Information Leaflets23 and taking into account feedback provided by its potential users, commissioners and service providers. Following local discussion, it was agreed to design only one version of the leaflet for both family caregivers and individuals living with dementia. The leaflet was intended to cover essential information about, and contact details of, the key local organisations that provide dementia-related support and services, which, in practice, often cater for both groups. The intention was to signpost people to such organisations, rather than providing an exhaustive list of services offered. Hence, two versions of the leaflet would not have added substantially different content but could have contributed to information overload, sometimes experienced by those living with dementia and their families (eg, around the time of receiving the diagnosis). In terms of graphics, the design aimed to maximise accessibility, irrespective of the intended user’s group. From the implementation perspective, it was felt that it would be easier to embed one version of the leaflet in the practice of professionals and, equally, designing and using one version of the leaflet could, more likely, contribute to its sustainability in the longer term, for example, by simplifying the updating process.
In its final version, the leaflet presents services clustered around four main themes (‘I want to speak to someone for advice and support’, ‘I want to live safely and with the right support’, ‘I want to know what to do in a crisis’ and ‘Where can I go for more information?’) and it can be printed out on a two-sided A4 using a high contrast colour combination.
Subsequently, the working group agreed a phased implementation process: the Community Mental Health Services for Older People (CMHSOP) within the local mental health trust were identified as the first implementation route because of their key role in diagnosing dementia and in the resulting postdiagnostic care planning. Following diagnosis and initial care planning, those whose needs are stable are discharged to primary care, while those with more complex needs continue to receive secondary services.
The working group preliminarily explored with the CMHSOP service manager, team managers and representatives of the community psychiatric nurses (CPNs) how the leaflet could be used in clinical practice. In January 2019, following positive and supportive feedback, the leaflet working group members and the CMHSOP service manager, team managers and two CPNs took part in a focus group aimed at articulating the theory of change associated with the intervention itself. This exercise helped making explicit the expected process of implementing the intervention in practice and its anticipated impact. Mobilisation activities (eg, information sessions within team meetings and regular reminders emailed to team managers and cascaded to CPNs) were planned and carried out to inform, and subsequently to remind, all CPNs about the leaflet and its use in practice. In June 2019, all the CPNs across the four locality teams covering the study site were invited to start using the leaflet in their clinical practice.
CPNs agreed to hand out the leaflet to patients on their caseload who received a diagnosis of any type of dementia and/or to their family carers. CPNs were expected to use their clinical judgement in deciding to whom and when to offer the leaflet, dependent on the patient’s readiness to receive the leaflet and their individual and family circumstances. They expected to hand it out during their encounter with patients and families, when they felt that the leaflet would support their conversations with them. For these reasons, the use of the leaflet was not embedded at a specific point of the clinical pathway.
A detailed description of the intervention and its implementation strategy are outlined in accordance with the StaRI framework, respectively in tables 1 and 2 (item 9).
Methods and analysis
Evaluation framework
The study adopts an effectiveness–implementation hybrid type 2 design, which allows simultaneous evaluation of the intervention and its implementation strategy,24 as reflected in the evaluation framework (figure 3).
The evaluation framework maps the phases of the study (design–implementation–evaluation) on to the key components of the intervention and shows the relations among them. In the design phase, the intervention and the implementation strategy (ie, the bundle of professional and material interventions intended to sustain the adoption and use of the leaflet among CPNs) were articulated with the theory of change focus group. Following the implementation, the evaluation will:
Assess the unfolding of the implementation strategy and its outcomes.
Assess the outcomes of the intervention as perceived by people with dementia and family carers, the implementers of the intervention, as well as the providers of local services available to people with dementia and their families.
Identify the mechanisms of action, that is, the ‘active ingredients’ that ignited the implementation strategy and the intervention and, hence, contributed to the observed outcomes.
Reflect on the contribution of context (eg, the social, economic, policy and organisational factors surrounding the intervention, including, but not limited to, the co-creation initiative, as described in table 1, item 7) to the design and implementation of the intervention and its implementation strategy and relative outcomes.
Outcomes of the implementation strategy
The implementation strategy is evaluated with respect to both its outcomes and its process dimension, as per the StaRI framework. Following Proctor et al’s taxonomy,25 the following outcomes will be assessed with respect to the implementation strategy (table 1, item 11):
Acceptability of the intervention by CPNs: perception among CPNs that the intervention is agreeable, palatable or satisfactory.
Adoption of the intervention by CPNs: CPNs’ intention, initial decision or action of trying out the intervention.
Appropriateness of the intervention implemented as judged by CPNs: perceived fit, relevance or compatibility of the intervention for the setting of the CMHSOP and perceived fit of the intervention to address the issue.
Cost of the intervention for the implementers.
Feasibility of the implementation of the intervention: the extent to which the intervention can be successfully used or carried out within the setting of the CMHSOP.
Penetration of the intervention: the extent to which the intervention is integrated into the practice of the CMHSOP.
Sustainability of the intervention: extent to which the intervention is maintained or embedded within the CMHSOP.
Outcomes of the intervention
The theory of change exercise run in January 2019 highlighted that the intervention was expected to have an impact on (table 2, item 11):
Its main target population, that is, people with dementia and family carers (direct outcomes).
The implementers of the intervention, that is, CPNs, team and service managers of the CMHSOP delivering the intervention (proximal outcomes).
Local providers of care and support to people with dementia and their families, such as local third sector organisations (distal outcomes).
With respect to people with dementia and family carers, the intervention was expected to improve three outcomes directly associated with the intervention: (1) their satisfaction with the availability of information on local services (eg, in terms of timing and quality of information), (2) their awareness of local services and (3) in the longer term, their access and use of local services.
With respect to the implementers, the intervention was expected to influence two proximal outcomes : (1) CPN, team and service manager’s awareness of local services and support available to people diagnosed with dementia and to their family carers and (2) CPN, team and service manager’s confidence to hold conversations with people living with dementia and their family carers about seeking support and using local services.
With respect to local service providers, the intervention was expected to impact on two distal outcomes: (1) volume of referrals into their services and (2) appropriateness of the use of services among those living with dementia and family carers (eg, with respect to the type and level of need).
Process evaluation
The Medical Research Council (MRC) framework for process evaluation26 is used to investigate (tables 1 and 2, item 12):
The actual implementation strategy, in terms of implementation fidelity (or, conversely, adaptation), reach and dose.
The mechanisms of action of the implementation strategy and of the intervention which brought about change.
The process evaluation will highlight how the implementation process has unfolded in practice and any discrepancies between the actual implementation process and how it was articulated in the theory of change. Moreover, it will help to surface how and why the intervention and its implementation strategy have contributed to achieve these outcomes.
Analytical framework
The framework underpinning the realist evaluation will be adopted as analytical tool. Realist evaluation acknowledges that interventions work (have successful outcomes) in so far as they introduce appropriate ideas and opportunities (mechanisms) to groups of stakeholders in the appropriate social and cultural conditions (context), as condensed in the following formula27 mechanism+context=outcome.
According to the realist framework, mechanisms are the underlying processes or hidden causal levers of intervention activities that make them work28 and are represented by the psychological or social explanations of behaviour of intervention participants and their ideas and reasoning about how change is achieved.29 30 Context refers to the conditions likely to enable or constrain the activation of interventions’ mechanisms (eg, the organisational setting or the broader policy and institutional environment in which an intervention is embedded). Outcome is the anticipated and unanticipated consequence that is brought about by the interaction of different interventions’ mechanisms in different contexts.
The realist framework will be overlaid with Dervin’s sense-making theory, grounded in the user-centred paradigm in information science research, which allows assessment of ‘how people make sense of their worlds and how they use information and other resources in the process’.31 The theory assumes that in the complexity of the real world people constantly face ‘discontinuous’ situations and breaks in an individual’s knowledge exist: this may signal information need and provoke the seeking and use of information to help create new knowledge (‘sense’).19 31 32 The sense-making process is operationalised in terms of four elements: the situation, that is, the time–space context in which the information needs arise and sense is constructed; the gap, that is, a cognitive barrier that could be caused by the lack of relevant information or the uncertainty of a situation; the bridge, that is, the strategy supporting the information seeking process that helps to close the gap; the use, that is, the outcome of the information seeking process in terms of how the information obtained or constructed is used and what sense the information enables users to make.18 19 31
Dervin’s concepts of situation, gap/bridge and use will be adopted to operationalise the realist concept of context, mechanisms and outcomes in the analytical framework.
Data collection methods and samples
The mixed-method evaluation will build on data collected from four sets of stakeholders involved in the intervention (tables 1 and 2, items 13 and 14):
People living with dementia and family carers who are at the immediate receiving end of the intervention.
CPNs, team and service managers based in the CMHSOP who are implementing the intervention.
Local providers of care and support to people with dementia and their families, as the organisations expected to observe a change in their service access and use as a consequence of the intervention.
Local working group, who contributed to the development of the intervention.
Figure 4 provides an overview of the data collection timeline.
People with dementia and family carers
Semistructured in-depth interviews with up to 30 people living with dementia (any type of dementia, diagnosed at any time, willing and able to consent) and family carers (of somebody with any type of dementia, any degree of severity, diagnosed at any time) will take place at two points in time: interview 1 will be carried out 4–6 weeks after the appointment with the CPN when they receive the leaflet and interview 2 (follow-up) will be scheduled about 6 months after interview 1.
The two timepoints have been established to best capture participants’ experiences of receiving (interview 1) and using (follow-up interview) the leaflet (the interview guides are provided as online supplemental file 1). Accordingly, interview 1 will aim to understand (1) if, when and how they received the leaflet; (2) whether and how they anticipate using the leaflet; (3) what they think of the leaflet overall (eg, whether it has changed their perceptions and awareness of the local services offering care and support to those living with dementia and their family carers). Interview 2 will aim to understand (1) whether, how, when and why they used (or not) the leaflet; (2) whether and how the leaflet has changed their care seeking behaviour; (3) their satisfaction with the leaflet.
Supplemental material
Longitudinal interviews with people living with dementia and their family carers are a well-established research method.33–37 The follow-up period is appropriate for research purposes. Over a period of 6 months, personal and family circumstances can change and this may prompt those living with dementia and their family carers to seek help and support. Similarly, those recently diagnosed may not feel immediately ready to get in touch with local services but this could change after 6 months.
This purposeful sample will be recruited at the appointment when the CPN hands out the leaflet. The size of the sample is expected to allow to capture a wide range of perspectives (varying for roles, ie, people with dementia vs family carers, types of dementia diagnosis, levels of cognitive impairment, age, gender and living arrangements) while achieving data saturation. The recruitment will be carried out on a rolling basis alongside the implementation of the intervention and will stop when the desired sample size is reached.
Community Mental Health Services for Older People
In-depth interviews with CPNs, team and service managers from the implementing CMHSOP will be carried out about 6 months from the start of dissemination of the leaflet. Up to 20 staff members will be recruited across the four localities in which the CMHSOP operate.
The interviews aim to understand (1) whether, when and how they used (or not) the leaflet in clinical practice; (2) whether and how the leaflet has changed any aspect of their clinical practice; (3) their views about the impact of adopting the principles of co-creation on the implementation and impact of the intervention. The interview guides are provided as online supplemental file 1.
Local service providers
Referral data from a minimum of three out of the seven local organisations listed on the leaflet which provide care and support to people living with dementia and their family carers will be collected for a period of 24 months (in the 12 months preceding the implementation and for the 12-month implementation period). Semistructured in-depth interviews with service managers will be undertaken about 10 months from the start of the dissemination of the leaflet to (1) review the referral data and (2) identify and interpret the patterns of referral into each service (eg, over time, with respect to the referral source or the user group) and their appropriateness (eg, with respect to the type and level of need). The interview guide is provided as online supplemental file 1.
Local working group
A focus group with the local working group (about 12 participants including commissioners, statutory service providers, staff from CMHSOP, managers from third sector organisations, a family carer) will be held towards the end of the implementation period, following preliminary analysis of the evaluation data (see online supplemental file 1 for the focus group guide). This will aim to:
Discuss and validate the emerging findings about the evaluation of the implementation strategy.
Discuss and validate the emerging findings around the impact of the intervention on people with dementia and their families, CPNs and mental health services and local service providers.
Reflect on the impact of co-creation initiative on the implementation and impact of the intervention.
Informed consent
Written and verbal versions of the Participant Information Sheet and Informed Consent Sheet will be presented to participants. The Participant Information Sheet details the nature of the study, what participation will involve, any risks involved. It will be clearly stated that the participant is free to withdraw at any time, for any reason, without impact on future care or service access, and with no obligation to give the reason for withdrawal. Participants will be allowed as much time as they wish to consider the information (at least 48 hours), and will have the opportunity to ask questions to the investigator, their General Practitioner (GP) and/or CPNs (for people with dementia and family carers), or other independent parties, in order to decide whether they will participate in the study. Written informed consent will be obtained.
Maximum efforts will be devoted to recruit people living with dementia into this study. Historically, people with dementia have been excluded from much research in social and healthcare on the basis that their declining cognitive abilities may affect capacity to consent.38 However, the ability of many people with dementia to provide informed consent, based on their ability to understand information, evaluate risks and benefits, and communicate decisions effectively, has been established.39 40 Prior to the interview, an informal conversation about the research will allow the researchers to ascertain that the potential participant has capacity to consent. If the researcher believes that the person with dementia has capacity, she will take written informed consent and will start the interview.
Data analysis
The referral data provided by each local provider will be collated, tabulated and described with simple metrics (eg, number of referrals in total, by user group, by referral source, by reason for referral). The results will be triangulated with interview data collected from the participants living with dementia and family carers, the implementers and the managers of the local service providers.
The interviews will be transcribed, anonymised and analysed using thematic analysis (performed with NVivo12) (tables 1 and 2, item 15).41 Following the realist approach, the initial codebook will be developed around the themes (codes) of context conditions, underlying mechanisms, observed outcomes matched on to implementation strategy and intervention. These will be then refined into subcodes to capture specificities of data collected from different interviewee roles or to highlight different dimensions featuring in a main code. For example, for intervention strategy, the subcodes of ‘fidelity’, ‘dose’, ‘adaptations’, ‘reach’ may be developed or ‘mechanisms’ may be clustered around the subcode ‘ideas’.
Patient and public involvement
A person living with dementia and a family carer of somebody living with dementia are members of the advisory board of the co-creation initiative. In this capacity they are overseeing the development of the study, provided feedback in the design phase of the intervention, reviewed the interview guides and are expected to review the results.
Ethics and dissemination
Ethical and research governance approvals have been obtained from the West Midlands—South Birmingham Research Ethics Committee (REC reference 16/WM/0397).
The study has been designed as part of a 5-year co-creation initiative by the local working group in collaboration with the research team. Updates about the implementation of the intervention are regularly shared with the working group and with the wider group of stakeholders involved in the co-creation initiative.
Findings of the work will be disseminated across professionals, for example, at national conferences and events for providers and commissioners of care and support for people with dementia, policy-makers in the area of dementia, representatives of services users. Articles stemming from this work will be submitted for publication to peer-reviewed journals and to national and international scientific conferences in the field of applied health research.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors CDP was involved in the design of the intervention and of the implementation strategy, conceived the evaluation framework, drafted the first version of the protocol and revised it following discussions with the other authors. JRO and MA oversaw the design of the intervention and of the implementation strategy, contributed to developing the evaluation framework and revised the manuscript. CB was involved in the design of the intervention and of the implementation strategy, supported local engagement and revised the manuscript. RG was involved in the design of the intervention and of the implementation strategy, supported local engagement and revised the manuscript. All authors have read and approved the final manuscript.
Funding CDP, JRO, RG and MA were supported by The Health Foundation, grant number 1274233.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods and analysis section for further details.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.