Systems for early detection of clinical deterioration in older people in non-hospital settings – a systematic scoping review

Older patients with underlying health conditions constitute a vulnerable group that is particularly prone to clinical deterioration (1). They often display non-specific symptoms of their condition, and these can stem from a completely different organ system to the one where their acute illness is located (2).

In addition, acute illness in older people is often characterised by loss of function, which can be misinterpreted as a care problem as opposed to a health issue (1). Older people can quickly become acutely ill, and observation skills is needed to identify clinical deterioration at an early stage (2). Lack of such competence can lead to conditions and diseases not being diagnosed, which constitutes a serious risk in terms of patient safety. Acute functional decline in older people can go undetected, and signs of serious illness can, at worst, be overlooked (1).

A high level of observation skills is therefore a particularly important part of patient safety when caring for older people. This is in line with the primary care service’s responsibility for providing services that are safe, and which are preferably founded on evidence-based guidelines (3–5).

Many primary care staff are unskilled, and registered nurses (RNs) often have sole responsibility for patients (6). Being able to observe and critically assess a patient’s condition is a vital skill for RNs (7). They must also know how to react and what to do when a patient’s condition deteriorates (8).

The uncertain situation surrounding clinical observations and assessments is confirmed in the review by Preston and Flynn (8). A review of basic observation skills in primary care in Norway by Bing-Jonsson et al. (9) found that competence among primary care staff varies in relation to observation, basic assessments and measuring vital signs.

We know that one-third of hospital deaths could be prevented as they are associated with a lack of clinical monitoring of patients (10). About half of all patient injuries occur in primary care, where older people constitute the largest patient group (4).

Older patients are a heterogenous group, ranging from people living at home with minor health challenges to frail nursing home residents with comorbidities and functional impairments (11).

In Norway, the National Early Warning Score (NEWS/NEWS2) (Figure 1) and Modified Early Warning Score (MEWS) are perhaps the most well-known and most-used tools for early detection of clinical deterioration. The generic term for such systems is early warning score (EWS).

The EWS system was developed to standardise and quality assure the assessment of and response to clinical deterioration (12). Older patients often have atypical symptoms, and they need to be monitored for changes in vital signs in order to identify clinical deterioration at an early stage (2).

All of these scoring systems are based on observations of vital signs such as systolic blood pressure, pulse rate, respiration rate and temperature, which are then recorded on a standardised clinical chart.

Different scores trigger different responses, which serves as further decision support for healthcare personnel (13). Use of an EWS to support clinical assessments is a recognised method in Norway and internationally, and is recommended in guidelines and clinical guidance (14–16).

Earlier research into the use of the scoring system in non-hospital settings mainly relates to patient outcomes in ambulances. Research on older people in other settings, such as nursing homes, is less common (13, 14, 17, 18). It would be useful to form a picture of which aspects of this topic are illuminated by research and which are not.

Objective of the study

The objective of this scoping review is to identify, present and describe research on tools for early detection of clinical deterioration in older people in non-hospital settings.

Method

The protocol for this scoping review was published on the Open Science Framework platform in August 2020 (19). We conducted the study in line with Joanna Briggs Institute's methodology for scoping reviews (20), which in turn is based on Arksey and O'Malley’s (21) widely used methodological framework.

The methodology is suitable for obtaining a profile of existing research literature on a given topic and identifying any knowledge gaps. Through nine explicit methodological steps, we found and presented an overview of research on the use of tools for early detection of clinical deterioration in older people in non-hospital settings.

Inclusion criteria

We used the Patient Context Concept (PCC) framework to specify the population, context and concept of the scoping review (20). These elements are reflected in the inclusion criteria that form the basis for a systematic literature search.

We included studies where the population (P) was over the age of 65. We also included studies in which this age group was part of a larger population, and here we tried to retrieve the data that related to the over 65s.

Furthermore, we included studies if they were conducted in non-hospital contexts (C), such as nursing homes, home-based nursing care, GP practices and ambulances. Studies that covered both hospital and non-hospital settings were included if they involved measurements of vital signs in the latter setting.

In addition to these, we included qualitative studies in which healthcare personnel were defined as the study population if they worked with older patients in, for example, nursing homes or home-based nursing care.

The concept (C) in this scoping review is tools for early detection of clinical deterioration. We did not delimit the range of systems, but they had to be used for identifying clinical deterioration based on vital signs. We included all types of primary studies regardless of study design.

Literature search

We performed a three-stage iterative literature search (20) between 1 August and 30 October 2020 and an updated search on 27 May 2021. The first of the three stages entailed an initial search in two databases and subsequent analyses of titles and summaries with a view to identifying additional relevant search terms.

We then conducted the main search in the following databases: Medline Ovid, Cinahl Ebsco, Cochrane Library Central, Epistemonikos and Embase Ovid. In line with the PCC scoping review, we prepared a separate search strategy for each database.

We did not delimit the year of publication or language. All searches were quality assured by a university librarian. In addition, we conducted a manual search, where we went through all reference lists of selected studies as well as those of relevant systematic reviews.

Furthermore, we searched for ongoing studies in Prospero, ClinicalTrials.gov, the WHO International Clinical Trials Registry Platform, the EU Clinical Trials Register and the Open Science Framework (OSF). In order to identify relevant literature that was not found in the databases, we also searched Google Scholar and OpenGrey. The full search strategy is presented in Appendix 1 (partly in Norwegian). 

Selecting studies

Two people screened the entire result of the literature search independently (EGS, BG and TP). We initially assessed the title and abstract in relation to the inclusion criteria.

In cases where it was clear that an article met the inclusion criteria, or if it was unclear, we obtained the full text versions, and two people would then assess them for inclusion independently. Disagreements about inclusion were resolved through discussion or by consulting a third person (DC).

Data collection and analysis

We developed a data collection form based on the inclusion criteria and the following variables: reference, country of origin, purpose, study design, study population, clinical population, age of clinical population, EWS system used, and in what situation, any patient outcomes, how validation studies were carried out, and topic of qualitative studies.

The data collection form was piloted on five articles by two people independently. Data from the remainder of the selected studies were then extracted and collected by one person (EGS) and controlled by another (BG or TP).

We used simple descriptive statistical analyses (frequency, proportion and percentage) to describe the selected studies in terms of context, study design and study population. We used a graph to show which EWS system was used and how often.

We described the results of the qualitative studies in a simple content analysis of the themes we identified in the studies.

Ethics

As this is a scoping review, it does not require approval by an ethics committee or data protection officer.

Results

In total, we identified 15 853 unique references in the literature search. We ended up including 45 articles after 84 articles were read in full text (Figure 2). One of these was a protocol for ongoing studies (22).

Many of the selected studies originated from European countries (n = 39), mostly the UK (n = 16). Five of the studies were conducted in Asia and one in North America. A total of 89% of the studies were carried out in 2015 or later.

Furthermore, a large proportion of the selected studies had different quantitative designs (n = 34), all were observational studies and none were experimental. The qualitative studies (n = 6) were based on either focus group interviews or individual interviews. The characteristics of selected studies are presented in Table 1.

Only a small number of studies exclusively included older people (n = 6) (23–28). Nevertheless, the mean age (and/or median age) was over 65 years in all the studies selected. A few had a median or mean age of over 80 years (n = 4) (26–29). Some studies focused exclusively on sepsis (27, 30–35), but most studies were not limited to a specific diagnosis.

The largest proportion of studies were conducted in the ambulance service (n = 25). A smaller proportion (n = 6) was carried out in nursing homes (23, 25, 26, 28, 36), at GP practices (n = 1) (37) and municipal acute bed units (38), and two were from home-based nursing care (n = 2) (29, 39).

Three of the studies took place across different settings (32, 40, 41). The study populations in the qualitative studies and implementation studies were made up of healthcare personnel.

We identified 14 different EWS systems

We identified a total of 14 different EWS systems, and more than half of the articles related to NEWS or NEWS2 (Figure 3 and Table 2). Only a few of the studies examined the actual use of the systems (n = 5). In most cases, data were collected retrospectively (24, 25, 27, 30, 31, 33-35, 42–62).

We found that early warning scoring was largely based on clinical indication, i.e. when acute illness was suspected. We only identified one validation study, which examined reliability and validity for the Pre-hospital Medical Emergencies Early Warning Scale (Pre-MEWS) (63).

The most common patient outcome in the quantitative studies was mortality (n = 23). Admission to hospital or an ICU (n = 15) and further deterioration and complications (n = 8) were also widely investigated outcomes. A few studies examined whether the use of an EWS affected attendance by a doctor (n = 2) and re-admission (n = 2).

The qualitative studies focused on experiences with EWS as decision support and a communication tool. Topics such as reassurance and support in clinical assessments were reported most often (n = 7). Other topics that were highlighted in qualitative research were implementation, training and challenges relating to the correct use of EWS (29, 32).

In the tables, the selected studies are sorted into quantitative studies ( Appendix 2. Quantitative studies included) (24–28, 30, 31, 33–35, 38, 42–62, 64–66), qualitative studies (Appendix 3. Qualitative studies included) (23, 28, 36, 39–41, 67), implementation and validation studies (Appendix 4. Implementation and validation studies and protocols included) (29, 32, 63), and protocols (22) respectively.

We also produced an overview of articles that we read in full text and then excluded, and state the reason for the exclusion (Appendix 5. Studies excluded, with reason for exclusion).

Discussion

In this scoping review, we identified and described research-based knowledge on the use of tools for early detection of clinical deterioration in older people in non-hospital settings. We identified 14 different EWS systems, all with small variations. MEWS, NEWS and NEWS2 were the most common.

Research in the field mainly consists of quantitative observational studies as well as some qualitative studies and a few implementation studies. There is a clear trend for more research on EWS systems in recent years, but in relation to older people in non-hospital settings this research is still limited. Few studies have been conducted in nursing homes and in home-based nursing care.

Lack of research on EWS and older people

There is surprisingly little research on the use of EWS in settings where RNs meet vulnerable older people, such as nursing homes and home-based nursing care.

Since 2018, the Norwegian Directorate of Health has recommended using NEWS2 in the primary health and care services, as a component in the patient safety programme (5), and recently the package of measures in the Safe Hands programme (I trygge hender 24/7) was continued in the form of national clinical guidance (15).

This means that national health authorities are making recommendations using a seemingly poor evidence base. Nevertheless, the growing trend for studies reflects the strong focus on generating knowledge about these tools, which in turn may be a sign of increased use and a major need for such systems.

The alternative to such tools is discretionary assessments, a poor decision basis and perhaps imprecise communication about the patient’s condition. Given that the observational competence of healthcare personnel may be inadequate (9), it is probably safer to use such a tool – provided that it is used correctly (14).

In addition, the evidence base for using EWS in the specialist health service is clearer (13, 14, 68, 69), and it may be possible to use this indirectly in non-hospital settings. For example, several studies from the specialist health service found that using EWS has an impact on patient outcomes, including mortality (14, 68, 69).

A discernible knowledge gap persists

This scoping review shows that the evidence base for recommending use of EWS in non-hospital settings is weak. We have identified a discernible knowledge gap that should be prioritised. The need for research has been pointed out in earlier systematic reviews (17, 18), but our review shows that the knowledge gap still remains.

This scoping review shows that the evidence base for recommending use of EWS in non-hospital settings is weak.

In particular, we lack studies with robust prospective designs to assess the effects of using EWS in pre-hospital settings (17, 18), which is a prerequisite for an evidence-based approach (70). Nevertheless, the number of studies conducted in non-hospital settings has generally increased, which may reflect the interest in and need for such tools.

We identified as many as 14 different EWS systems. The question is therefore which of these is best suited for use on older people in non-hospital settings. The national authorities in Norway and the Royal College of Physicians in the UK recommend use of NEWS2 in their guidelines and clinical guidance (12, 15).

The studies do not compare different EWS systems

The selected studies offer little comparison of the various EWS systems. Consequently, they do not provide any enlightenment as to which tool is ‘the best’, and this is also beyond the remit of a scoping overview.

More tools were identified than in earlier systematic reviews (17, 18). Downey and Tahir (14) recommend that only one tool is used in order to create a universal language across the different contexts.

However, it is also important to consider whether the same tools can actually be used on frail older people with complex clinical pictures and comorbidities in non-hospital settings and on young adults in hospitals or in other contexts.

EWS systems have been criticised for their use in settings and for populations other than those for which they were developed (71). This is not covered to any extent in the research literature.

EWS cannot replace clinical assessments by healthcare personnel

The qualitative studies and implementation studies show that healthcare personnel have expressed a need for decision support in situations where EWS is intended to serve as an aid, and for such systems to be useful and relevant. Nevertheless, it appears that EWS can never replace clinical assessments performed by healthcare personnel (23, 28, 36, 39, 67).

One of the objections to EWS is precisely because it can be regarded as a substitute for a comprehensive clinical assessment as opposed to a supporting tool (71). This is confirmed by Jeppestøl and Kirkevold (39), who highlight the importance of knowing the patients when using the instruments.

Healthcare personnel have expressed a need for decision support in situations where EWS is intended to serve as an aid.

RNs are an important resource, and their competence should be used properly and developed where necessary. Observation skills is crucial for assessing health challenges in older people with complex clinical pictures (2), and the need for basic observation skills was identified by Bing-Jonsson et al. (9).

The tools can provide good support, but require correct use and the right skills. It is therefore important not to underestimate the importance of the implementation of such tools (15) as part of a systematic, planned process, in which there should also be a focus on validity over time (72).

The scoping review seems to document a major need for decision support, while the literature shows challenges in relation to competence, training and implementation of EWS systems.

Implications

The qualitative studies in this scoping review highlight the benefits of using EWS, and show that it has a particularly positive impact on decision support and communication. However, targeted training is crucial for personnel being able to use the tools properly. This is often the responsibility of the RNs in primary health and care services.

A number of knowledge gaps exist in relation to the use of EWS in nursing homes, in home-based nursing care and for older people in general. One of the areas where there is reason to update and refine systematic reviews is in the ambulance service.

A number of knowledge gaps exist in relation to the use of EWS in nursing homes, in home-based nursing care and for older people in general.

Furthermore, there is a marked need for prospective cohort studies that can evaluate the effect of EWS systems aimed at older patients in nursing homes and home-based nursing care. It would also be useful to know which of the 14 systems is best suited for early detection of clinical deterioration in older people in non-hospital settings.

Strengths and limitations of the methodology

Systematic scoping reviews are intended to identify the research that exists in a specific field, in this case tools for early detection of clinical deterioration in older people in non-hospital settings. The methodology is the greatest strength of this review, namely that it provides an overview of the research that exists in the field.

Nevertheless, it is important to point out that it is beyond the remit of a scoping review to assess the quality of the methodologies in the selected studies, including the risk of systematic biases. This must be taken into account when interpreting and using the results from this scoping review.

One of the strengths of the scoping review is the protocol we published beforehand, which reduces the risk of selection bias in the study. Another strength is the broad and iterative literature search that formed the basis of the data collection, which was also quality assured by a university librarian.

It should nevertheless be noted that there is always a certain risk that some studies will not be picked up in the search. However, the fact that two people selected the studies independently of each other is a strength of the review.

One weakness of the method was that that two people independently extracted data from the articles in only 15 of the 45 studies selected. However, we piloted the process in order to extract data in advance, since we found good conformity when two people went through 15 articles.

Conclusion

This scoping review shows that research activity in tools for early detection of clinical deterioration is experiencing rapid growth both in Norway and internationally. However, this trend is not seen in relation to use in typical primary care settings such as home-based nursing care and nursing homes.

Little is known about the contribution of these tools in such settings, which is a paradox considering that they are recommended in national clinical guidance. Most studies on EWS for older people are from the ambulance service.

The general increase in research on EWS is probably due to several factors, the most important of which is perhaps the real, significant need for standardised tools to aid decision-making in complex assessments.

Future studies on the use of EWS for older people in non-hospital settings should have a more robust, prospective and controlled design in order to generate knowledge on the benefits for the patient group and healthcare personnel in primary care.

The study is part of the IMPAKT project, which is funded by the Research Council of Norway (256569) (73).

The study was conducted for a master’s thesis in the master's degree in evidence-based practice in health sciences at the Western Norway University of Applied Sciences. I would like to thank the university's librarian, Gøril Tvedten Jorem, for her invaluable support in the design and peer review of the literature search.

References

1.       Wyller TB. Geriatri. En medisinsk lærebok. 3rd ed. Oslo: Gyldendal; 2020.

2.       Nickel C, Bellou A, Conroy S. Geriatric emergency medicine. Cham: Springer International Publishing; 2017.

3.       Forskrift 28.10.2016 nr. 1250 om ledelse og kvalitetsforbedring i helse- og omsorgstjenesten. Available at: https://lovdata.no/dokument/LTI/forskrift/2016-10-28-1250 (downloaded 03.02.2022).

4.       Helsedirektoratet. Nasjonal handlingsplan for pasientsikkerhet og kvalitetsforbedring 2019–2023. Oslo: Helsedirektoratet; 2021. Available at: https://www.helsedirektoratet.no/tema/pasientsikkerhet-og-kvalitetsforbedring (downloaded 03.02.2022).

5.       Helsedirektoratet. Pasientsikkerhetsprogrammet i trygge hender 24/7. Oslo: Helsedirektoratet; 2019. Available at: https://www.itryggehender24-7.no/om-i-trygge-hender-24-7 (downloaded 03.03.2021).

6.       Flodgren G, Bidonde J, Berg RC. Konsekvenser av en høy andel ufaglærte på kvalitet og pasientsikkerhet i helse‐ og omsorgstjenestene: en systematisk oversikt. Oslo: Folkehelseinstituttet; 2017.

7.       Stanyon MR, Goldberg SE, Astle A, Griffiths A, Gordon AL. The competencies of registered nurses working in care homes: a modified Delphi study. Age Ageing. 2017;46(4):582–8. DOI: 10.1093/ageing/afw2244

8.       Preston RM, Flynn DJ. Observations in acute care: evidence-based approach to patient safety. Br J Nurs. 2010;19(7):442–7. DOI: 10.12968/bjon.2010.19.7.47446

9.       Bing-Jonsson PC, Hofoss D, Kirkevold M, Bjørk IT, Foss C. Sufficient competence in community elderly care? Results from a competence measurement of nursing staff. BMC Nurs. 2016 Jan 14;15(1):5. DOI: 10.1186/s12912-016-0124-z

10.     Hogan H, Healey F, Neale G, Thomson R, Vincent C, Black N. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. London: BMJ Publishing Group Ltd; 2012.

11.     Helsedirektoratet. Kommunale helse- og omsorgstjenester 2019. Statistikk fra Kommunalt pasient- og brukerregister (IPLOS). Oslo: Helsedirektoratet; 2019.

12.     Royal College of Physicians. National Early Warning Score (NEWS) 2. Standardising the assessment of acute-illness severity in the NHS. London: Royal College of Physicians; 2017.

13.     Le Lagadec MD, Dwyer T. Scoping review: the use of early warning systems for the identification of in-hospital patients at risk of deterioration. Australian Critical Care. 2017;30(4):211–8. DOI: 10.1016/j.aucc.2016.10.003

14.     Downey CL, Tahir W, Randell R, Brown JM, Jayne DG. Strengths and limitations of early warning scores: a systematic review and narrative synthesis. Int J Nurs Stud. 2017;76:106–19. DOI: 10.1016/j.ijnurstu.2017.09.003

15.     Helsedirektoratet. Tidlig oppdagelse og rask respons ved forverret somatisk tilstand. Oslo: Helsedirektoratet; 2020. Available at: https://www.helsedirektoratet.no/faglige-rad/tidlig-oppdagelse-og-rask-respons-ved-forverret-somatisk-tilstand#referere (downloaded 01.06.2021).

16.     National Institute for Health and Care Excellence (NICE). Sepsis: recognition, diagnosis and early management. NICE guideline [NG51]. London: NICE; 2016 [oppdatert 13.09.2017]. Available at: https://www.nice.org.uk/guidance/NG51 (downloaded 24.04.2021).

17.     Williams TA, Tohira H, Finn J, Perkins GD, Ho KM. The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: a systematic review. Resuscitation. 2016;102:35–43. DOI: 10.1016/j.resuscitation.2016.02.011

18.     Patel R, Nugawela MD, Edwards HB, Richards A, Le Roux H, Pullyblank A, et al. Can early warning scores identify deteriorating patients in pre-hospital settings? A systematic review. Resuscitation. 2018;132:101–11. DOI: 10.1016/j.resuscitation.2018.08.028

19.     Steinseide E, Graverholt B, Espehaug B, Ciliska D. The use of early warning scores (EWS) in older people in nonhospitalized settings. Protocol for a scoping review. Charlottesville: Center for Open Science; 2020. Available at: https://osf.io/5rg78/ (downloaded 17.08.2020).

20.     Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews [updated 02.07.2020; cited 10.02.2021]. In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. Available at: https://jbi-global-wiki.refined.site/space/MANUAL/3283910770/Chapter+11%3A+Scoping+reviews 

21.     Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19–32. DOI: 10.1080/1364557032000119616

22.     Graverholt B. Tailoring an intervention to the educational needs of healthcare professionals in nursing homes: does it have an impact on the use of an early warning score? Bergen: ISRCTN registry; 2020. Available at: https://www.isrctn.com/ISRCTN12437773 (downloaded 03.02.2022).

23.     Steinsheim G, Malmedal W. Tidlig oppdagelse av forverret tilstand. Tidsskrift for omsorgsforskning. 2020;6(02):10–22. DOI: 10.18261/issn.2387-5984-2020-02-02

24.     Mitsunaga T, Hasegawa I, Uzura M, Okuno K, Otani K, Ohtaki Y, et al. Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department. Peerj. 2019;7:e6947. DOI: 10.7717/peerj.6947

25.     Stow D, Barker R, Matthews F, Hanratty B. National Early Warning Scores and covid-19 deaths in care homes: an ecological time series study. Innovation in Aging. 2020;4(Suppl 1):962. DOI: 10.1136/bmjopen-2020-045579

26.     Barker RO, Stocker R, Russell S, Roberts A, Kingston A, Adamson J, et al. Distribution of the National Early Warning Score (NEWS) in care home residents. Age Ageing. 2019;49(1):141–5. DOI: 10.1093/ageing/afz130

27.     Camm CF, Hayward G, Elias TCN, Bowen JST, Hassanzadeh R, Fanshawe T, et al. Sepsis recognition tools in acute ambulatory care: associations with process of care and clinical outcomes in a service evaluation of an Emergency Multidisciplinary Unit in Oxfordshire. BMJ Open. 2018;8(4):e020497. DOI: 10.1136/bmjopen-2017-020497

28.     Hodgson P, Cook G, Thompson J, Abbott-Brailey H. Assessment and clinical decision making of the acutely ill older care home resident: implementation of NEWS in Gateshead care homes. Final report. Newcastle: Northumbria University; 2017.

29.     Ammitzboll O, Maarslet L. Early Warning Score in primary care in Denmark. Ugeskrift for laeger. 2014;176(41).

30.     Bayer O, Schwarzkopf D, Stumme C, Stacke A, Hartog CS, Hohenstein C, et al. An early warning scoring system to identify septic patients in the prehospital setting: the PRESEP score. Academic Emergency Medicine. 2015;22(7):868–71. DOI: 10.1111/acem.12707

31.     Jouffroy R, Saade A, Ellouze S, Carpentier A, Michaloux M, Carli P, et al. Prehospital triage of septic patients at the SAMU regulation: comparison of qSOFA, MRST, MEWS and PRESEP scores. Am J Emerg Med. 2018;36(5):820–4. DOI: 10.1016/j.ajem.2017.10.030

32.     Pullyblank A, Tavare A, Little H, Redfern E, le Roux H, Inada-Kim M, et al. Implementation of the national early warning score in patients with suspicion of sepsis: evaluation of a system-wide quality improvement project. British Journal of General Practice. 2020;70(695):E381–8. DOI: 10.3399/bjgp20X709349

33.     Lane DJ, Wunsch H, Saskin R, Cheskes S, Lin S, Morrison LJ, et al. Screening strategies to identify sepsis in the prehospital setting: a validation study. Cmaj. 2020;192(10):E230–9. DOI: 10.1503/cmaj.190966

34.     Usul E, Korkut S, Kayipmaz AE, Halici A, Kavalci C. The role of the quick sequential organ failure assessment score (qSOFA) and modified early warning score (MEWS) in the pre-hospitalization prediction of sepsis prognosis. Am J Emerg Med. 2021;41:158–62. DOI: 10.1016/j.ajem.2020.09.049

35.     Hargreaves DS, de Carvalho JLJ, Smith L, Picton G, Venn R, Hodgson LE. Persistently elevated early warning scores and lactate identifies patients at high risk of mortality in suspected sepsis. Eur J Emerg Med. 2020;27(2):125–31. DOI: 10.1097/MEJ.0000000000000630 

36.     Russell S, Stocker R, Barker RO, Liddle J, Adamson J, Hanratty B. Implementation of the National Early Warning Score in UK care homes: a qualitative evaluation. The British Journal of General Practice. 2020 okt. 29;70(700);e793–e800. DOI: 10.3399/bjgp20X713069

37.     Scott LJ, Redmond NM, Garrett J, Whiting P, Northstone K, Pullyblank A. Distributions of the National Early Warning Score (NEWS) across a healthcare system following a large-scale roll-out. Emergency Medicine Journal. 2019;36(5):287–92. DOI: 10.1136/emermed-2018-208140

38.     Hernes SK, Baste V, Krokmyrdal KA, Todnem SL, Ruths S, Johansen IH. Associations between characteristics of the patients at municipal acute bed unit admission and further transfer to hospital: a prospective observational study. BMC Health Services Research. 2020;20(1):963. DOI: 10.1186/s12913-020-05823-0

39.     Jeppestøl K, Kirkevold M, Bragstad LK. Applying the Modified Early Warning Score (MEWS) to assess geriatric patients in home care settings: a qualitative study of nurses’ and general practitioners’ experiences. 2019 okt. DOI: 10.21203/rs.2.16666/v2

40.     Brangan E, Banks J, Brant H, Pullyblank A, Le Roux H, Redwood S. Using the National Early Warning Score (NEWS) outside acute hospital settings: a qualitative study of staff experiences in the West of England. BMJ Open. 2018;8(10):e022528. DOI: 10.1136/bmjopen-2018-022528

41.     Sletner A, Halvorsrud L. Opplæring i systematisk observasjon (ALERT) i kommunehelsetjenesten og betydningen for jobbtilfredshet. Sykepleien Forskning. 2020;15(80410):e-80410. DOI: 10.4220/Sykepleienf.2020.80410

42.     Challen K, Walter D. Physiological scoring: an aid to emergency medical services transport decisions? Prehospital and Disaster Medicine. 2010;25(4):320–3. DOI: 10.1017/s1049023x00008268

43.     Fullerton JN, Price CL, Silvey NE, Brace SJ, Perkins GD. Is the Modified Early Warning Score (MEWS) superior to clinician judgement in detecting critical illness in the pre-hospital environment? Resuscitation. 2012;83(5):557–62. DOI: 10.1016/j.resuscitation.2012.01.004

44.     Gray JT, Challen K, Oughton L. Does the pandemic medical early warning score system correlate with disposition decisions made at patient contact by emergency care practitioners? Emerg Med J. 2010 des.;27(12):943–7. DOI: 10.1136/emj.2009.072959

45.     Hoikka M, Lankimaki S, Silfvast T, Ala-Kokko TI. Medical priority dispatch codes-comparison with National Early Warning Score. Scand J Trauma Resusc Emerg Med. 2016;24(1):142. DOI: 10.1186/s13049-016-0336-y

46.     Hoikka M, Silfvast T, Ala-Kokko TI. Does the prehospital National Early Warning Score predict the short-term mortality of unselected emergency patients? Scand J Trauma Resusc Emerg Med. 2018;26(1):48. DOI: 10.1186/s13049-018-0514-1

47.     Kitahara O, Nishiyama K, Yamamoto B, Inoue S, Inokuchi S. The prehospital quick SOFA score is associated with in-hospital mortality in noninfected patients: a retrospective, cross-sectional study. PLoS ONE. 2018;13(8):e0202111. DOI: 10.1371/journal.pone.0202111

48.     Leung SC, Leung LP, Fan KL, Yip WL. Can prehospital Modified Early Warning Score identify non-trauma patients requiring life-saving intervention in the emergency department? Emergency Medicine Australasia. 2016;28(1):84–9. DOI: 10.1111/1742-6723.12501

49.     Magnusson C, Herlitz J, Axelsson C. Pre-hospital triage performance and emergency medical services nurse's field assessment in an unselected patient population attended to by the emergency medical services: a prospective observational study. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2020;28(1):81. DOI: 10.1186/s13049-020-00766-1

50.     Martín-Rodrígues F, López-Izqierdo R, Vegas CDP, Delgado-Benito JF, Perez CDP, Rodriguez VC, et al. A multicenter observational prospective cohort study of association of the prehospital national early warning score 2 and hospital triage with early mortality. Emerg Med Int. 2019 jul. 1;5147808. DOI: 10.1155/2019/5147808

51.     Martín-Rodríguez F, López-Izquierdo R, Vegas CDP, Sanchez-Soberon I, Delgado-Benito JF, Martin-Conty JL, et al. Can the prehospital National Early Warning Score 2 identify patients at risk of in-hospital early mortality? A prospective, multicenter cohort study. Heart and Lung. 2020;49(5):585–91. DOI: 10.1016/j.hrtlng.2020.02.047

52.     Martín-Rodríguez F, López-Izquierdo R, Vegas CDP, Delgado-Benito JF, Ibanez PDB, Mangas IM, et al. Predictive value of the prehospital NEWS2-L – National Early Warning Score 2 Lactate – for detecting early death after an emergency. Emergencias. 2019 jun.;31(3):173–9.

53.     Martín-Rodríguez F, Vegas CDP, Mohedano-Moriano A, Polonio-Lopez B, Miquel CM, Vinuela A, et al. Role of biomarkers in the prediction of serious adverse events after syncope in prehospital assessment: a multi-center observational study. J Clin Med. 2020;9(3):651. DOI: 10.3390/jcm9030651

54.     Martín-Rodríguez F, Lopez-Izquierdo R, Benito JFD, Sanz-Garcia A, Vegas CP, Villamor MAC, et al. Prehospital point-of-care lactate increases the prognostic accuracy of national early warning score 2 for early risk stratification of mortality: results of a multicenter, observational study. J Clin Med. 2020;9(4):1156. DOI: 10.3390/jcm9041156

55.     Pirneskoski J, Kuisma M, Olkkola KT, Nurmi J. Prehospital National Early Warning Score predicts early mortality. Acta Anaesthesiol Scand. 2019;63(5):676–83. DOI: 10.1111/aas.13310

56.     Silcock DJ, Corfield AR, Gowens PA, Rooney KD. Validation of the National Early Warning Score in the prehospital setting. Resuscitation. 2015;89:31–5. DOI: 10.1016/j.resuscitation.2014.12.029

57.     Spangler D, Hermansson T, Smekal D, Blomberg H. A validation of machine learning-based risk scores in the prehospital setting. PLoS ONE. 2019;14(12):e0226518. DOI: 10.1371/journal.pone.0226518

58.     Vihonen H, Laaperi M, Kuisma M, Pirneskoski J, Nurmi J. Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study. Emergency Medicine Journal. 2020;37(5):286–92. DOI: 10.1136/emermed-2018-208309

59.     Pirneskoski J, Laaperi M, Kuisma M, Olkkola KT, Nurmi J. Ability of prehospital NEWS to predict 1-day and 7-day mortality is reduced in the older adult patients. Emergency Medicine Journal. 2021;38(12):913–8. DOI: 10.1136/emermed-2019-209400

60.     Martin-Rodriguez F, Sanz-Garcia A, Medina-Lozano E, Villamor MAC, Rodriguez VC, Vegas CDP, et al. The value of prehospital early warning scores to predict in – hospital clinical deterioration: a multicenter, observational base – ambulance study. Prehosp Emerg Care. 2020:25(5):597–606. DOI: 10.1080/10903127.2020.1813224

61.     Ivic R, Kurland L, Vicente V, Castren M, Bohm K. Serious conditions among patients with nonspecific chief complaints in the pre-hospital setting. A retrospective cohort study. BMJ Open. 2019;9(Suppl 2):A2.2-A2. DOI: 10.1136/bmjopen-2019-EMS.5

62.     Dillon K, Hook C, Coupland Z, Avery P, Taylor H, Lockyer A. Pre-hospital lowest recorded oxygen saturation independently predicts death in patients with COVID-19. British Paramedic Journal. 2020;5(3):59–65. DOI: 10.29045/14784726.2020.09.5.3.59

63.     Ebrahimian A, Masoumi G, Jamshidi-Orak R, Seyedin H. Development and psychometric evaluation of the pre-hospital medical emergencies early warning scale. Indian Journal of Critical Care Medicine. 2017;21(4):205–12. DOI: 10.4103/ijccm.IJCCM_49_17

64.     Essam N, Windle K, Mullineaux D, Knowles S, Gray J, Siriwardena AN. Modified Early Warning Scores (MEWS) to support ambulance clinicians’ decisions to transport or treat at home. Lincoln: University of Lincoln; 2014. Available at: http://eprints.lincoln.ac.uk/id/eprint/13448/ (downloaded 03.02.2022).

65.     Martin-Rodriguez F, Castro-Villamor MA, Vegas CDP, Martin-Conty JL, Mayo-Iscar A, Benito JFD, et al. Analysis of the early warning score to detect critical or high-risk patients in the prehospital setting. Internal & Emergency Medicine. 2019;14(4):581–9. DOI: 10.1007/s11739-019-02026-2

66.     Scott LJ, Redmond NM, Tavaré A, Little H, Srivastava S, Pullyblank A. Association between National Early Warning Scores in primary care and clinical outcomes: an observational study in UK primary and secondary care. British Journal of General Practice. 2020;70(695):e374–e80. DOI: 10.3399/bjgp20X709337

67.     McClelland G, Haworth D. A qualitative investigation into paramedics' thoughts about the introduction of the National Early Warning Score. British Paramedic Journal. 2016;1(1):9–14. DOI: 10.1136/emermed-2016-206139.10

68.     Alam N, Hobbelink EL, van Tienhoven A-J, van de Ven PM, Jansma EP, Nanayakkara PW. The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review. Resuscitation. 2014;85(5):587–94. DOI: 10.1016/j.resuscitation.2014.01.013

69.     McGaughey J, O'Halloran P, Porter S, Blackwood B. Early warning systems and rapid response to the deteriorating patient in hospital: a systematic realist review. J Adv Nurs. 2017;73(12):2877-91. DOI: 10.1111/jan.13398

70.     Polit DF, Beck CT. Nursing Research: generating and assessing evidence for nursing practice. 10th ed. Philadelphia: Wolters Kluwer; 2017.

71.     Grant S, Crimmons K. Limitations of track and trigger systems and the National Early Warning Score. Part 2: sensitivity versus specificity. Br J Nurs. 2018;27(12):705–10. DOI: 10.12968/bjon.2018.27.11.624

72.     Grol R. Improving patient care: the implementation of change in health care. 2nd ed. Chichester: Wiley-Blackwell BMJ Books; 2013.

73.     Graverholt B. IMPAKT – Implementation and action for knowledge translation. Oslo: Unit; 2016 [updated 07.12.2021, cited 02.05.2021]. Available at: https://app.cristin.no/projects/show.jsf?id=550905