Clinical decision support system based on artificial intelligence for adjusting insulin pump parameters in children with type 1 diabetes mellitus

BACKGROUND

Insulin therapy is the main component of treatment for type 1 diabetes mellitus (T1D). The main insulin administration regimen is an intensified regimen (basal-bolus), and one of the methods is continuous subcutaneous insulin infusion (SCII) using insulin pumps (IP). Glucose self-monitoring is carried out using glucometers (blood glucose self-monitoring, BGSM), continuous glucose monitoring (CGM) or flash glucose monitoring (FGM). Intensified insulin therapy helps to reduce risk of diabetes-related complications [1][2], with the combination of IP with CGM/FGM being the most effective.

Despite the increasing availability of IPs and the emergence of highly accurate and affordable CGM and FGM, only about 21% of all children, 16% of adolescents [3], and 35% of adults [4] with T1D achieve glycemic control compensation (target glycated hemoglobin (HbA_1c) <7%). This is due to a large number of reasons, one of which is the need for regular monthly adjustments to insulin dosages. The widespread use of electronic devices for diabetes management (SCII, CGM, FGM) leads to the accumulation of a large array of electronic data, the analysis of which can be difficult. Previous studies have shown the effectiveness of regular and frequent insulin dose adjustments in improving glycemic control [5][6]. Despite more than 30 years of experience with an intensified insulin therapy regimen, there are no consensus guidelines for doctors and recommendations for patients on ways to optimize insulin therapy parameters [7][8]. Because of the absence of consensus, healthcare professionals are subjective in adjusting doses, mainly relying on their individual experience and therefore the doses vary widely. These problems make it relevant to develop an algorithm for adjusting insulin pump settings and assess the degree of agreement between recommendations provided by the pump and doctors’ expert opinions. A possible approach to the development of such an algorithm and a clinical decision support system (CDSS) based on it may use artificial intelligence (AI) technologies, including machine learning. AI can expand the possibilities of full-fledged, personalized and intellectual management of diabetes.

There is currently limited research available on the use of machine learning in the adjustment of insulin therapy in T1D patients [9][10]. Nimri R. et al. [9] evaluated the clinical performance of an automated Advisor Pro algorithm (DreaMed Diabetes Ltd, Petah Tikva, Israel) in a multicenter randomized trial. The limitations of this study are age of patients and the degree of underlying disease compensation at baseline – the age was 15.6 ± 3.0 years, HbA_1c 8.4 ± 0.8%. In this age group, hormonal shifts associated with growth and development of the body are approaching their completion in such patients, which greatly affect the course of T1D towards worsening; patients develop responsible attitude towards their disease, which leads to improved compensation of T1D. Tyler N. et al. [10] assessed the agreement of recommendations given by the algorithm and the doctors. The limitation of this study is that electronic syringe insulin delivery pens featured in the publication do not have marketing authorization in the Russian Federation.

Thus, the development of CDSS to optimize T1D insulin therapy continues to be relevant.

PURPOSE OF THE STUDY

Development of a mathematical model and CDSS based on it to optimize insulin therapy in children with T1D and assessment of the agreement of recommendations provided by CDSS and doctor on the parameters of insulin pump (IP) therapy: BP, basal profile; CF, carbohydrate factor, ISF, insulin sensitivity factor.

MATERIALS AND METHODS

Two studies were conducted sequentially.

I. Development of a mathematical model for glucose profile prediction

Study site and time – Pediatric Diabetes Mellitus Department, Endocrinology Research Centre, data were collected from January 2015 to June 2022.

Study population – children with T1D.

Inclusion criteria

1. Children of both sexes.
2. Age from 1 year to 18 years.
3. T1D (E10 according to ICD-10).
4. Disease duration – 1 year or more.
5. Insulin therapy using CSII during 3 months or more.
6. Glucose monitoring using CGM/FGM during months or more.

Exclusion criteria

1. Clinically significant acute diseases of the cardiovascular, nervous, genitourinary systems, or gastrointestinal tract; blood diseases.
2. Insulin therapy by multiple insulin injections for more than 14 days in the last month.
3. Systemic therapy with glucocorticoids.
4. Clinical diagnosis of diabetic retinopathy or maculopathy.
5. History of emotional, behavioral, or other disorders that may interfere with diabetes control and participation in the study.

Method of sample formation from the studied population – continuous.

Study design – prospective.

Materials

The following patient data were analyzed: sex, age (years), body weight (kg), height (cm), duration of diabetes (years), HbA_1c (%), data obtained from electronic devices Medtronic Paradigm MMT-715, Paradigm Real Time MMT-722, Paradigm VEO MMT-754, MiniMed 640G (start date of analysis period, end date of analysis period, IP settings (administered basal and bolus insulin, carbohydrates consumed, blood glucose), glucose monitoring data (glucose profile)).

The data obtained from the IPs manufactured by Medtronic did not contain information on the pump settings at the time of data transfer, therefore, the daily values of BP, CR, and ISF averaged over 3 hours for the selected period of time were calculated.

Methods

Considering the possibility of errors in the CGM/FGM data, these data were pre-processed:

1. removal of outliers – if the rate of change in glucose level exceeded 0.7 mmol/l per minute, then the current and neighboring (20 minutes) measurements were replaced with values calculated by linear interpolation;
2. smoothing – averaging of measurements by the moving average method with a period of 20 minutes (4 measurements).

Patients were randomized into two groups: 80% of patients included in the training set (n=403), 20% in the test set (n=101). When developing the pilot version of the model, the data of each patient of the training set were, in turn, divided into two parts – the first 80% of records of each patient were assigned to the training set, subsequent 20% of records were assigned to the test set.

To develop a mathematical model for predicting glucose levels, the method of building a recurrent artificial neural network (ANN) was used. The development tools used were the high-level language Python 3 and the open-source machine learning environment PyTorch [12]. Final recommendations of IP parameters are provided in intervals of 3 hours. Bolus and basal dosages are adjusted independently. The interval is assessed as basal if the bolus is not administered within three hours.

The prediction accuracy of the model on test data was estimated by root mean squared error (RMSE) and mean absolute error (MAE).

For the software implementation of the model and the IP setting optimizer based on it, the Python 3.9 programming language and the Dash Open Source 2.4.1 open library were used.

II. Evaluation of the agreement of recommendations provided by CDSS with doctors’ expert opinions

Study site and time – Pediatric Diabetes Mellitus Department, Endocrinology Research Centre, data was collected from August – December 2022.

Study population – same population as above.

Method of sample formation from the studied population – continuous.

Study design – cross-sectional.

Materials

Analyzed data included data from insulin pumps, CGM/FGM data, personalized patient metadata, and data on the adjustment of insulin therapy by doctors of the Pediatric Diabetes Mellitus Department, Endocrinology Research Centre, at remote telemedicine consultations based on documents in deferred mode for the last 28 days at the time of consultation. The IP parameters were corrected based on the last day and included 24 hourly settings of three IP parameters: BP, CF and ISF (72 settings in total).

Remote consultations were carried out by six highly qualified pediatric endocrinologists of the Pediatric Diabetes Mellitus Department, Endocrinology Research Centre. The glucose profile and IP settings (without prior adjustments by doctor) for each patient were fed to CDSS input, and 24 hourly settings for the same three parameters were obtained at the output.

Methods

The consistency of the hourly IP settings between the recommendations provided by CDSS and doctor was assessed in relation to 24 values of three parameters (BP, CR, ISF), which means 72 data points for each of the patients.

Consistency was assessed in two aspects:

1) direction of adjustment;

2) adjustment value.

Consistency of adjustments by direction was assessed using quadratic weighted kappa index (https://www.medcalc.org/calc/kappa.php) and by assessing the consistency of the directions of adjustments between CDSS and doctor as follows:

• complete consistency when the directions of adjustments from baseline values made by the doctor and CDSS matched, e.g. both decided to increase the parameter;
• partial consistency, where one decided to change the parameter and the other to keep it the same. For example, the doctor decided to increase the parameter, while CDSS decided to leave it unchanged;
• complete disagreement when the directions of adjustments from baseline values made by the doctor and CDSS were opposite. For example, the doctor decided to increase the parameter, while CDSS decided to reduce this parameter.

Consistency of adjustments by magnitude was assessed using:

• mean hourly values of IP parameters (regardless of consistency in the direction of adjustment) were estimated as the average value of the recommendations provided by CDSS or doctors’ recommendations over 24 hours for BP, CR, and ISF according to the formula:

where M – mean hourly value of IP parameter,

Val_i – value of IP parameter for the corresponding hour of adjustment;

• difference between the values recommended by CDSS and doctors’ recommendations (with direction of adjustment being in agreement), which was assessed using the mean absolute relative difference (MARD, %):

where R_doct – doctor’s recommendation,

R_CDSS – recommendation by CDSS,

n – number of periods.

Difference of up to 10% is considered clinically acceptable [11].

• number of intervals for adjustment of IP parameters (BP, CR, ISF) provided by CDSS and doctor (regardless of agreement in terms of direction).

Statistical analysis

Data analysis was performed in Statistica v.13 (TIBCO Inc., USA), MS Excel 2019 (Microsoft, USA), Python 3.10.2 using open-source libraries: statsmodels 0.13.2, and SciPy 1.9.1. Distributions of quantitative variables were described using median (Me) and lower and upper quartiles [ Q₁; Q₃]. The difference between quantitative features in dependent samples was evaluated using the Wilcoxon test (W-test), for independent samples the Mann-Whitney test (U-test) was used. Descriptive statistics of qualitative data are presented as absolute (n) and relative (%) frequencies. The confidence interval (CI) for the fractions was calculated using the Agresti-Coull method, for medians the bootstrap method was used. The quadratically weighted Kappa index and its 95% CI (https://www.medcalc.org/calc/kappa.php) were calculated to assess the agreement between IP adjustments. The significance level (P₀) was set at 0.05. In the case of multiple hypothesis testing, Bonferroni correction was applied.

Ethical review

The local ethics committee of the National Medical Endocrinology Research Centre in accordance with Minutes No. 17 of the committee meeting dated 28.10.2020 decided that the planned scientific activities meet the ethical standards of good clinical practice and can be allowed for realization at the site of the Pediatric Endocrinology Institute, Endocrinology Research Centre. Legal representatives of patients gave informed consent to anonymous use of clinical data.

RESULTS

I. Development of a mathematical model for glucose profile prediction

The pilot version of the model (INS v.1) and the IP setting optimizer based on it were developed using the data of 167 patients (44,376 patient-hours) [12]. Further, we continued to collect IP and CGM/FGM data; a total of 189,000 patient-hours (7,875 days) of data were obtained from 504 pediatric patients with T1D. Clinical and laboratory characteristics of patients are presented in Table 1.

The patient sample is representative – median age and HbA_1c in studies [12][13][14] are within the 95% CI of the medians of these characteristics of our population (Table 2).

After receiving additional data and their similar preprocessing, the next (final) version of the model (INS v.2) was developed, and the IP setting optimizer was also updated. The prediction of the glucose profile provided by the pilot and final versions of the model at time horizons of 30, 60, 90 and 120 minutes was evaluated (Table 3). The accuracy of glucose profile prediction provided by the final version of the model is arithmetically better according to the MAE criterion than those provided by the pilot version and foreign analogues.

The mathematical model INS v.2 and the optimizer of insulin dosages were implemented as a computer software – CDSS for optimization of IP parameters. The web application is located on the Internet (currently not freely available). Examples of interfaces are shown in Figures 1, 2.

II. Evaluation of agreement between recommendations provided by CDSS and expert opinions of doctors

The study included 40 patients, their clinical and laboratory characteristics are presented in Table 4. The total number of analysis points was 2880 for 40 patients, 960 points each for BP, CR and ISF.

The patient sample is representative – the medians of age and HbA_1c in other studies (Table 2) are within the 95% CI of the medians of the same characteristics in our sample (Table 2).

Assessment of agreement in BP, CR, and ISF adjustments (Table 5) by kappa index (quadratically weighted) showed a weak degree of agreement. When assessing the degrees of agreement, it was found that the complete agreement of the directions of adjustments of the IP parameters is in the range of 37.5-53.8%, the complete disagreement is 4.5-17.4% (Table 6, Fig. 1). From a clinical point of view, the degree of agreement is more important, and from the comparison of CI it follows that:

• complete disagreement of adjustment directions is statistically significantly less common than complete or partial agreement;
• for BP and CR, the frequencies of complete and partial agreement do not differ;
• for ISF, complete agreement is more common than partial agreement.

It can be seen from Table 7 that CDSS tends to adjust BP (decrease or increase) rather than left it unchanged; vice versa, the doctors more often left BP unchanged. Both CDSS and doctors more often changed BP parameters by decreasing than increasing them. CDSS increased CR more often than decreased it; the frequences of increases and decreases of CR by doctors are nearly equal. Concerning ISF, both CDSS and doctors left it unchanged in most cases, but if a change was made, CDSS more often decreased ISF, while doctors increased it.

Next, the agreement of adjustments by their magnitude was evaluated. Table 8 shows that there are no statistically significant differences in the median hourly parameters of ISF between CDSS and doctors. There are also no statistically significant differences between the CDSS recommendations and medical opinions when compared with the baseline ISF settings.

Table 9 provides descriptive statistics and CI for MARD. Discrepancies of up to 10% in the recommendations by CDSS and doctor are clinically insignificant, therefore, we consider that the agreement between recommendations by CDSS and doctor for adjustment of CR and ISF according to the MARD criterion is satisfactory. For BP, MARD exceeded 10% in 14% of cases (55 of 390 cases), 95% CI (11%; 18%)

CDSS tended to increase the number of intervals for adjusting CR in comparison with the baseline settings and doctors’ recommendations (Table 10). For BP and ISF, no statistically significant differences were found between the recommendations by CDSS and doctors.

DISCUSSION

I. Development of a mathematical model for glucose profile prediction

Sample representativeness

The sample is representative relative to the target population of children with T1D in terms of age and degree of carbohydrate metabolism compensation (Tables 1–2).

Comparison with other publications

Glucose profile prediction is an important component of CDSS. According to Oviedo S, et al.[18], glucose prediction models can be divided into 3 groups: physiological models use mathematical formulas describing food absorption and insulin action (Dall Man model [19], Hovorka model [20]); data-driven models use machine learning methods (on an electronic data array); hybrid models use machine learning methods supplemented by physiological models. The mathematical model we developed for predicting glucose levels belongs to the group of data-driven models, since only input and output data (electronic data from IP and CGM, patient metadata) were used in ANN training, between which patterns were revealed. Assessment of the effectiveness of our final model with other models from the group of data-driven models [16][17] showed a decrease in the glucose profile prediction error by about one and a half times.

II. Evaluation of the agreement between recommendations of CDSS and expert opinion of doctors

Sample representativeness

The sample is representative relative to the target population of children with T1D in terms of age and degree of carbohydrate metabolism compensation (Tables 2–4).

Comparison with other publications

To assess the possibility of clinical use of CDSS, we evaluated the agreement between recommendations of CDSS and doctors when adjusting IP settings, supplemented with CGM/FGM, in children with T1D. The rate of complete agreement across all three IP parameters (BP, CR, ISF) was in the range of 37.5–53.8% and partial agreement was 38.9–50.7%, which is clinically satisfactory and consistent with previous studies [21].

With complete agreement on the direction of adjustment, more pronounced differences between doctors and CDSS in the magnitude of adjustment (according to MARD) were found for BP rather than for CF and ISF. Disagreements up to 10% are considered clinically insignificant [11], therefore we consider the agreement between the recommendations by CDSS and doctors to be satisfactory in terms of adjustment magnitude.

Compared to doctors, CDSS increases the number of intervals for CR adjustment. Previous studies have shown that the need for insulin during the day is different, and this depends mainly on the age and stage of puberty [22]. Euglycaemic clamp study showed that it takes 2.5 to 4 hours until a significant change in basal infusion leads to a new stable level [23], so up to 10 intervals are effective with existing insulin analogues. CDSS has a limited number of intervals for all IP parameters – a maximum of 8.

CDSS tended to recommend increases in CRs and decreases in BP; thus, more insulin can be administered as boluses and less as basal doses. The conclusion that more insulin should be administered as boluses than basal doses was observed in studies with pump [24] and closed loop [25], and was found to be associated with better glycemic control and HbA_1c levels.

In a study by Nimri R. et al. [21] agreement was assessed between doctors (26 specialists from 16 countries) and between doctors and the automated algorithm Advisor Pro (DreaMed Diabetes Ltd, Petah Tikva, Israel) in IP adjustment in 15 patients with T1D (mean age 16.2 years ± 4.3, of which 4 patients were over 20 years old, mean HbA_1c 8.3% ± 0.9). Complete agreement between doctors, as well as between doctors and the algorithm on the direction of adjustment on all three IP parameters was the same ~ 45% (mean). It was found that the level of complete disagreement in the direction of adjustment was slightly higher for BP than for CF and ISF. This was also found in our study.

Distinctive features of our study as compared to the Advisor Pro study are the age range and the degree of compensation of carbohydrate metabolism. In the latter study, the majority of patients were from older age groups than in our study. This was associated with a stable course of T1D in patients of a foreign study due to the completion of sexual development (absence of sudden hormonal changes associated with the growth and development of the body), general stabilization of the lifestyle (the lifestyle of adults is more regular), these patients also develop responsibility and greater independence in T1D management. Our study included patients of all age groups, with all features of the course of T1D in each period of body formation. In a foreign study, carbohydrate metabolism compensation was worse than in our study. For patients with high HbA_1c levels compensation more often focuses on insulin therapy adjustment, while in case of medium HbA_1c levels the focus is more often on other issues. The results of the agreement of recommendations for the adjustment of insulin pump therapy between the doctor and the software in our and foreign studies are in good agreement. If our study was fully corresponding in the age range and the degree of compensation for carbohydrate metabolism to this foreign study, the degree of agreement between the recommendations of our software and medical opinions could be higher.

A study conducted in the Netherlands assessed factors that influenced the decision of 190 healthcare professionals to titrate basal insulin for T2D patients. I was found that even in T2D for which guidelines and official recommendations exist, and adjustment is simpler because only basal insulin is adjusted, the magnitude of insulin titration was found to be significantly different between specialists [26]. Thus, among doctors, even when treatment guidelines are available, there is a significant disagreement in approaches to treatment adjustment, which is probably due to different experience and motivation to achieve certain target parameters, as well as other subjective reasons. Therefore, comparing the agreement between CDSS and expert opinion of doctors can also show significant disagreement, which was demonstrated in our study.

Clinical relevance of results

The use of CDSS in clinical practice can help in regular and frequent observation of children with T1D, standardize the approach to the adjustment of IP parameters, supplemented with CGM/FGM. This will allow more efficient allocation of health care resources, personalized treatment and patient management.

Study limitations

Potential limitations of the use of CDSS are the need to use electronic devices for diabetes management (IP, CGM/FGM) of certain manufacturers (Medtronic, FreeStyle Libre), the minimum amount of data for analysis, possible data loss, and the need for manual data loading.

Further research areas

It seems promising to continue collecting data and using them to develop new versions of the model. In order to more accurately predict the glucose profile, it seems advisable to use such model predictors as data from approved electronic devices reflecting the influence of external factors on glucose patterns, for example, a pulse oximeter, a physical activity tracker, etc. Data collection could be simplified by using a cloud storage for automatic data download from devices in real time.

CONCLUSION

On a large sample of patients, a software product, CDSS, was developed to predict the glucose profile and adjust the parameters of insulin pump therapy supplemented with CGM/FGM. In terms of prediction accuracy, the developed model surpasses foreign analogues. The agreement between the recommendations provided by CDSS and the expert opinions of doctors is acceptable.

FURTHER INFORMATION

Funding. The study was carried out with the financial support of Alfa Group and the Culture of Charity foundation as part of the Alfa-Endo National Charity Program for helping children with endocrine diseases.

Contributions of authors. Sorokin D.Yu. – study design, creation of a database for training an artificial neural network, development of a program module (program interface module), research, statistical analysis of data, writing the manuscript; Trufanova E.S. – development of mathematical models, design of a software product, writing program code; Rebrova O.Yu. – scientific advice for data analysis, presenting study results, scientific editing of the manuscript; Bezlepkina O.B. – system analysis and justification of the need for development, formulation of requirements; Laptev D.N. – system analysis and justification of the need for development, formulation of requirements, development of a software module (module for graphic presentation of data), organization of research.

All authors approved the final version of the manuscript before publication, agreed to be responsible for all aspects of the manuscript, ensuring proper investigation and resolution of issues related to the accuracy or fidelity of any part of the manuscript.

Acknowledgments. The authors express their gratitude to the employees of the Pediatric Diabetes Mellitus Department, Endocrinology Research Centre: Ekaterina Andreevna Andrianova, Andrey Olegovich Emelyanov, Irina Alexandrovna Eremina, Tamara Leonidovna Kuraeva, Galina Nikolaevna Svetlova, Elena Alexandrovna Sechko, Elena Vitalievna Titovich – for participation in the study of the agreement between the recommendation provided by CDSS and expert opinion of doctors in the adjustment of insulin pump parameters.