An analysis of qualitative and mixed methods abstracts from Japanese, UK and US primary care conferences

Abstract

Background

As research in family medicine covers varied topics, multiple methodologies such as qualitative research (QR) and mixed methods research (MMR) are crucial. However, we do not know about the difference in the proportion of QR or MMR between Japan, the UK and the US. This knowledge is needed to shape future research within countries with developing primary care such as Japan and other Asian countries. This study aims to describe the use of QR and MMR in Japanese primary care and compare this to the UK and US; then to make informed recommendations for primary care research.

Methods

A repeated cross-sectional study (2012–2016) based on the abstracts submitted to the annual conferences of the Japanese Primary Care Association in Japan, the Royal College of General Practitioners in the UK, and the North American Primary Care Research Group in the US and other North American countries. The proportions of QR/MMR among all the posters and paper presentations for each of these three conferences were assessed. Also examined were trends and types of qualitative techniques for all three countries and participants/settings for Japan.

Results

There were 1080 abstracts for Japan, 575 for UK and 3614 for US conferences. QR/MMR proportions were 7.5%, 15.1% and 28.1%, respectively. Japan’s proportion was lower than that of UK and US (p < 0.001). The proportion was increasing over time for the UK (p = 0.02). Steps for coding and analyses was most popular for Japan, thematic analysis for the UK and grounded theory for the US. Primary care doctors and hospitals were the commonest contexts for Japan.

Conclusions

QR and MMR were not as popular in primary care in Japan compared to the UK and the US, whereas their use was increasing in the UK. Approaches, participants and settings may differ among these countries. Education and promotion of QR/MMR and multi-disciplinary collaborations need to be recommended in Japan with developing primary care.

Background

Quantitative research had been predominant in health care research for a long time. Recently, however, the importance of qualitative research (QR) has been recognized as it contributes to deeper understanding and interpretation of the meaning of phenomena in the real world [1]. In addition, mixed methods research (MMR), which combines and integrates quantitative methods with qualitative methods, has become popular in order to capitalize on the advantages of both methods [1].

Although QR and MMR were more often used in nursing disciplines compared to medicine [2], QR and MMR are indispensable for family medicine. The reason is that family medicine is, according to Miller and Crabtree, “a clinical domain where balancing qualitative and quantitative research styles benefits both patients and health care professionals” (A quantitative analysis of qualitative studies in clinical journals for the 2000 publishing year page 2) [2]. Moreover, research in family medicine also encompasses a variety of topics so that multiple methodologies are particularly crucial [3]. Therefore, researchers in family medicine need to acquire proficiency in QR and MMR skills.

In countries with long-established family medicine systems such as the United Kingdom (UK) and the Unites States (US), QR and MMR have become popular. For instance, in the UK, a quarter of submissions to the British Journal of General Practice used qualitative methods with a similar acceptance rate to quantitative studies and these articles were highly cited [4]. In addition, in the US, the Society of Teachers of Family Medicine stressed the importance of qualitative methods in family medicine education [5]. On the other hand, in Japan, family medicine has not been well established, where physicians who received an internal medicine based training program have played a principal role in the primary care setting [6] and the system of certification for family physicians as a new category of specialist has just established since 2017 by an independent third-party organization [7].

Although the editorial board of An Official Journal of the Japan Primary Care Association is planning to develop a guideline for submitting qualitative research [8], in countries with developing family medicine systems such as Japan and other Asian countries, the importance of QR and MMR is not emphasized. This is important to know for these key research-leading countries to enable collaboration with Japanese and other Asian family physicians in order to appropriately mold the future of primary care research.

The question is, therefore, how are qualitative and mixed methods used in primary care research in Japan compared to the UK and US in recent years, and what can be done to improve this?

The aim is to describe the proportion, trend, types and characteristics of QR and MMR in primary care research in Japan compared to in the UK and US between 2012 and 2016, using the numbers of abstracts submitted to major annual conferences, and to make recommendations for QR and MMR development based on these results.

Methods

We conducted a repeated cross-sectional study on the numbers of abstracts accepted to major annual conferences in primary care in Japan, UK, and US

Subjects

As major academic conferences in primary care, we selected the Japan Primary Care Association (JPCA) annual conference as a representative of primary care in Japan; the Royal College of General Practitioners (RCGP) annual conference for the UK; and the North American Primary Care Group (NAPCRG) annual meeting for the US. The JPCA is the main primary care association in Japan with approximately 11,000 members [9]. The RCGP is the oldest primary care society in the world with 50,000 members [10]. The NAPCRG is the major primary care society in the US and northern American countries [11].

The study subjects were the accepted abstracts for all posters and oral presentations in the annual conferences of the JPCA, the RCGP and the NAPCRG between 2012 and 2016. In April 2017, we searched these associations’ websites and if insufficient information was available, we contacted the associations for the titles and abstracts. We defined the studies using QR and MMR as all posters and oral presentations that included any of the words below in the title or abstract; for study design, “qualitative/qualitative research/survey”, “mixed-methods” and “qualitative and quantitative”; for analytical method, “grounded theory”, “phenomenology”, “ethnography”, “case study”, “discourse analysis”, “narrative”, “KJ method”, “content analysis”, “action research”, “field work”, “life story”, “thematic analysis”, SCAT”, “immersion-crystallization approach” and “constant comparative approach”.

We also categorized the participants, settings and diseases/conditions of the included studies in the JPCA conferences. The details of eligibility criteria and categorization were shown in Table 1. We shared the work of classification/categorization between authors and when the classification of the study was ambiguous, discussion continued until a unanimous consensus was reached.

Table 1 Eligibility criteria and categorization of QR and MMR in the study

Statistical analysis

We calculated the total proportion of the selected studies that are QR and MMR in each conference; the denominators were the numbers of all studies and the numerators were the numbers of QR and MMR.

We used Chi squared test for the comparison between the proportion in the JPCA annual conference and the RCGP annual conference and, separately, the NAPCRG annual meeting; the null hypothesis was of no difference in proportions. The significance level selected was 5%. We also used the Cochran–Armitage test to assess the annual change in this proportion for each conference; the null hypothesis was that of no change. The significance level selected was 5% with two-sided tests. We assessed the number and proportions of the different types of qualitative approaches used for each country’s conferences. Finally, we assessed the number of eligible studies in each category of participants and settings for the JPCA conferences. The statistical package used was Stata, version 15.

Results

We searched 1080 studies for the JPCA Annual Conference, 575 for the RCGP Annual Conference, 3614 for the NAPCRG Annual Meeting respectively. The abstracts of the RCGP annual conference 2012 and 2013 were not available despite enquiry and efforts by the office of the RCGP.

The total proportions of QR and MMR were 7.5 (n = 81, 95% CI 5.9, 9.1), 15.1% (n = 87, 95% CI 12.2, 18.5) and 28.1% (n = 1016, 95% CI 19.0, 21.7) for the conferences of the JPCA, RCGP and NAPCRG, respectively for the study period. The proportion of QR and MMR in the JPCA Annual Conference was significantly lower than that in the RCGP Annual Conference (p < 0.001) and the NAPCRG Annual Meeting (p < 0.001). The proportion of QR and MMR in the RCGP Annual Conference increased yearly (p = 0.02) and those of the JPCA Annual Conference and the NAPCRG Annual Meeting displayed no significant changes. Although the proportion of MMR in the US was increasing, the trend was not statistically significant (p = 0.054).

Table 2 displays these results and the breakdown of studies into QR or MMR. QR was more often conducted in comparison with MMR.

Table 2 The proportion of posters and oral presentations that were QR or MMR in each conference and the annual change

The most frequently used qualitative method in the JPCA Annual Conference was Steps for Coding and Analyses (SCAT), thematic analysis in the RCGP Annual Conference, and grounded theory in the NAPCRG Annual Meeting (Fig. 1).

Fig. 1

Qualitative approach for QR and MMR studies at each country’s conference

In QR and MMR in Japan, family physicians were the commonest participants (31 studies). Other common participants were patients (9) and nurses (6). The most frequent study setting was hospital (29), followed by community (24) and clinic (20). In terms of diseases/conditions, almost all of the studies were classified in Various. Other diseases/conditions were mental health (2), death and dying (2) and frail elderly (2).

Discussion

In comparing the numbers of abstracts submitted to major annual conferences in primary care, the proportion that were QR or MMR in Japan were lower than those in the UK and the US. The trend was not changing in recent years, but it was increasing in the UK. SCAT was the most popular qualitative approach for primary care research in Japan, thematic analysis in the UK and grounded theory in the US.

Our finding of the low proportion of QR and MMR in Japanese primary care may be explained by insufficient development of clinical research in primary care [12], low awareness of the significance of QR and MMR, and lack of educational resources such as mentors. In addition, reporting of QR and MMR studies in the English language may be difficult to perform for Japanese researchers compared to quantitative studies because QR and MMR conducted in Japanese may be harder to translate into English for publication due to a higher reliance on complex language skills. English language QR and MMR may also, therefore, be harder to understand for Japanese researchers, which may discourage learning about and specializing in QR and MMR. Saiki, however, indicated that QR was increasing annually in Japanese health-care journals overall and the number of QR studies had increased 10-fold in the past 10 years [13], which was explained by an increase in Japanese textbooks about QR and MMR [13]. Therefore, although our study revealed no significant change in the proportion of QR and MMR in primary care, these research methods may become popular in the near future. Based on the results of our study and the current literature, it is necessary to examine further the barriers and facilitators for QR and MMR in Japan and other Asian countries with developing primary care. In the UK, large organizations have mentioned that QR/MMR are essential to primary care research, e.g. the Academy of Medical Sciences in their report on research in general practice [14]. Also, in line with this, almost all primary care departments have qualitative research groups or forums to foster these skills amongst new researchers, which expands the research base, e.g. Oxford and Cambridge [15, 16]. These factors may affect the increase of the proportion of QR/MMR in the results.

We also do not know if the proportions of QR and MMR in the UK and US are sufficient or appropriate for the needs of their primary care systems. The proportion of QR and MMR is not the only indicator of the influence of qualitative and mixed methods; we also need to consider the quality of research and relevance to clinical practice.

In addition, the most frequently used qualitative method in the JPCA was SCAT. SCAT is a qualitative analytical method developed by Ohtani, Japan [17]. The method consists of 4-step coding and description of storyline, suitable for the analysis of small-size qualitative data and easy to use for novices [17]. The SCAT website has described “how to use SCAT”, “Frequently Asked Questions” and “Tips and Pitfalls” for free in Japanese [17]. Also, the author of SCAT has provided family physicians with workshops overall Japan [17]. Moreover, some research course for family physicians have included a lecture series on SCAT [18]. These rich resources may explain why it is more widely understood and used in Japan. Provision of such resources in other methods may be helpful to distribute QR/MMR among Japanese family physicians. This study also found that, in Japan, family physicians were the commonest participants in QR and MMR, and hospitals were the most frequent study settings. The results may be explained by the fact that almost of all members of the JPCA were family physicians [9] and 43.7% of the JPCA certified family physicians work not only at clinics but also at hospitals [19].

In contrast, according to an analysis of QR in 170 English language journals, patients and family were the first and second most common types of participants, and the community was the commonest setting [2]. They also found that half of the qualitative studies were conducted by nurses [2]. This situation is similar in Japan, where most QR is published in nursing journals [20]. In order to enhance the use of QR and MMR for patient-centered care in Japanese and other Asian countries’ primary care, multi-disciplinary collaborations would be useful.

To deal with the wide-range research questions in primary care, both qualitative and quantitative methods play important roles [3]. In Japan, Aomatsu pointed out that the qualitative method is indispensable to deeply understand the problems of patients with respect to not only the biological phenomena but also the psycho-social and contextual phenomena in primary care settings [8]. QR and MMR are also useful to describe and understand complex issues without an over-simplification and can identify interactive relationships between problems [8].

Strengths and implications

This study covers a largely unexamined issue that is important for the global development of primary care via research. The large sample of studies over time and the use of abstracts of conference posters and presentations had possibly increased the representativeness of our findings for the general state of primary care research in these countries. The examination of the characteristics of qualitative research illustrated the need to further describe the problem, and by this, seek possible solutions for the underuse of qualitative research. The following lessons could be learned for Japan and other Asian countries from the pattern of UK and US primary care research: education and promotion of QR and MMR are highly recommended in Japanese and other Asian countries’ primary care, which could be achieved by promoting domestically developed analytical methods, supporting translations of their studies reported in non-English languages into English for publication, and multi-disciplinary collaborations. QR and MMR may also be made more relevant by focusing on patients and the community instead of secondary care settings.

Limitations

We only included presented posters and oral presentations, therefore the data from workshops, reports and others were not used, which may have caused selection bias. We may therefore have underestimated the use of QR and MMR in Japan. Moreover, we did not differentiate poster presentations and oral presentations in the process of data collection. Thus, this study could not compare the proportion of oral presentations of QR/MMR with other study designs. We also assumed conferences from the JPCA, the RCGP and the NAPCRG were representative of each country’s primary care research. The selection of the conferences was based on our knowledge and information from their websites, but selection bias may have been introduced by not examining other conferences. If other conferences about QR/MMR and primary care included more studies with QR/MMR in primary care, we might have underestimated the proportion of QR and MMR conducted in these countries. Moreover, preference and knowledge about QR/MMR among reviewers in these conferences may have influenced the results. Also, we reviewed only abstracts of the studies, and sometimes the information about the analytic approach was not included. Thus, our results did not necessarily describe the precise picture of the analytic approach due to its lack of description in conference abstracts. The results therefore need careful interpretation. Moreover, as the nationalities of researchers were not known from the title/abstracts, this study assumed that authors were from the countries in which the conferences were based. As such, there is a possibility that the research conducted by authors from other countries could have been included in the conference abstracts, therefore introducing misclassification. However, we excluded “international session” from our study. Thus, the proportion of studies by other countries’ researchers may not change our results. Lastly, we shared the work of classification/categorization, and it could have led to the variances of classification.

Conclusions

QR and MMR were not highly prevalent in primary care within Japan compared to the UK and the US, whereas their use was increasing in the UK. Approaches, participants and settings may differ among these countries. Education and promotion of QR/MMR and multi-disciplinary collaborations should be recommended in Japan.