국내 커뮤니케이션 연구의 통계분석방법의 현황과 문제점 지난 10년간 ˂한국언론학보˃ 게재 논문의 내용분석

Abstract

본 연구는 2010년부터 2019년까지 <한국언론학보>에 게재된 양적 연구 논문을 대상으로 통계 분석 방법 사용의 현황과 경향을 살펴보고, 이를 통해 국내 커뮤니케이션 연구의 통계 사용의 관행과 문제점을 논의하는 것에 그 목적이 있다. 이를 위해 10년간 <한국언론학보>에 게재된 총 845편의 논문 중 통계분석방법을 사용한 612편을 대상으로 내용분석을 진행하였다. 분석 결과, 지나치게 특정 통계분석방법에 의존하거나, 해석의 문제, 인위적인 결과를 생산하기 위한 통계적 방법의 오용 현상이 발견되었다. 이와 함께, 대부분의 연구들에게서 통계 방법에 관한 중요한 정보를 제시하지 않는 잘못된 관행도 발견되었다. 본 연구는 통계 분석 방법 및 보고에 대해 지금까지 많은 통계 학자들로부터 지속적으로 제기 되어온 비판과 논의들을 실증적으로 확인했다는 점에서 그 의의를 가진다. 도출된 결과를 바탕으로 통계 분석의 오용, 남용, 관행적 사용 등에 대한 문제점이 논의되었다.

Although statistical analysis methods are essential tools for quantitative researchers, controversy has persisted over the objectivity of inference through statistical procedures. The purpose of this study is to review the use and trends of statistical analysis methods in quantitative research papers published between 2010 to 2019 in the Korean Journal of Journalism and Communication Studies and to discuss its practices and problems of statistical use in communication research in Korea. For this purpose, 612 quantitative research papers using statistical analysis methods were content-analyzed out of a total of 845 papers published across 10 years. In this 612 quantitative research, the basic characteristics of the study, as well as the reliability between coders, the number of coders, and power for experimental design, were reviewed according to the methodology used. This study also checked for the use of null hypotheses and multi-item measures, and more specific details including the statistical packages, the basic assumptions of statistical analysis methods used in individual studies, such as t-test and regression, and the description of statistical results. Research papers were collected, reviewed, and analyzed by four graduate students majoring in advertising and public relations. The inter-coder reliability was measured by Kripendorff alpha, and the reliability for each item was between 0.71 and 1, thus ensuring a stable level of reliability. As a result of the analysis, research using surveys was the most common among the research methods, and regression was the most frequently used statistical method except for descriptive statistics. The most commonly used statistical package was SPSS. The current study found problems of over-reliance on a specific statistical package, erroneous interpretation of statistical analysis results, and misuse of statistical methods for yielding contrived results. In particular, over-reliance on a specific statistical package was related to over-reliance of specific statistical analysis methods such as partial eta squared and cronbach alpha. At the same time, in most studies, wrong practices were also found. For example, important information regarding the processes and results of statistical analysis was not provided, such as basic statistical assumptions and correlation between major variables, and neither was information on confidence intervals as supplementary indicators of null hypothesis significance testing. Given that this study empirically identified the criticisms and discussions that have been continuously raised about statistical analysis methods and reporting, this study holds implications. Through this study, it is hoped that future Korean communication researchers will actively carry out research on various problems and alternatives raised in individual statistical analysis methods, thereby enriching academic discussions.