Effectiveness of Alternative Statistical Approaches in Testing the Significance of the Null Hypothesis to Improve Reliability in Marketing Research
DOI:
https://doi.org/10.47134/jobm.v2i2.31Keywords:
Significance testing, Bayesian inference, bootstrap resampling, false discovery rate, marketing researchAbstract
This study examines the effectiveness of various alternative statistical approaches in testing the null hypothesis's significance to improve marketing research's reliability. Conventional methods such as p-value are often misunderstood and have limitations in interpreting the results of the analysis. Therefore, this study compares several alternative approaches, including Bayesian inference, bootstrap resampling, and false discovery rate, to improve the validity and repeatability of marketing research results. By using a simulation-based quantitative approach and a survey of marketing academics and practitioners, the findings of this study show that alternative methods can provide more informative results than conventional techniques. The implications of this study contribute to strengthening marketing research methodologies to be more accurate and reliable.
References
Benjamin, D. J., & Berger, J. O. (2019). Three Recommendations for Improving the Use of p -P-Values. The American Statistician, 73(sup1), 186–191. https://doi.org/10.1080/00031305.2018.1543135
Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society Series B: Statistical Methodology, 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
Berman, R., & Van den Bulte, C. (2022). False Discovery in A/B Testing. Management Science, 68(9), 6762–6782. https://doi.org/10.1287/mnsc.2021.4207
Berselli, N., Filippini, T., Adani, G., & Vinceti, M. (2021). Dismissing the use of P-values and statistical significance testing in scientific research: new methodological perspectives in toxicology and risk assessment. In Toxicological Risk Assessment and Multi-System Health Impacts from Exposure (pp. 309–321). Elsevier. https://doi.org/10.1016/B978-0-323-85215-9.00002-7
Boscardin, C. K., Sewell, J. L., Tolsgaard, M. G., & Pusic, M. V. (2024). How to Use and Report on P-values. Perspectives on Medical Education, 13(1), 250–254. https://doi.org/10.5334/pme.1324
Chernov, G. (2025). The Alternative Factors Leading to Replication Crisis: Prediction and Evaluation. Evaluation Review, 49(1), 147–164. https://doi.org/10.1177/0193841X241229106
Dr. Vijai Tiwari. (2024). Role of Data Analytics in Business Decision Making. Knowledgeable Research: A Multidisciplinary Journal, 3(01), 18–27. https://doi.org/10.57067/0zr57x43
Goodman, W. M., Spruill, S. E., & Komaroff, E. (2019). A Proposed Hybrid Effect Size Plus p -Value Criterion: Empirical Evidence Supporting its Use. The American Statistician, 73(sup1), 168–185. https://doi.org/10.1080/00031305.2018.1564697
Hartig, F., & Barraquand, F. (2022). The evidence contained in the P-value is context-dependent. https://doi.org/10.1016/j.tree.2022.02.011
Hofmann, M., & Meyer-Nieberg, S. (2018). Time to dispense with the p-value in OR? Central European Journal of Operations Research, 26(1), 193–214. https://doi.org/10.1007/s10100-017-0484-9
Horowitz, J. L. (2019). Bootstrap Methods in Econometrics. Annual Review of Economics, 11(1), 193–224. https://doi.org/10.1146/annurev-economics-080218-025651
Hung, K., & Fithian, W. (2020). Statistical methods for replicability assessment. The Annals of Applied Statistics, 14(3). https://doi.org/10.1214/20-AOAS1336
Indrayan, A. (2020). The Conundrum of P-Values: Statistical Significance is Unavoidable but Needs Medical Significance Too. Journal of Biostatistics and Epidemiology. https://doi.org/10.18502/jbe.v5i4.3862
Johansson, T. (2011). Hail the impossible: P-values, evidence, and likelihood. Scandinavian Journal of Psychology, 52(2), 113–125. https://doi.org/10.1111/j.1467-9450.2010.00852.x
Kamalbasha, S., & Eugster, M. J. A. (2020). Bayesian A/B Testing for Business Decisions. https://doi.org/10.48550/arXiv.2003.02769
Mayer, C. S. (1970). Assessing the Accuracy of Marketing Research. Journal of Marketing Research, 7(3), 285–291. https://doi.org/10.1177/002224377000700301
McShane, B. B., Bradlow, E. T., Lynch, J. G., & Meyer, R. J. (2024). “Statistical Significance” and Statistical Reporting: Moving Beyond Binary. Journal of Marketing, 88(3), 1–19. https://doi.org/10.1177/00222429231216910
Muehlemann, N., Zhou, T., Mukherjee, R., Hossain, M. I., Roychoudhury, S., & Russek-Cohen, E. (2023). A Tutorial on Modern Bayesian Methods in Clinical Trials. Therapeutic Innovation & Regulatory Science, 57(3), 402–416. https://doi.org/10.1007/s43441-023-00515-3
van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J., & van Aken, M. A. G. (2014). A Gentle Introduction to Bayesian Analysis: Applications to Developmental Research. Child Development, 85(3), 842–860. https://doi.org/10.1111/cdev.12169
Vidgen, B., & Yasseri, T. (2016). P-Values: Misunderstood and Misused. Frontiers in Physics, 4. https://doi.org/10.3389/fphy.2016.00006
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p -Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ardhitya Alam Wiguna, Aditya Nizar Al Ardi, Dinu Saadillah, Lukman Hakim, Candra Pramula Pinandita
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Phone. +6282230101105
Email. 
Web. 
