Near Real-Time Experimentation for Quick Decision Making

Authors

Arjun Reddy Lingala

Abstract

With modern applications growing at rapid pace, it also grows the user base and it is very important to maintain and improve the user experience of the application. A/B Testing or experimentation is used for launching or making any change to the application. As companies are not sure which change users like and to decide which change users like more without launching the actual change, experimentation is used a lot these days in all major companies. Experimentation usually runs over multiple days or weeks in some cases depending on the population or exposures it has reached. Applications some times want to launch a new feature or change on a critical surface of the application for some amount of time. Critical surfaces quickly exposes the change to users and running experiments on critical surfaces for multiple days or weeks will impact the experience of the user and may change entire perspective of the application for the user. Running experiments for multiple days or weeks on surfaces which have high exposure is risky for the application. In this paper, we discuss near real-time experimentation address this risk by running experiments and computing results of the experiment frequently to check if the experiment results are statsig and stop the experiment and launch the variant that has highest positive results from the experiment

Keywords

Experimentation, A/B Testing, Real-Time, Statsig, Decision Making, Iot Systems, Data Pipelines, Adaptive Systems, Kafka, Batch Processing, Spark, Flink

Near Real-Time Experimentation for Quick Decision Making. Arjun Reddy Lingala. 2023. IJIRCT, Volume 9, Issue 2. Pages 1-7. https://www.ijirct.org/viewPaper.php?paperId=2412056