Cloud Intelligence Optimizely: Cara Nggunakake Mesin Stats Kanggo A / B Tes sing luwih cerdas, lan luwih cepet

Mesin Stats Optimisely lan Strategi Tes A / B

Yen sampeyan pengin mbukak program eksperimen kanggo mbantu nyoba nyoba & sinau bisnis, kemungkinan sampeyan nggunakake Cloud Intelligence Optimizely - utawa paling ora sampeyan wis mriksa. Ngoptimalake minangka salah sawijining alat sing paling kuat ing game, nanging kaya alat apa wae, sampeyan bisa nggunakake salah yen ora ngerti cara kerjane. 

What makes Optimizely so powerful? At the core of its feature set lies the most informed and intuitive statistics engine in a third-party tool, allowing you to focus more on getting important tests live – without needing to worry that you’re misinterpreting your results. 

Kaya studi buta tradisional ing obat-obatan, A / B testing kanthi acak bakal nuduhake beda pangobatan situs sampeyan menyang pangguna liyane kanggo mbandhingake khasiat saben perawatan. 

Statistik banjur mbantu kita nggawe inferensi babagan sepira efektife perawatan kasebut sajrone jangka panjang. 

Most A/B testing tools rely on one of two types of statistical inference: Frequentist or Bayesian stats. Each school has various pros and cons – Frequentist statistics require a sample size to be fixed in advance of running an experiment, and Bayesian statistics mainly care about making good directional decisions rather than specifying any single figure for impact, to name two examples. Optimizely’s superpower is that it’s the only tool on the market today to take a paling apik kaloro donya pendekatan.

Asile pungkasan? Ngoptimalake pangguna bisa mbukak eksperimen kanthi luwih cepet, luwih andal, lan luwih intuisi.

In order to take full advantage of that, though, it’s important to understand what’s happening behind the scenes. Here are 5 insights and strategies that will get you using Optimizely’s capabilities like a pro.

Strategi # 1: Ngerti Ora Kabeh Metrik Digawe Merata

Ing umume alat uji coba, masalah sing umume diabaikan yaiku yen metrik sing ditambahake lan dilacak minangka bagean saka tes sampeyan, sampeyan bakal bisa ndeleng sawetara kesimpulan sing salah amarga ana kemungkinan acak (ing statistik, iki diarani "macem-macem masalah pengujian "). Supaya asil tetep bisa dipercaya, Optimizely nggunakake seri kontrol lan koreksi supaya kemungkinan kedadeyan kasebut paling sithik. 

Kontrol lan koreksi kasebut duwe rong implikasi nalika sampeyan nyiyapake tes ing Ngoptimalake. Kaping pisanan, metrik sing sampeyan pilih minangka Metrik Utama bakal entuk pinunjul statistik paling cepet, kabeh perkara tetep. Kapindho, luwih akeh metrik sing ditambahake menyang eksperimen, bakal luwih dawa metrik mengko bisa nggayuh pinunjul statistik.

Nalika ngrancang eksperimen, make sure you know which metric will be your True North in your decision-making process, make that your Primary Metric. Then, keep the rest of your metrics list lean by removing anything that’s too superfluous or tangential.

Strategi # 2: Mbangun Atribut Khusus Sampeyan

Optimizely is great at giving you several interesting and helpful ways to segment your experiment results. For example, you can examine whether certain treatments perform better on desktop vs. mobile, or observe differences across traffic sources. As your experimentation program matures though, you’ll quickly wish for new segments – these may be specific to your use case, like segments for one-time vs. subscription purchases, or as general as “new vs. returning visitors” (which, frankly, we still can’t figure out why that isn’t provided out of the box).

The good news is that via Optimizely’s Project Javascript field, engineers familiar with Optimizely can build any number of interesting custom attributes that visitors can be assigned to and segmented by. At Cro Metrics, we’ve built a number of stock modules (like “new vs. returning visitors”) that we install for all of our clients via their Project Javascript. Leveraging this ability is a key differentiator between mature teams who have the right technical resources to help them execute, and teams who struggle to realize the full potential of experimentation.

Strategi # 3: Explore Optimizely’s Stats Accelerator

One often-overhyped testing tool feature is the ability to use “multi-armed bandits”, a type of machine learning algorithm that dynamically changes where your traffic is allocated over the course of an experiment, to send as many visitors to the “winning” variation as possible. The issue with multi-armed bandits is that their results aren’t reliable indicators of long-term performance, so the use case for these types of experiments are limited to time-sensitive cases like sales promotions.

Nanging, kanthi optimal duwe macem-macem jinis algoritma bandit sing kasedhiya kanggo pangguna kanthi rencana sing luwih dhuwur - Stats Accelerator (saiki dikenal minangka opsi "Akselerasi Sinau" ing Bandit). Ing persiyapan iki, tinimbang nyoba nyedhiyakake lalu lintas kanthi dinamis kanthi variasi kinerja paling dhuwur, kanthi optimal kanthi dinamis nyedhiyakake lalu lintas menyang variasi sing paling gampang nggayuh pinunjul statistik. Kanthi cara iki, sampeyan bisa sinau luwih cepet, lan njaga replika asil tes A / B tradisional.

Strategi # 4: Tambah Emoji kanggo Jeneng Metric Sampeyan

Sekilas, ide iki bisa uga ora cocog, sanajan ora ana gunane. Nanging, aspek utama kanggo nggawe manawa sampeyan maca asil eksperimen sing bener diwiwiti kanthi manawa para pamirsa bisa ngerti pitakon kasebut. 

Kadhangkala, sanajan upaya paling apik, jeneng metrik bisa dadi bingung (ngenteni - apa metrik murub nalika pesen ditampa, utawa nalika pangguna entuk kaca matur nuwun?), Utawa eksperimen duwe akeh metrik sing nggulung munggah lan mudhun asil kaca nyebabake kabotan kognitif total.

Nambahake emoji menyang jeneng metrik (target, tandha centhang ijo, lan tas dhuwit sing bisa digunakake) bisa ngasilake kaca sing luwih bisa dipindai. 

Dipercaya karo kita - maca asil bakal luwih gampang dirasa.

Strategi # 5: Coba maneh Tingkat Signifikansi Statistik Sampeyan

Results are deemed conclusive in the context of an Optimizely experiment when they’ve reached pinunjul statistik. Signifikansi statistik minangka istilah matematika sing angel, nanging intine kemungkinan pengamatan sampeyan minangka asil prabédan nyata ing antarane rong populasi, lan dudu mung kasempatan acak. 

Optimizely’s reported statistical significance levels are “always valid” thanks to a mathematical concept called tes urut-urutan - iki sejatine nggawe dheweke luwih dipercaya tinimbang alat uji coba liyane, sing gampang ngalami macem-macem masalah "ngintip" yen sampeyan maca kanthi cepet.

It’s worth considering what level of statistical significance you deem important to your testing program. While 95% is the convention in the scientific community, we’re testing website changes, not vaccines. Another common choice in the experimental world: 90%.  But are you willing to accept a little more uncertainty in order to run experiments faster and test more ideas? Could you be using 85% or even 80% statistical significance? Being intentional about your risk-reward balance can pay exponential dividends over time, so think this through carefully.

Waca liyane babagan Cloud Intelligence Optimizely

Lima prinsip lan wawasan sing cepet iki bakal migunani banget kanggo mbudidaya nalika nggunakake Optimizely. Kaya alat apa wae, sampeyan kudu ngerti manawa sampeyan duwe pangerten sing apik babagan kabeh kustomisasi ing mburi layar, dadi sampeyan bisa nggawe manawa sampeyan nggunakake alat kanthi efisien lan efektif. Kanthi pangerten kasebut, sampeyan bisa entuk asil sing dipercaya sing digoleki, nalika dibutuhake. 

Apa sampeyan mikir?

Situs iki nggunakake Akismet kanggo ngurangi spam. Sinau babagan proses data sampeyan.