class: center, middle, inverse, title-slide .title[ # Survey Data Analysis with {Kobocruncher} ] .subtitle[ ## Session 7 - Weighting ] .author[ ### <a href="https://edouard-legoupil.github.io/kobocruncher/">Link to Documentation</a> – <a href="06-Indicator_calculation.html">Link to Previous Session</a> – <a href="08-Interpreting.html">Link to Next Session</a> ] .date[ ### Training Content as of 02 November 2023 ] --- ## Weighting: reducing selection bias! In the absence of survey data weighting, there's a risk over or under-represent certain demographic segments compared to their true level in the whole population. The use of weights in survey analysis compensates for this differential representation, thus producing estimates that relate to the target population. Weights vary due to differential sampling rates because of oversampling and because response and coverage rates vary across sub populations (specifically hard-to-reach demographic groups). The term weighting refers to the .large[__different amounts of load or emphasis__] we give to each individual survey response. Responses from a type of person who is very rare will get greater emphasis in the final results. In contrast, respondents who represent common types of people will be de-emphasized. Weighting are also important for the anonymization process too (see [next session](08-Anonymising.html)!) --- ## Original sampling designs - the design weights .pull-left[ When devising the orginal sampling strategy, an initial decision is made in term of design, each of them coming with weight calculation, between: * __Simple random sampling__: every unit has the same chance of being selected -- _Without replacement_: units can only be selected once -- or _With replacement_: units can be selected more than once * __Systematic sampling__: sample n individuals from a ordered list and sampling individuals at an interval with a random starting point * __Probability proportional to size__: probability of selection is proportional to "size" * __Stratified sampling__: divide population into mutually exclusive subgroups (strata). Randomly sample within each stratum * __Clustered sampling__: divide population into mutually exclusive subgroups (clusters). Randomly sample clusters and then individuals within clusters ] .pull-right[ Look at documentation associated with the analysis file to find out about the weight! Keywords to look for: methodology, design, analysis guide, technical documentation Documentation will indicate the variables needed to specify the design. Look for weight and: * strata and/or clusters/Primary Sampling Uunits. Sometimes pseudo-strata and pseudo-cluster with FPC (finite population correction) * replicate weights (in case they got calibrated afterwards...) ] --- ## Post stratification / raking - the calibration weights .pull-left[ Often, the original sampling strategy and plan is not respected.... * People that were initially targeted do not reply and do not want to answer * High non-response from certain groups * In case of telephone interviews, people change their phone and the data were not updated... .bg-blue[ In those situation,the original weights that were calculated when drawing the original sample needs to be recalculated for the analysis accounting for common demographic variables (_for instance: age, gender, location, and education_), but also other variables (_such as internet usage or mobile versus land-line phone users_) when available... ] ] .pull-right[ After weighting, when the population is compared to the weighted sample, the proportion of respondent from each age group should match  In order to reduce the potential risk of data weighting, it’s recommended to weight by as few variables as possible. ] ??? As the number of weighting variables goes up, the greater the risk that the weighting of one variable will confuse or interact with the weighting of another variable. Also, when data must be weighted, try to minimize the sizes of the weights. A general rule of thumb is never to weight a respondent less than .5 (a 50% weighting) nor more than 2.0 (a 200% weighting). https://www.pewresearch.org/methods/2018/01/26/how-different-weighting-methods-work/ --- ## Weighting to be used may also depends on the final indicator to be calculated .pull-left[ Also different weighting can be applied depending on the unit of analysis: * Household * Individual ] .pull-right[ .bg-blue[ Post-stratification and the development of different weighting is complex. .large[Reach out to UNHCR Global Data Service and your regional DIMA to get support] ] ] --- ## How to set it up in Kobocruncher for common weighting approach? By default, there's no weight. Add the weight depending on the level by adding a `weight_main` worksheet in your `xlsform`. The function `kobo_weight` will automatically look for the spreadsheet based on the name an based on the structure (prefix of the first variable name) will identify whether a cluster and stratification weighting should be applied. If you want to add specific weight for individuals (aka a nested table for instance called `members`), add another spreadsheet named accordingly (in the example above: `weighting_members`) .pull-left[ #### Setting up cluster weights The structure of your worksheet should include precisely the following variables: * `ids_XXX` : where XXX is precisely the name of the variable to use to identify the cluster in your dataset * `weights` : here's where you add the numeric values for the different weights * `fpc` : stands for Finite Population Correction is a numerical estimation of the population sizes within the specific cluster ] .pull-right[ #### Setting up stratification weights The structure of your worksheet should include precisely the following variables: * `strata_XXX` : where XXX is precisely the name of the variable to use to identify the strata in your dataset * `weights` : here's where you add the numeric values for the different weights ] ??? In case of more complex design such as two-stage cluster sample or stratified, cluster design, request for support Full presentation and course on [Tidy Survey Analysis with R](https://tidy-survey-r.github.io/tidy-survey-short-course/) data_srs %>% as_survey_design(fpc = fpc)# simple random sample (SRS) data_strat %>% as_survey_design(strata = stype, weights = pw)# stratified sample data_clus1 %>% as_survey_design(ids = dnum, weights = pw, fpc = fpc)# one-stage cluster sample data_clus2 %>% as_survey_design(ids = c(dnum, snum), fpc = c(fpc1, fpc2))# two-stage cluster sample, weights computed from pop size data_strat %>% as_survey_design(ids = dnum, strata = stype, weights =pw, nest = TRUE)# stratified, cluster design --- ## What will you get? A representation of uncertainity! .pull-left[ Once you set up your weights, all your charts will be represented by defaults with error bars (also called whisker) Error bars give a .large[__general idea of how precise a measurement is__], or conversely, how far from the reported value the true (error free) value might be. ] .pull-right[ Observe the error bar for the Proportion of PoC with Acces to health services  ] ??? Example of standard deviation, standard error and confidence interval library(ggplot2) library(dplyr) data <- iris %>% select(Species, Sepal.Length) # Data my_sum <- data %>% group_by(Species) %>% summarise( # Calculates mean, sd, se and IC n=n(), mean=mean(Sepal.Length), sd=sd(Sepal.Length) # sd amount of dispersion of the variable. Calculated as the root square of the variance ) %>% mutate( se=sd/sqrt(n)) %>% # se: standard deviation of the vector sampling distribution mutate( ic= se * qt( (1-0.05)/2 + .5, n-1)# ic: defined so that there is a specified probability that a value lies within i ) ggplot(my_sum) + geom_bar( aes(x=Species, y=mean), stat="identity", fill="forestgreen", alpha=0.5) + geom_errorbar( aes(x=Species, ymin=mean-sd, ymax=mean+sd), width=0.4, colour="orange", alpha=0.9, size=1.5) +# Standard deviation ggtitle("using standard deviation") ggplot(my_sum) + geom_bar( aes(x=Species, y=mean), stat="identity", fill="forestgreen", alpha=0.5) + geom_errorbar( aes(x=Species, ymin=mean-se, ymax=mean+se), width=0.4, colour="orange", alpha=0.9, size=1.5) +# Standard Error ggtitle("using standard error") # Confidence Interval ggplot(my_sum) + geom_bar( aes(x=Species, y=mean), stat="identity", fill="forestgreen", alpha=0.5) + geom_errorbar( aes(x=Species, ymin=mean-ic, ymax=mean+ic), width=0.4, colour="orange", alpha=0.9, size=1.5) + ggtitle("using confidence interval") --- class: inverse, center, middle # TIME TO PRACTISE ON YOUR OWN! ### .large[.white[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M368.4 18.3L312.7 74.1 437.9 199.3l55.7-55.7c21.9-21.9 21.9-57.3 0-79.2L447.6 18.3c-21.9-21.9-57.3-21.9-79.2 0zM288 94.6l-9.2 2.8L134.7 140.6c-19.9 6-35.7 21.2-42.3 41L3.8 445.8c-3.8 11.3-1 23.9 7.3 32.4L164.7 324.7c-3-6.3-4.7-13.3-4.7-20.7c0-26.5 21.5-48 48-48s48 21.5 48 48s-21.5 48-48 48c-7.4 0-14.4-1.7-20.7-4.7L33.7 500.9c8.6 8.3 21.1 11.2 32.4 7.3l264.3-88.6c19.7-6.6 35-22.4 41-42.3l43.2-144.1 2.8-9.2L288 94.6z"/></svg>] **5 minutes! **] <div class="countdown" id="timer_96ddf263" data-update-every="1" data-play-sound="true" tabindex="0" style="right:0;bottom:0;margin:5%;font-size:8em;position: relative; width: min-content;"> <div class="countdown-controls"><button class="countdown-bump-down">−</button><button class="countdown-bump-up">+</button></div> <code class="countdown-time"><span class="countdown-digits minutes">05</span><span class="countdown-digits colon">:</span><span class="countdown-digits seconds">00</span></code> </div> - Open again locally and fill in the weight tab in the xlsform - Regenerate the report and check the results Do not hesitate to raise your questions in the [ticket system](https://github.com/Edouard-Legoupil/kobocruncher/issues/new) or in the chat so the training content can be improved accordingly! --- class: inverse, center, middle ### .large[.white[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M256 0a256 256 0 1 1 0 512A256 256 0 1 1 256 0zM232 120V256c0 8 4 15.5 10.7 20l96 64c11 7.4 25.9 4.4 33.3-6.7s4.4-25.9-6.7-33.3L280 243.2V120c0-13.3-10.7-24-24-24s-24 10.7-24 24z"/></svg>] **Let's take a break! **] <div class="countdown" id="timer_385cae84" data-update-every="1" data-play-sound="true" tabindex="0" style="right:0;bottom:0;margin:5%;font-size:8em;position: relative; width: min-content;"> <div class="countdown-controls"><button class="countdown-bump-down">−</button><button class="countdown-bump-up">+</button></div> <code class="countdown-time"><span class="countdown-digits minutes">10</span><span class="countdown-digits colon">:</span><span class="countdown-digits seconds">00</span></code> </div> __Last session__: [08-AInterpreting](08-Interpreting.html)