NCR COVID-19 Survey: Purposive Sampling and Panel Selection Explainer
Mr. AURELI C. SINSUAT
Executive Director
PUBLiCUS Asia, Inc.
This article provides a more in-depth discussion of the Purposive or Non-Probability sampling method used by PUBLiCUS Asia, Inc. and VOX Opinion Research, in partnership with Lightspeed Research/Kantar Asia-Pacific (Singapore), to construct the panel for the NCR COVID-19 Online Panel Survey.
What is Purposive Sampling?
Purposive sampling is a non-random sampling procedure as it does not give every element in the population a nonzero chance of selection, nor does it base sample selection through a randomization procedure. Instead, its sample is drawn according to a set of parameters or variables (i.e. samples have to be a certain age, gender, etc.).
Purposive sampling is generally done when insights from the units with the preferred parameters are desired or randomized procedures are unavailable, such as when units of a population are inaccessible via face-to-face survey because of COVID-19.
Purposive sampling has become especially prevalent as more and more surveys have moved online. More often than not, the primary sample source for online research is a panel of individuals who have been recruited in advance and agreed to do surveys. This is otherwise known as an opt-in panel, or an online panel not recruited via probability sampling.
Why partner with Lightspeed Research/Kantar Asia-Pacific?
The sampling approaches used with online panels vary substantially. In recent years, researchers working with opt-in panels have begun to explore techniques that go beyond simple quota sampling. The key point is that opt-in panels are not based on a single sampling method, but rely on myriad and varied methods.
This is the reason PUBLiCUS has partnered with Lightspeed Research/Kantar Asia-Pacific in conducting its panel surveys. The NCR COVID-19 Survey is the second partnership of PUBLiCUS with Lightspeed, the first being the pilot PAHAYAG NCR Online Panel Survey conducted in November 2017.
Lightspeed is a firm based in Singapore with 24 years of experience in recruiting and managing diverse and representative panels for corporate market and social research across Southeast Asia. According to its website, Kantar’s mammoth international data research network includes 88 million respondents in 90 markets worldwide.
We decided to avail of Lightspeed/Kantar’s services so that the general public could rest assured that the survey sample is not polluted and that the selection of panelists is done professionally and transparently. As per our standard operating procedure for all independent, non-commissioned surveys run by our firm, PUBLiCUS paid for the services of Lightspeed out of our own operating budget.
Lightspeed/Kantar’s Philippine Panel
The above graphic presents the demographic characteristics of Lightspeed’s panel reach. 24% of Lightspeed’s 100,000 panel reach reside in Metro Manila. It should be noted that Lightspeed’s country panel is recruited and maintained with the aim of being as representative as possible of the Philippine population based on official statistics.
Lightspeed/Kantar provided us a detailed write-up on its recruitment, profiling, incentive, and quality and validation methodologies. Excerpts are provided here for the sake of brevity:
RECRUITMENT: Kantar’s LifePoints respondent panel is an actively managed research panel. Several methodologies are used by Kantar to recruit panelists to the LifePoints panel, including opt-in email, co-registration, and e-newsletter campaigns, as well as both internal and external affiliate networks. Social media is included through our recruiting partners. To build a high-quality panel and remove the inherent bias that could result from using one or a few recruiting sources, we believe it is important to use a diverse set of recruitment sources that utilize a variety of recruitment methods. Kantar measures recruitment sources on multiple metrics to track both activity and engagement by demographic group, which contributes to the quality of data from our panelists.
PROFILING: Kantar monitors the panel composition and variables needed for sample selection by collecting household and demographic information from every panelist. We collect extensive socio-demographic profile information through a range of sector-specific screener surveys. Kantar employs a global profiling process across the LifePoints panel, which allows us to create quick and accurate feasibility estimates. Kantar’s dynamic profiling platform allows for each profiled attribute to be assigned an expiration date. When a profiled attribute expires, the dynamic profiling platform serves up the question during the respondent’s next online session to ensure all profiled questions are up to date.
INCENTIVES: We [Kantar] use LifePoints Reward Points to encourage panelists to participate in surveys. Upon completion of a survey, points are deposited immediately into a panelist’s account, which gives instant gratification for survey completion. The number of points awarded for survey completion is based on survey length, complexity, and incidence rate. Once a point threshold is reached, panelists may redeem their points for online gift certificates, charity donations, and PayPal cash deposits.
QUALITY AND VALIDATION: Kantar understands the importance of data and panel quality and was among the first to implement technology-driven quality programs. We will continue to deliver new quality methodologies to provide clients with the highest panel quality in the industry. Kantar invests significantly in the development and technology of our unique and proprietary quality and verification tools. They include a series of real-time checkpoints that new panel registrants pass through while completing the panel registration survey, as well as when they enter a new survey. Registrants who fail these checks are unable to join our panels and cannot participate in surveys.
Panel Selection for NCR COVID-19 Survey
This is how the panel for the NCR COVID-19 Survey was selected by Lightspeed:
- Initial discussions between PUBLiCUS and Lightspeed were conducted to establish what sampling method is the most appropriate to support the measure of the target population based on cost, coverage, and speed needs. Lightspeed set controls on key variables and select sample based on stored profile characteristics in its proprietary LifePoints panel and integrated suppliers in their private Profiles Network. During fieldwork, Lightspeed closely monitored additional invites based on response rates and specified quotas.
- An automatic queuing system sends email invitations to panelists pre-screened and qualified based on the survey parameters set by PUBLiCUS (aged 18-70 and residing in Metro Manila) at designated times stipulated by the Lightspeed project manager in consultation with PUBLiCUS. The survey link directs respondents to the LifePoints member home page. Lightspeed sends respondents only generic survey invitations with limited information about the survey. Generally, the survey invitation emails notify the respondent of an opportunity to share his/her opinions and, if applicable, informs him/her of the opportunity to earn rewards called LifePoints. The invitation instructs the respondent to log-on to lifepointspanel.com to access the structured survey questionnaire designed by PUBLiCUS. The survey responses are then collected and saved in the system and transmitted in real time to Lightspeed and PUBLiCUS simultaneously.
- A total of 5,850 survey invites were sent for this study. Respondents were sampled based on the required city quotas:
Disruption and Innovation in the Philippine Polling Industry
Certain individuals have questioned the credibility of the NCR COVID-19 Survey’s methodology. Regardless of whether these criticisms were scientific or partisan in nature, we hope the information provided here has put their minds at ease.
PUBLiCUS is fully aware that the purposive sampling method does not provide a sample as representative of the target population as the standard face-to-face random sampling method generally used in Philippine polls. This is precisely why the results were not reported as representative of the entire population of Metro Manila within a certain margin of error. PUBLiCUS will continue conducting face-to-face random sampling surveys via Computer Assisted Personal Interviewing (CAPI) in the future when circumstances allow for it.
PUBLiCUS is also aware that online panels are reliant on respondents with Internet access. According to the Digital 2020 Report by We Are Social and Hootsuite there are 73 millions Filipinos online as of January 2020 – 67% of the total population. 97% of these (70.45 million) are Mobile Internet users. It is a safe guess that the internet penetration rate of Metro Manila would be even higher than the national rate.
Source: 2020 Digital Report – The Philippines (We Are Social and Hootsuite)
All things considered, the NCR COVID-19 Survey conducted in partnership with one of the world’s leading Big Data companies and its methodology are scientific, transparent, and reliable. We also see the value of online panel surveys for their cost effectiveness and drastically reduced turnaround times from the end of fieldwork to the presentation of results. It takes days or even weeks to process, analyze, and report data generated from face-to-face surveys. PUBLiCUS can report actionable information to relevant stakeholders from its online panel surveys less than 24 hours after the last respondent clicks on the “Submit” button on the questionnaire – at a fraction of the cost incurred from face-to-face surveys.
Online panel surveys are already an established method of political and social research in the United States and other data-driven economies. As the emerging third player in the Philippine polling industry, PUBLiCUS will continue to develop its online polling capabilities as it seeks to conduct surveys with greater speed, higher efficiency, and at lower cost.
Reference: Mr. Aureli C. Sinsuat, PUBLiCUS Executive Director
Email: acsinsuat@publicusltd.com