Population Assessment of Tobacco and Health (PATH) reliability and validity study: selected reliability and validity estimates

TLDR

The PATH Study questions were developed using a careful protocol, and the results indicate that the answers provide reliable and valid information about tobacco use. This study aimed to estimate the reliability and validity of responses to the Youth and Adult questionnaires of the Population Assessment of Tobacco and Health (PATH) Study by reinterviewing 407 adults and 117 youth 6–24 days after the initial interview. Reliability was assessed by comparing reinterview data with initial responses using kappa statistics, gross discrepancy rates, and correlations for all items with at least 100 observations. The analysis revealed very high agreement, with average kappa values of 0.79 for adults, comparable levels for youth, and 0.72 agreement with saliva test results, indicating that PATH questionnaire responses are reliable and valid.

Abstract

This paper reports a study done to estimate the reliability and validity of answers to the Youth and Adult questionnaires of the Population Assessment of Tobacco and Health (PATH) Study.407 adults and 117 youth respondents completed the wave 4 (2016-2017) PATH Study interview twice, 6-24 days apart. The reinterview data were used to estimate the reliability of answers to the questionnaire. Kappa statistics, gross discrepancy rates and correlations between answers to the initial interview and the reinterview were used to measure reliability. We examined every item in the questionnaire for which there were at least 100 observations. After the reinterview, most respondents provided a saliva sample that allowed us to assess the accuracy of their answers to the tobacco use questions.There was generally a very high level of agreement between answers in the interview and reinterview. On the key current tobacco use items, the average kappa (the agreement rate adjusted for chance agreement) was 0.79 for adult respondents (age 18 or older). Youth respondents exhibited equally high levels of agreement across interviews. The items on current tobacco use also exhibited high levels of agreement with saliva test results (kappa=0.72). Rating scale items showed lower levels of exact agreement across interviews but the answers were generally within one scale point or category.The PATH Study questions were developed using a careful protocol and the results indicate the answers provide reliable and valid information about tobacco use.