2003 · 15 citations · 6 references
ComputingData ScienceData AnalyticsData MiningTime Series AnalysisFault DetectionMissing Data ComputationMissing Data ImputationMissing Data MethodsSurveillance ApplicationsSurveillance Data AnalysisData ImputationDetectorsHandling Missing DataLinear RegressionOutlier ImputationReal-time ProcessingReal-time SurveillanceTraffic Surveillance
Single loop detectors provide the most abundant source of traffic data in California, but loop data samples are often missing or invalid. We describe a method that detects bad data samples and imputes missing or bad samples to form a complete grid of clean data in real time. The diagnostics algorithm and the imputation algorithm that implement this method are operational on 14,871 loops in six Districts of the California Department of Transportation. The diagnostics algorithm detects bad (malfunctioning) single loop detectors from their volume and occupancy measurements. Its novelty is its use of time series of many samples, rather than basing decisions on single samples, as in previous approaches. The imputation algorithm models the relationship between neighboring loops as linear, and uses linear regression to estimate the value of missing or bad samples. This gives a better estimate than previous methods because it uses historical data to learn how pairs of neighboring loops behave. Detection of bad loops and imputation of loop data are important because they allow algorithms that use loop data to perform analysis without requiring them to compensate for missing or incorrect data samples
6