Beautiful Data

Data Processing

Data processing is the important underpinning of the personal data collection system that users almost never see and usually are not interested in. They tend to be more interested in the results of the processing. This is the case for YFD. PEIR users, on the other hand, benefit from seeing how their data is processed, and it in turn affects the way they interpret impact and exposure.

The analytical component of PEIR consists of a series of server-side processing steps that start with GPS data to estimate impact and exposure. To be precise, we can divide the processing into four separate phases^[2]

1. Trace correction and annotation: Where possible, the error-prone, undersampled location traces are corrected and annotated using estimation techniques such as map matching with road network and building parcel data. Because these corrections and annotations are estimates, they do carry along uncertainties.

2. Activity and location classification: The corrected and annotated data is automatically classified as traveling or stationary using web services to provide a first level of refinement to the model output for a given person on a given day. The data is also split into trips based on dwell time.

3. Context estimation: The corrected and classified location data is used as input to web-based information sources on weather, road conditions, and aggregated driver behaviors.

4. Exposure and impact calculation: Finally, the fine-grained, classified data and derived data is used as input to ...