Water Data Transfer Format (WDTF): Guiding principles, technical challenges and the future

Abstract

In response to the growing water crisis in Australia, the Bureau of Meteorology has been given a mandate to develop and maintain an integrated national water information system. Section 126 of the Commonwealth Water Act 2007 provides for the making of Regulations to support these functions, which came into effect on 30th June 2008. These regulations (BoM, 2008) define the requirements for the collection of water data by the Bureau of Meteorology in 10 data type categories including flows, groundwater levels, reservoir storage, water quality, water use, water entitlements and water trades. Over 200 data providers are involved, grouped in 8 categories. This paper describes the joint effort by CSIRO and the Bureau of Meteorology to design the Water Data Transfer Format (WDTF), an eXtensible Markup Language (XML) (W3C, 2008) format for the ingestion of water data by the Bureau. The paper focuses on defining principles, technical challenges and the future of this work. WDTF development was guided by the following principles: • A solid modelling foundation: The format must be underpinned by a conceptual model expressed in a modelling language such as the Unified Modelling Language (UML) and supported by a model driven approach used to derive products such as XML Schema, documentation and validation tools. • Adopt, adapt, invent: Where best practice exists models and encoding should be adopted, adapted as required and finally invented if no other options exist. • Validation: All derived documents must be testable for compliance to the conceptual model and the Bureau of Meteorology's business rules. • XML for current tools: The format should be suitable to use with relevant standards-based commercial off the shelf tools. • Normalisation: Similar to database normalisation, the main entities or concepts should be encoded as distinct well-contained XMLSchema fragments, and identifiers used to define relationships between fragments. In following these principles WDTF has built on the International Standard Organisation's (ISO) General Feature Model, ISO 19109 (ISO/TC-211, 2005), and the Open Geospatial Consortium's (OGC) Observations and Measurements (O&M) model (Cox, 2007a & b). Encodings were based on an adaptation of O&M to a simple profile of OGC's Geography Markup Language (GML), to make it more compliant with current tools and ease the validation process. The paper also presents a summary of some of the challenges faced during development. These include: