WEATHER-5K: A Large-scale Global Station Weather Dataset Towards Comprehensive Time-series Forecasting Benchmark (2024)

2.1 Time-series Forecasting

Time-series forecasting involves many domains[14, 17, 18, 19] and their methods have undergone three stages since its birth: statistical learning, machine learning, and deep learning.

Statistical learning methods, including ARIMA[20], ETS[21], StatsForecast[22], VAR[14], and Kalman Filter (KF)[23], are among the early proposals and are widely utilized. These methods rely on historical data to predict future values and are based on the assumption that past observations hold predictive power. While they provide a solid foundation, their performance may be limited when faced with complex patterns or nonlinear relationships.

Machine learning methods have gained prominence in time-series forecasting due to rapid advancements in the field. Algorithms such as XGBoost[24], GBRT[25], Random Forests[26], and LightGBM[27] offer enhanced capabilities to handle nonlinear relationships and complex patterns. These methods demonstrate flexibility in handling different types and lengths of time-series data and generally provide superior forecasting accuracy compared to traditional statistical methods.

Leveraging the representation learning capabilities of deep neural networks, Deep-Learning (DL) methods have shown promising results in time-series forecasting. DL treats time-series as sequences of vectors and utilizes architectures such as convolutional neural networks (CNNs)[28], recurrent neural networks (RNNs)[29], or Transformers[30] to capture temporal dependencies. For example, TCN[31] and DeepAR[32] implement CNNs or RNNs to model the temporal structure of the data. Transformer architectures, like REformer[33], Informer[34], Pyformer[35], FEDformer[15], Autoformer[13], Triformer[36], and PatchTST[37], have also been applied in time-series forecasting tasks, allowing for the capture of more complex temporal dynamics and significantly improving forecasting performance. In addition, while pursuing forecasting accuracy with complex models, MLP-based models such as N-HiTS[38], N-BEATS[39], and DLinear[40] employ a straightforward architecture with a relatively low number of parameters while achieving competitive performance. Recently, Mamba[41], a selective state space model, has also gained traction due to its ability to process dependencies in sequences while maintaining near-linear complexity. Some variants of Mamba[42, 43, 44] have been successfully applied to time-series forecasting.

2.2 Data-driven Numerical Weather Prediction

Since 2022, there has been a growing interest in data-driven Numerical Weather Prediction (NWP) models within the AI and atmospheric science communities. The correlations between NWP and GSWF are as follows: 1) GSWF can be obtained by interpolating the forecast results of NWP models to specific latitudes and longitudes. Similarly, GSWF can also be used to bias-correct the forecast results of NWP models. These models, like, Pangu-Weather[9], GraphCast[8], FengWu[10], and FengWu-GHR[11], have shown the potential to outperform traditional physical-based NWP models in terms of forecast skill and operational efficiency. However, these models, operating at the mesh space ( e,g., the grid resolution of $0.25^{\circ}$ and $0.09^{\circ}$), are may not be the optimal solution for GSWF as discussed in Section1. Prior to our work, some initial attempts, like Corrformer[12], have treated GSWF as an independent forecasting task, demonstrating promising results but remaining a great gap compared with the data-driven NWP models.

3.1 Collection and Processing

Our dataset is collected using data sourced from the National Centers for Environmental Information (NCEI), specifically the Integrated Surface Database (ISD),¹¹1https://www.ncei.noaa.gov/products/land-based-station/integrated-surface-databasea global repository of hourly and synoptic surface observations gathered from numerous original data sources. The ISD encompasses a wide range of meteorological parameters recorded by each weather station, including wind speed and direction, temperature, dew point, cloud data, sea level pressure, altimeter setting, station pressure, present weather, visibility, precipitation, snow depth, and various other elements.Although the ISD contains records from over $20,000$ stations spanning several decades, not all stations are suitable for machine learning applications.For instance, certain stations are no longer operational, many do not report data on an hourly basis, and numerous stations have missing values for critical weather elements.To create the WEATHER-5K dataset, a meticulous selection process was conducted to include only long-term, hourly reporting stations that are currently operational and provide essential observations such as temperature, dew point temperature, wind, and sea level pressure.

To obtain a high-quality dataset of weather stations, a series of post-processing steps were performed on the raw weather station data collected from 2014 to 2024. Initially, $10,701$ commonly operating stations were identified. The first step involved selecting stations that reported data every hour on the hour. However, many stations did not meet this criterion. To address this, a replacement method estimated missing hourly data points using the nearest available time points within a 30-minute window, significantly improving the distribution of valid hourly data.

Despite this improvement, a tiny portion of hourly data points remained missing due to the lack of observations within the 30-minute window. To fill these gaps, linear interpolation was employed using data from the 12 consecutive hours surrounding each missing point, ensuring the reliability of the interpolated data.Moreover, stations with more than 90% valid hourly data were chosen as the final candidates. To ensure the dataset’s completeness, ERA5 reanalysis data[45] was utilized to interpolate and fill any remaining gaps in the weather station data.

As a result of these post-processing steps, a subset of $5,672$ weather stations worldwide was selected, spanning the period from 2014 to 2023, ensuring a recent and relevant time frame. This selection process focused on balancing the longevity of station operation, hourly data availability, and the inclusion of diverse weather variables. Consequently, our dataset provides a robust foundation for evaluating forecasting models in the context of GSWF.

3.2 Qualitative Analysis

Regional disparities of stations.The WEATHER-5K dataset reveals regional disparities in the distribution of weather stations, which can significantly impact the learning and understanding of atmospheric dynamics in certain areas. As illustrated in Figure1, some land regions have sparse data coverage compared to others. These disparities can be attributed to factors such as geographical characteristics, levels of economic development, and the strategic placement of weather stations. Note that the number of oceanic stations is also very limited due to the expensive cost of establishing stations at sea. Regions with limited station coverage may exhibit unique weather patterns and phenomena that are not adequately captured due to insufficient data. Addressing these disparities and expanding coverage in underrepresented areas is crucial for improving the accuracy and reliability of weather forecasting and analysis in those regions.

Data characteristics. Here we present a statistical analysis for different characteristics of different variables and the discrepancy of the same variable over different latitudes based on one year of data, By visualizing the four noteworthy observations: temperature, wind speed, wind direction, and sea level pressure, There are three primary findings: First, as shown in Figures2 a), b), and c), temperature characteristics in the Northern and Southern Hemispheres are opposite during the same seasons. In the equatorial region, the variance for its seasonal pattern is generally lower compared to higher latitudes. Second, Figures2 d) and e) indicate Wind Speed and Direction. They show that the wind speed and direction are non-stationary, characterized by intense fluctuations and a lack of clear patterns, making them challenging to predict. Last, As depicted in Figure2 f), Sea Level Pressure shows a stable distribution with fewer fluctuations, making it relatively easier to predict.

Comparison with existing datasets.Table1 presents a comparison between the WEATHER-5K and other popular time-series datasets. The development of time-series datasets suffers from several limitations. 1) Small scale and Out-of-date: The mainstream time-series datasets[46, 47, 13] used for research purposes remain relatively small in scale. For instance, datasets related to electricity consumption or exchange rates are sparse or outdated, which limits the practical application of forecasting models. These two natures also hinder the exploration of more scientific challenges associated with time-series forecasting.2) Lagging behind other fields: Compared to natural language processing and computer vision, the time-series forecasting domain has been slower in incorporating large-scale datasets. The use of extensive datasets (e.g., CommonCrawl and LAION-5B[50]) in other fields have demonstrated unprecedented economic value and significantly advanced scientific discoveries. However, until recently, the first large-scale time-series dataset, LargeST[48], is only introduced.In contrast, WEATHER-5K addresses the limitation of small-scale datasets by providing a comprehensive and large-scale collection of weather station data. This abundance of data enables researchers to tackle more complex forecasting challenges and explore the dynamics of weather patterns on a global scale.

4.1 Problem Definition

Considering $N$ weather stations around the world and each station collects $V$ meteorological variables, the data of all weather stations can be represented by a spatial-temporal time-series $X\in\mathbb{R}^{N\times T\times V}$ for a given look-back window of fixed length $T$. At timestamp $t$, time-series forecasting is to predict $\hat{X}_{t+1:t+\tau}=\{X_{t+1},...,X_{t+\tau}\}$ based on the past $T$ frames $X_{t-T+1:t}=\{X_{t-T+1},...,X_{t}\}$. Here, $\tau$ is the length of the forecast horizon. Using $X$ and $\hat{X}$ to represent the observation data and the forecasted data, respectively, the process of GSWF can be simplified by a mapping: $\hat{X}=\mathcal{M}(X)$, where $\mathcal{M}$ can be different kinds of time-series forecasting methods. For example, by setting $N=1$ and ignoring the spatial information, many state-of-the-art time-series forecasting methods[49, 34, 35, 13, 15, 41, 51, 37, 16] can be explored on this task. When $N$ is multiple scattered stations, method[12] based on spatial-temporal modeling can also be applied to this task.

4.2 Evaluation Metrics

Overall performance. Mean Absolute Error (MAE) and Mean Square Error (MSE) are used to evaluate the overall performance of the GSWF. MAE measures the predictive robustness of an algorithm but is insensitive to outliers, whereas MSE is sensitive to outliers and can amplify errors.

4.3 Experimental Protocols

Dataset splitting. The WEATHER-5K dataset is divided into three subsets: training, validation, and testing datasets. The training set consists of weather data from 2014 to 2021, the validation set includes data from the year 2022, and the testing set comprises data from the year 2023. The division of the dataset follows an 8:1:1 ratio, allowing the model to be trained on sufficient historical data, validated on a separate year, and tested on the most recent data for accurate evaluation.

Baselines. As discussed in Section2.1, many time-series forecasting methods can be modified to fit this task. We here compare 9 baselines. 1) Temporal-only methods: The transformer-based models won the most popularity in the past three years, some methods between 2021 and 2024, like Informer (2021)[34], Autoformer (2021)[13], Pyraformer (2021)[35], FEDformer (2022)[15], PatchTST (2023)[37], iTransformer (2024)[16] are selected to represent the state-of-the-art long-term forecasting methods. Apart from them, we also adopt the efficient MLP-based DLinear (2023)[40] and the new trending architecture Mamba (2023)[41] as baselines. 2) Spatial-temporal method: Temporal-only deep models that do not consider the spatial correlations, we, therefore, survey the Corrformer (2023)[12] as a baseline model that considers the dynamic characteristics of correlations among weather stations at different locations.

Task settings. To facilitate fair comparison between different baselines, we align the input length of all baselines. The final setting is to predict the $\tau$-step future based on 48 historical steps. Specifically, we predict the future weather conditions for lead times of 1, 3, 5, and 7 days, corresponding to predicting the 24-step, 72-step, 120-step, and 168-step future data, respectively. Note the report results are only performed once instead of multiple times in original implementations. This does not affect the comparison as we observe the results are stable for different seeds due to the large dataset volume.

Implementation details. We develop and implement the baselines based on the Time-Series-Library.²²2https://github.com/thuml/Time-Series-Library To ensure a fair comparison, we made no changes to the intermediate network structures of all baseline models. Instead, we make corresponding adjustments to the inputs and outputs to adapt them to the WEATHER-5K dataset. Additionally, all optimization parameters are kept the same. For instance, the training is performed for a total of 300,000 iterations, starting with a learning rate of 1e-4. We employ the cosine decay strategy and gradually decay the learning rate to 0 by the end of training. The batch size for all models is set to 1,024 except for Correformer. During the validation phase, an early stopping is executed if training loss does not decrease for three consecutive times. The checkpoint with the lowest validation loss encountered prior to the early stop is saved and used for testing. Experiments are conducted on an 224 Intel(R) Xeon(R) Platinum 8480CL CPUs @ 3.80 GHz, 2.0 TB RAM computing server, equipped with 8 NVIDIA H800 GPUs.

4.4 Quantitative Comparison

Table2 reports the MAE and MSE for five variables under different predictive lengths. Some findings are concluded as follows.The simple linear implementation DLinear[40] shows a relatively poor performance while the early method, like Pyraformer[35], shows a significant advantage among all baselines. Overall, the remaining baselines show similar performance. There could be several reasons for this. Firstly, the existing time-series forecasting models have relatively small parameter settings and computational capacity, which can achieve good fitting performance on small-scale datasets but may not be suitable for large-scale datasets. Additionally, these models, except Corrformer[12], only consider temporal dependencies and overlook the spatial distribution differences and correlations in meteorological data. Furthermore, we observe that Corrformer, despite considering spatial relationships, does not perform as well as some simpler models like Informer[34]. One possible reason for this may be that Corrformer’s spatial modeling relies on pre-defined local regions. Moreover, we noticed that different models exhibit preferences for some variables. For example, Informer[34] performs better in predicting wind speed and direction than other methods. In addition, some models show preferences for long-term forecasting, such as the Mamba[41] model, which achieves the best performance in long-term predictions even though it has a general short-term performance. This suggests that the Mamba structure may be more suitable for long-term forecasting. Finally, based on the current results, the cumulative error stabilizes and increases significantly after the third day. This indicates the predictive errors are large after three days, and there is still ample room for improvement in long-term prediction performance.

Table3 presents a report on the predictive performance of various models for extreme values. Our finding is that the current time-series forecasting models studied in this paper can not effectively capture the extreme values, especially with the lower and upper quantiles at $0.05\%$ and $99.5\%$. Additionally, it is observed that the performance of wind prediction is the worst among all variables. This can be attributed to the non-stationary nature of the wind distribution, which makes it extremely challenging to predict accurately. The evaluation results also indicate a research direction for future time-series forecasting, which is to pay more attention to extreme values.

4.5 Ablation Study

The ablation experiments are conducted to explore the impact of different input lengths on predictions. Here, we utilized the iTransformer[16] model for verification. Specifically, we set the input length to $24,48,72,96$, and $120$, while keeping the predicted length constant at 72. The experimental results are shown in Figure3, illustrating the performance variation of MAE for four weather variables as the input sequence length increases. The results demonstrate that the performance for some variables (temperature and wind) is slightly improved when increasing the input length. On the other hand, the MAE error for sea level pressure rises slightly. We ultimately set the input length as 48 to balance computation and performance.

A.1 Motivation

For what purpose was the dataset created? Was there a specific task in mind? Was there aspecific gap that needed to be filled? Please provide a description.

This dataset is created with the following motivations: 1)The current weather station dataset limits the applicability of forecasting models in real-world scenarios. Hence, it is urgent to develop a comprehensive global station weather dataset that enables forecasting models to generalize across diverse stations and regions worldwide. 2) The limited sizes of existing time-series datasets may not reflect the real performance of the forecasting models, the proposed large weather station dataset can also serve as an extensive time-series dataset to perform comprehensive time-series forecasting benchmarks for various forecasting methods. 3) The existing simple datasets fail to encompass the complex scientific problems that researchers need to discover and resolve, thereby hindering progress in the field of time-series prediction. the proposed dataset offers a diverse temporal and spatial range of time-series data, enabling a comprehensive evaluation of time-series forecasting methods and driving significant advancements in the field. 4) A large-scale weather station dataset is a crucial source of observational data for numerical weather prediction models, effectively bridging the gap between numerical models and station-based predictions. This not only improves the accuracy of numerical forecasts but also plays a vital role in verifying and evaluating the predictive performance of numerical weather prediction models.

Who created the dataset (e.g., which team, research group) and on behalf of which entity (e.g.,company, institution, organization)?

This dataset was created by Tao Han, Song Guo, Zhenghao Chen, Wanghan Xu, and Lei Bai. The authors are researchersaffiliated with the Hong Kong University of Science and Technology, Shanghai AI Laboratory, and the University of Sydney.

A.2 Composition

What do the instances that comprise the dataset represent (e.g., documents, photos, people,countries)? Are there multiple types of instances (e.g., movies, users, and ratings; people andinteractions between them; nodes and edges)? Please provide a description.

WEATHER-5K consists of 5,672 CSV files. Each CSV file represents data from a single weather station, with hourly observations recorded from 2014 to 2023. The dataset represents a collection of weather observation data, where each instance corresponds to an hourly observation from a specific weather station, with various meteorological measurements and auxiliary information.

How many instances are there in total (of each type, if appropriate)?

WEATHER-5K has a total number of 5,762 stations with a 10-year time coverage and includes 8 mandatory variables and 2 auxiliary features. For each sensor. It also possesses 87,648 instances.

Does the dataset contain all possible instances or is it a sample (not necessarily random) ofinstances from a larger set? If the dataset is a sample, then what is the larger set? Is the samplerepresentative of the larger set (e.g., geographic coverage)? If so, please describe how thisrepresentativeness was validated/verified.

Our dataset is collected and further processed using data sourced from the National Centers for Environmental Information (NCEI), specifically the Integrated Surface Database (ISD),³³3https://www.ncei.noaa.gov/products/land-based-station/integrated-surface-database. Although the ISD contains records from over $20,000$ stations spanning several decades, not all stations are suitable for machine learning applications.For instance, some stations are no longer operational, many do not report data on an hourly basis, and numerous stations have missing values for critical weather elements.To get a high-quality weather station dataset, a meticulous selection process was conducted to include only long-term, hourly reporting stations that are currently operational and provide essential observations such as temperature, dew point temperature, wind, and sea level pressure. After that, we use the process procedure detailed in Section 3 to make the final WEATHER-5K, which is in the principle of applicability for time-series forecasting research.

What data does each instance consist of? “Raw” data (e.g., unprocessed text or images) or features? In either case, please provide a description.

The key characteristics of each instance are:

Is there a label or target associated with each instance? If so, please provide a description.

No, weather observation data can take itself as label in the forecasting task, and weather forecasting can be considered as a self-supervised learning task.

Is any information missing from individual instances? If so, please provide a description, explaining why this information is missing (e.g., because it was unavailable).

No, many efforts have been made to ensure there is no missing data in the WEATHER-5K dataset.

Are relationships between individual instances made explicit (e.g., users’ movie ratings, social network links)? If so, please describe how these relationships are made explicit.

Yes, the weather stations in the dataset have geographical relationships, and we have used latitude, longitude, and elevation to represent their geographic locations. This information can be leveraged in subsequent work to model the spatial relationships between the instances.

Are there recommended data splits (e.g., training, development/validation, testing)? If so, please provide a description of these splits, explaining the rationale behind them.

Yes, we chronologically split the data into train (2013-01-01 to 2021-12-31), validation (2022-01-01 to 2022-12-31), and test (2023-01-01 to 2023-12-31) sets, with a ratio of 8:1:1.

Are there any errors, sources of noise, or redundancies in the dataset? If so, please provide a description.

Yes, the errors and noise in the dataset arise from two main sources. Firstly, the use of meteorological automatic stations introduces a certain degree of observational error, particularly in the measurement of wind speed and direction, which are relatively difficult to measure accurately. Secondly, in our data processing efforts to ensure the completeness of the dataset, we have employed interpolation operations, which can introduce some additional error. However, the proportion of error introduced by the interpolation is relatively small.

Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)?

Yes, it is self-contained.

Does the dataset contain data that might be considered confidential (e.g., data that is protected by legal privilege or by doctor–patient confidentiality, data that includes the content of individuals’ non-public communications)? If so, please provide a description.

No, all our data are from a publicly available data source, i.e., NCEI.

Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety? If so, please describe why.

No, all our data are numerical.

A.3 Collection Process

How was the data associated with each instance acquired? Was the data directly observable (e.g., raw text, movie ratings), reported by subjects (e.g., survey responses), or indirectly inferred/derived from other data (e.g., part-of-speech tags, model-based guesses for age or language)?

We source the data from the the Integrated Surface Database (ISD) ogrinized and maintained by the National Centers for Environmental Information (NCEI). ISD is a global database that consists of hourly and synoptic surface observations compiled from numerous sources into a single common ASCII format and common data model. ISD integrates data from more than 100 original data sources.

What mechanisms or procedures were used to collect the data (e.g., hardware apparatuses or sensors, manual human curation, software programs, software APIs)? How were these mechanisms or procedures validated?

NCEI (formerly the National Climatic Data Center) started developing ISD in 1998 with assistance from partners in the U.S. Air Force and Navy, as well external funding from several sources. The database incorporates data from over 35,000 stations around the world, with some that include having, and includes observations data from as far back as 1901. The number of stations with data in ISD increased substantially in the 1940s and again in the early 1970s. There are currently more than 14,000 active ISD stations that are updated daily in the database. The total uncompressed data volume is around 600 gigabytes; however, it continues to grow as more data are added.

If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)?

The sampling strategy is deterministic.

Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)?

Our code collects publicly available data, which is free. On our side, we developed a download API to efficiently retrieve the source data, which was done by our team members.

Over what timeframe was the data collected? Does this timeframe match the creation timeframe of the data associated with the instances (e.g., recent crawl of old news articles)? If not, please describe the timeframe in which the data associated with the instances was created.

The WEATHER-5K dataset is collected and processed in 2024. This timeframe of the source data data is matches the creation timeframe of the data.

Were any ethical review processes conducted (e.g., by an institutional review board)?

No, such processes are unnecessary in our case.

A.4 Preprocessing/cleaning/labeling

Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)? If so, please provide a description.

Yes, to obtain a high-quality dataset of weather stations, a series of post-processing steps were performed on the raw weather station data collected from 2014 to 2024. Initially, $10,701$ commonly operating stations were identified. The first step involved selecting stations that reported data every hour on the hour. However, many stations did not meet this criterion. To address this, a replacement method estimated missing hourly data points using the nearest available time points within a 30-minute window, significantly improving the distribution of valid hourly data. Some following processing steps are described in Section 3.

Was the “raw” data saved in addition to the preprocessed/cleaned/labeled data (e.g., to support unanticipated future uses)? If so, please provide a link or other access point to the “raw” data.

Is the software that was used to preprocess/clean/label the data available? If so, please provide a link or other access point.

No.

A.5 Uses

Has the dataset been used for any tasks already? If so, please provide a description.

The dataset is used in this paper for the global station weather forecasting task.

Is there a repository that links to any or all papers or systems that use the dataset? If so, please provide a link or other access point.

No, but we may include a leader board and list papers using this dataset in the future.

What (other) tasks could the dataset be used for?

Weather data imputation, numerical weather prediction, and data assimilation

Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses?

We believe that our dataset will not encounter usage limit.

Are there tasks for which the dataset should not be used? If so, please provide a description.

No, users could use our dataset in any task as long as it does not violate laws.

A.6 Distribution

Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created? If so, please provide a description.

No, it will always be held on GitHub.

How will the dataset will be distributed (e.g., tarball on website, API, GitHub)? Does the dataset have a digital object identifier (DOI)?

The instructions for building WEATHER-5K are available at: https://github.com/taohan10200/WEATHER-5K.The dataset does not have a digital object identifier currently.

When will the dataset be distributed?

On June 07, 2024.

Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)? If so, please describe this license and/or ToU, and provide a link or other access point to.

Have any third parties imposed IP-based or other restrictions on the data associated with the instances? If so, please describe these restrictions, and provide a link or other access point to, or otherwise reproduce, any relevant licensing terms, as well as any fees associated with these restrictions.

Do any export controls or other regulatory restrictions apply to the dataset or to individual instances? If so, please describe these restrictions, and provide a link or other access point to, or otherwise reproduce, any supporting documentation.

No.

A.7 Maintenance

Who will be supporting/hosting/maintaining the dataset?

The authors of the paper.

How can the owner/curator/manager of the dataset be contacted (e.g., email address)?

Is there an erratum? If so, please provide a link or other access point.

Users can use GitHub to report issues or bugs.

Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete instances)? If so, please describe how often, by whom, and how updates will be communicated to dataset consumers (e.g., mailing list, GitHub)?

Yes, the authors will actively update the code and data on GitHub. Any updates of the dataset will be announced in our GitHub repository.

If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were the individuals in question told that their data would be retained for a fixed period of time and then deleted)? If so, please describe these limits and explain how they will be enforced.

The dataset does not relate to people.

Will older versions of the dataset continue to be supported/hosted/maintained? If so, please describe how. If not, please describe how its obsolescence will be communicated to dataset consumers.

Yes, we will provide the information on GitHub.

If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so? If so, please provide a description. Will these contributions be validated/verified? If so, please describe how. If not, why not? Is there a process for communicating/distributing these contributions to dataset consumers? If so, please provide a description.

Yes, we welcome users to submit pull requests on GitHub, and we will actively validate the requests.