|
|
Research on meteorological grid data processing based on spatiotemporal big data engine |
Yang Cai1, Guo Yuefan2, Li Wenzhao1, Li Jingrui3, Xue Shuaining1, Shi Yan1 |
1. Chongqing Meteorological Information and Technology Support Center,Chongqing 401147; 2. Sichuan Meteorological Observation and Data Center,Chengdu 610000; 3. Wuxi County Meteorological Bureau,Wuxi 405800 |
|
|
Abstract In order to effectively meet the demand for spatiotemporal services of massive meteorological grid data,precipitation elements from multi-source fused real-time grid data were selected as experimental data.Based on the spatiotemporal big data engine (GeoTrellis),a processing scheme for storing,retrieving,statistically analyzing,and visualizing massive meteorological grid data was proposed,and compared with traditional data storage schemes.Research has shown that this approach performs better than traditional grid data slicing storage methods,with time consumption reduced to seconds.The spatiotemporal retrieval of meteorological grid data can be shortened to milliseconds,which can support real-time statistics of massive meteorological grid data.The solution based on spatiotemporal big data engine can effectively support the efficient storage,retrieval,statistics and visualization of grid data,and can effectively enhance the spatiotemporal service capability of massive meteorological grid data.
|
Received: 28 April 2024
|
|
|
|
|
[1] |
崔杰,李陶深,兰红星.基于Hadoop的海量数据存储平台设计与开发[J].计算机研究与发展,2012,49 (增刊1):12-18.
|
[2] |
温馨,罗侃,陈荣国.基于Shark/Spark的分布式空间数据分析框架[J].地球信息科学学报,2015,17 (4):401-407.
|
[3] |
夏靖波,韦泽鲲,付凯,等.云计算中Hadoop技术研究与应用综述[J].计算机科学,2016,43 (11):6-11,48.
|
[4] |
曹布阳,冯华森,梁峻浩,等.利用Hilbert曲线与Cassandra技术实现时空大数据存储与索引[J].武汉大学学报(信息科学版),2021,46 (5):620-629.
|
[5] |
向虹锟,徐柱.基于GeoTrellis的海量栅格数据计算分布式系统建设[J].测绘与空间地理信息,2021,44 (8):44-47,51.
|
[6] |
张璐,潘旸,谷军霞,等.国际主流多源融合降水实况产品的研究进展与展望[J].气象科技进展,2022,12 (6):16-27.
|
[7] |
徐拥军,倪学磊,郑波,等.基于位置的天气实况数据服务接口设计与应用[J].计算机系统应用,2023,32 (5):77-86.
|
[8] |
杨英奎,王祝先,李伟金.数据湖在气象系统中的研究与应用[J].自动化技术与应用,2023,42 (7):91-94.
|
[9] |
雷鸣.气象大数据分布式存储设计与实现[J].计算机技术与发展,2021,31 (5):193-197.
|
[10] |
张春燕,郑艳萍,王沛东,等.多源融合实况分析1 km网格降水产品在广东省暴雨过程中的准确性评估[J].暴雨灾害,2023,42 (6):679-691.
|
[11] |
朱君,李奇临,赵美艳,等.多源融合实况分析产品在重庆地区的适用性分析[J].气象水文海洋仪器,2023,40 (3):38-41.
|
[12] |
张权,张志远,王燕,等.海量气象水文数据存储结构的研究与实现[J].气象水文海洋仪器,2010,27 (3):1-5.
|
[13] |
金国栋,卞昊穹,陈跃国,等.HDFS存储和优化技术研究综述[J].软件学报,2020,31 (1):137-161.
|
[14] |
胡俊,胡贤德,程家兴.基于Spark的大数据混合计算模型[J].计算机系统应用,2015,24 (4):214-218.
|
[15] |
吴信东,嵇圣硙.MapReduce与Spark用于大数据分析之比较[J].软件学报,2018,29 (6):1770-1791.
|
[16] |
张涵斐,黄忠红,孟永军.海量遥感影像的存储与快速调度显示方法[J].测绘与空间地理信息,2011,34 (3):36-39,43.
|
|
|
|