Large Language Model-Driven Structured Output
A Comprehensive Benchmark and Spatial Data Generation Framework
Bibliographic Data
| ID | 22034674 |
|---|---|
| Authors | Diya Li (0000-0002-3287-9385, Environmental Systems Research Institute (United States)), Yue Zhao (0000-0003-0636-7092, Environmental Systems Research Institute (United States)), Zhifang Wang (0000-0003-3382-3073, Environmental Systems Research Institute (United States)), Calvin Jung (Environmental Systems Research Institute (United States)), Zhe Zhang (0000-0003-3057-9490, Texas A&M University, corresponding author) |
| Year | 2024 |
| Volume | 13 |
| Issue | 11 |
| Pages | 405 |
| Publication date | 2024-11-10 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | ISPRS International Journal of Geo-Information (JOURNAL) |
| Journal identifiers | ISSN: 2220-9964 • E-ISSN: 2220-9964 |
| Publisher | MDPI AG (PUBLISHER • IT) |
| DOI | 10.3390/ijgi13110405 |
| OpenAlex | W4404243795 |
| Language | EN |
| Citations received | 2 |
| References cited | 20 |
Large language models (LLMs) have demonstrated remarkable capabilities in document processing, data analysis, and code generation. However, the generation of spatial information in a structured and unified format remains a challenge, limiting their integration into production environments. In this paper, we introduce a benchmark for generating structured and formatted spatial outputs from LLMs with a focus on enhancing spatial information generation. We present a multi-step workflow designed to improve the accuracy and efficiency of spatial data generation. The steps include generating spatial data (e.g., GeoJSON) and implementing a novel method for indexing R-tree structures. In addition, we explore and compare a series of methods commonly used by developers and researchers to enable LLMs to produce structured outputs, including fine-tuning, prompt engineering, and retrieval-augmented generation (RAG). We propose new metrics and datasets along with a new method for evaluating the quality and consistency of these outputs. Our findings offer valuable insights into the strengths and limitations of each approach, guiding practitioners in selecting the most suitable method for their specific use cases. This work advances the field of LLM-based structured spatial data output generation and supports the seamless integration of LLMs into real-world applications
Cartography · Data mining · Geography · Computer Science · Data Mining Algorithms and Applications · Data Quality and Management · Topic Modeling
| Citations per year | 2 |
|---|---|
| Citation span | 2025 - 2026 (2) |
| Highly cited | No |