Using an attention-based architecture to incorporate context similarity into spatial non-stationarity estimation

Abstract

Geographically weighted regression (GWR) facilitates spatial modeling by providing location-specific coefficients to capture spatial non-stationarity. GWR incorporates a distance decay effect, assigning greater weights to proximal observations under the assumption they exert more influence on the regression parameters. However, distant observations may share significant context similarities, such as socioeconomic or environmental factors, which can influence the regression model. This study introduces an attention-based architecture to address context similarity between samples. A deep learning model termed Context-Attention Geographically Weighted Regression (CatGWR) is proposed to integrate context similarity with distance-based proximity to enhance the estimation of spatial non-stationarity in spatial regression models. CatGWR Such an integration results in contextualized spatial weights for CatGWR to identify the varying patterns of nonstationary relationships across different spatial locations and context conditions. Validation through simulation experiments and an empirical study on housing prices in Shenzhen, China, shows the superior predictive accuracy and robustness of CatGWR in modeling complex spatial interactions, especially under contextual influences, in which CatGWR improves the R2 of fit and prediction results by at least 6% compared to existing models. Future work will focus on optimizing bandwidth selection and exploring additional attention mechanisms to enhance model performance.