面向边缘端的地震波大模型结构化剪枝方法

A structured pruning method for edge-oriented seismic wave large models

  • 摘要: 随着地震观测台网持续扩展和连续波形数据快速积累,地震智能监测对模型实时推理能力和边缘端部署效率提出了更高要求。近年来,亿级参数规模的DiTing地震波大模型在地震事件检测和震相拾取等任务中表现出较强的特征表达能力,但其参数规模大、计算开销高,难以直接部署野外监测终端和国产边缘计算设备等资源受限场景。针对上述问题,本文面向国产 RK3588 边缘计算平台,提出一种无需剪枝后微调的DiTing地震波大模型多层级结构化剪枝方法。该方法以编码器为主要压缩对象,从注意力头、前馈神经网络通道和编码层三个层级分析结构冗余,综合损失敏感性、遮蔽验证结果、激活响应和权重范数等信息评估结构重要性,据此执行结构化剪枝并在RK3588平台完成部署验证。实验结果表明,剪枝后模型的参数量和FLOPs分别减少54.12%和47.87%。在DiTingV3测试集上,剪枝模型的P波和S波F1分数分别为0.921 5和0.858 0,较原始模型分别下降1.58%和2.15%。在RK3588平台上,剪枝模型的平均推理延迟降低21.62%,RKNN模型文件大小减少49.75%,峰值内存占用下降约25.80%。结果表明,该方法在显著降低了DiTing地震波大模型的计算和存储开销的同时,较好的保持了震相拾取性能使RK3588平台的推理效率得到进一步提升,为地震智能监测模型在国产边缘设备和台站端的轻量化、实时化部署提供了可行方案。

     

    Abstract:
    With the continued expansion of seismic observation networks and the rapid accumulation of continuous waveform data, intelligent seismic monitoring has placed increasingly stringent demands on real-time model inference and efficient edge deployment. In recent years, the DiTing seismic foundation model, with approximately one hundred million parameters, has demonstrated strong feature-representation capabilities in tasks such as earthquake detection and seismic phase picking. However, its large parameter count and high computational cost make direct deployment difficult in resource-constrained environments, including field monitoring terminals and domestically produced edge-computing devices. To address this problem, this study proposes a multi-level structured pruning method for the DiTing seismic foundation model that requires no post-pruning fine-tuning and is designed for deployment on the domestically developed RK3588 edge-computing platform.
    The proposed method identifies the encoder as the primary compression target and performs importance evaluation and structured pruning at three structural levels: attention heads, feed-forward network (FFN) channels, and encoder layers. At the attention-head level, first-order Taylor importance, changes in P- and S-wave F1 scores after masking individual heads, and attention-distribution entropy are integrated to evaluate each head in terms of task-loss sensitivity, actual influence on task performance, and attention-distribution characteristics. At the FFN-channel level, first-order Taylor importance is combined with activation magnitudes under actual seismic waveform inputs and weight norms to jointly characterize task-loss sensitivity, data-driven response intensity, and parameter-connection strength. At the encoder-layer level, the relative change between the layer input and output is combined with the mean importance of the attention heads within that layer, thereby evaluating each layer from the perspectives of overall feature-transformation capability and internal attention structure. The different importance indicators are normalized and fused through weighted summation to rank the candidate structures, and masking experiments are then used to determine appropriate pruning ratios. Based on this analysis, the number of attention heads in each encoder layer is reduced from eight to four, the FFN hidden dimension is reduced from 2048 to 1024, and the encoder depth is reduced from 24 to 16 layers, jointly compressing both the width and depth of the model.
    The effects of structured pruning on model complexity and phase-picking performance are first evaluated on a GPU platform. The parameter count of the final pruned model decreases from 124.20 M to 56.89 M, corresponding to a reduction of 54.12%; its computational cost decreases from 59.03 GFLOPs to 30.77 GFLOPs, a reduction of 47.87%; and its weight-file size decreases from 475.50 MiB to 217.68 MiB, a reduction of 54.22%. Fixed-window evaluation is conducted on the complete DiTingV3 test set containing 78610 natural-earthquake records. The original model achieves P- and S-wave F1 scores of 0.937 3 and 0.879 6, respectively, whereas the final pruned model achieves corresponding F1 scores of 0.921 5 and 0.858 0. Thus, after more than half of the model parameters are removed, the P- and S-wave F1 scores decrease by only 1.58% and 2.15%, respectively. These results demonstrate that, without post-pruning fine-tuning, the proposed method reduces the parameter count by more than half and the computational cost by nearly half while preserving most of the phase-picking capability of the original DiTing model.
    The original and pruned models are further converted to the RKNN format and deployed on the RK3588 platform. In the fixed-window evaluation on the complete DiTingV3 test set, the original model achieves P- and S-wave F1 scores of 0.936 9 and 0.879 6, respectively, while the final pruned model achieves corresponding scores of 0.920 0 and 0.856 2. These results are close to the GPU results of 0.921 5 and 0.858 0, indicating that model conversion and FP16 inference on the RK3588 platform introduce only minor additional performance degradation. In an inference-speed experiment using 5000 randomly selected long waveforms, the mean forward inference time per window decreases from 344.75 ms to 270.21 ms, a reduction of 74.54 ms or 21.62%, corresponding to a speedup of approximately 1.276 times. In terms of resource consumption, the RKNN model size decreases from 271.44 MiB to 136.41 MiB, a reduction of 49.75%, while the peak resident set size during sustained inference decreases from 535.40 MiB to 397.27 MiB, a reduction of 25.80%. These results demonstrate that the reductions in parameter count and FLOPs achieved by the proposed method can be translated into measurable inference acceleration, storage savings, and runtime memory reduction on the RK3588 platform.
    In summary, the proposed multi-level structured pruning method jointly compresses attention heads, FFN channels, and encoder layers, thereby reducing both the width and depth of the DiTing seismic foundation model without requiring post-pruning fine-tuning. The method substantially reduces the parameter count, FLOPs, storage requirements, runtime memory consumption, and inference latency on the RK3588 platform while largely preserving P- and S-wave phase-picking performance on the complete DiTingV3 test set.

     

/

返回文章
返回