Novel Transformer-based deep neural network for the prediction of post-refracturing production from oil wells

Abstract

The accurate prediction of post-refracture production can be of great value in the selection of target wells for refracturing. Given that production from post-refracture wells yields time-series data, deep neural networks have been utilized for making these predictions. Conventional deep neural networks, including recurrent neural network and long shortterm memory neural network, often fail to effectively capture long-range dependencies, which is particularly evident in tasks such as forecasting oil well production over periods extending up to 36 years. To overcome this limitation, this paper presents a novel deep neural network based on Transformer architecture, meticulously designed by fine-tuning the key components of the architecture, including its dimensions, the number of encoder layers, attention heads, and iteration cycles. This Transformer-based model is deployed on a dataset from oil wells in the Junggar Basin that spans the period of 1983 to 2020. The results demonstrate that the Transformer significantly outperforms traditional models such as recurrent neural networks and long short-term memory, underscoring its enhanced ability to manage long-term dependencies within time-series data. Moreover, the predictive accuracy of Transformer was further validated with data from six newly refractured wells in the Junggar Basin, which underscored its effectiveness over both 90 and 180 days post-refracture. The effective application of the proposed Transformer-based time-series model affirms the feasibility of capturing long-term dependencies using Transformer-based encoders, which also allows for more accurate predictions compared to conventional deep learning techniques.

Document Type: Original article

Cited as: Jia, J., Li, D., Wang, L., Fan, Q. Novel Transformer-based deep neural network for the prediction of post-refracturing production from oil wells. Advances in Geo-Energy Research, 2024, 13(2): 119-131. https://doi.org/10.46690/ager.2024.08.06

References

Abdelaziz, A., Ha, J., Li, M., et al. Understanding hydraulic fracture mechanisms: Fromthe laboratory tonumerical modelling. Advances in Geo-Energy Research, 2023, 7(1):66-68.

Cheng, L., Xie, Y., Luo, Z., et al. Numerical analysis of 3D nonplanar hydraulic fracture propagation in fractured-vuggy formations using a hydromechanical coupled XFEMapproach. Computers and Geotechnics, 2024, 170:106267.

Davies, A., Cowliff, L., Simmons, M. D. A method for fine-scale vertical heterogeneity quantification from well data and its application to siliciclastic reservoirs of the UKCS. Marine and Petroleum Geology, 2023, 149:106077.

Esfandi, T., Sadeghnejad, S., Jafari, A. Effect of reservoir heterogeneity on well placement prediction in CO2-EOR projects using machine learning surrogate models: Benchmarking of boosting-based algorithms. Geoenergy Science and Engineering, 2024, 233:212564.

Faramarzi, N., Sadeghnejad, S. Fluid and rock heterogeneity assessment of gas condensate reservoirs by wavelet transform of pressure-transient responses. Journal of Natural Gas Science and Engineering, 2020, 81:103469.

Farhoodi, S., Sadeghnejad, S., Dehaghani, A. H. S. Simultaneous effect of geological heterogeneity and condensate blockage on well test response of gas condensate reservoirs. Journal of Natural Gas Science and Engineering, 2019, 66:192-206.

He, J., Okere, C. J., Su, G., et al. Formation damage mitigation mechanismfor coalbed methane wells via refracturing with fuzzy-ball fluid as temporary blocking agents. Journal of Natural Gas Science and Engineering, 2021, 90: 103956.

Hochreiter, S., Schmidhuber, J. Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780.

Huang, Y., Shen, L., Liu, H. Grey relational analysis, principal component analysis and forecasting of carbon emissions based on long short-term memory in China. Journal of Cleaner Production, 2019, 209: 415-423.

Jamshidi Gohari, M. S., Niri, M. E., Sadeghnejad, S., et al. Synthetic graphic well log generation using an enhanced deep learning workflow: Imbalanced multiclass data, sample size, and scalability challenges. SPE Journal, 2024, 29(1): 1-20.

Kakemem, U., Ghasemi, M., Adabi, M. H., et al. Sedimentology and sequence stratigraphy of automated hydraulic f low units-The Permian Upper Dalan Formation, Persian Gulf. Marine and Petroleum Geology, 2023, 147: 105965.

Kaur, J., Parmar, K. S., Singh, S. Autoregressive models in environmental forecasting time series: A theoretical and application review. Environmental Science and Pollution Research, 2023, 30(8): 19617-19641.

Liao, Q., Wang, B., Chen, X., et al. Reservoir stimulation for unconventional oil and gas resources: Recent advances and future perspectives. Advances in Geo-Energy Re search, 2024, 13(1): 7-9.

Li, Y., Zhao, Q., Lyu, Q., et al. Evaluation technology and practice of continental shale oil development in China. Petroleum Exploration and Development, 2022, 49(5): 1098-1109.

Lu, M., Su, Y., Zhan, S., et al. Modeling for reorientation and potential of enhanced oil recovery in refracturing. Advances in Geo-Energy Research, 2020, 4(1): 20-28.

Malki, M. L., Saberi, M. R., Kolawole, O., et al. Underlying mechanisms and controlling factors of carbonate reservoir characterization from rock physics perspective: A comprehensive review. Geoenergy Science and Engineer ing, 2023, 226: 211793.

McCausland, W. J., Miller, S., Pelletier, D. Simulation smoothing for state-space models: A computational efficiency analysis. Computational Statistics & Data Analysis, 2011, 55(1): 199-212.

Nie, Y., Nguyen, N. H., Sinthong, P., et al. A time series is worth 64 words: Longterm forecasting with transformers. ArXiv preprint Arxiv: 2211.14730, 2023.

Reeves, S. R., Bastian, P. A., Spivey, J. P., et al. Benchmarking of restimulation candidate selection techniques in layered, tight gas sand formations using reservoir simulation. Paper SPE 63096 Presented at the SPE Annual Technical Conference and Exhibition, Dallas, Texas, 1-4 October, 2000.

Rumelhart, D. E., Hinton, G. E., Williams, R. J. Learning representations by back-propagating errors. Nature, 1986, 323(6088): 533-536.

Shabani, F., Amini, A., Tavakoli, V., et al. 3D basin and petroleum system modelling of the early cretaceous play in the NW Persian Gulf. Geoenergy Science and Engineering, 2023, 226: 211768.

Vaswani, A., Shazeer, N., Parmar, N., et al. Attention is all you need. Paper 3058 Presented at the Thirty-first Annual Conference on Neural Information Processing Systems, Long Beach, California, 4-9 December, 2017.

Wang, C., Chen, Y., Zhang, S., et al. Stock market index prediction using deep Transformer model. Expert Systems with Applications, 2022, 208: 118128.

Wang, Z., Liang, W., Lian, H., et al. Numerical study of multiple hydraulic fractures propagation in poroelastic media based on energy decomposition phase field methods. Computers and Geotechnics, 2024, 170: 106259.

Wu, D., Xu, H., Jiang, Z., et al. EdgeLSTM: Towards deep and sequential edge computing for IoT applications. IEEE/ACM Transactions on Networking, 2021, 29(4): 1895-1908.

Yu, Y., Zhu, W., Li, L., et al. Multi-fracture interactions during two-phase flow of oil and water in deformable tight sandstone oil reservoirs. Journal of Rock Mechanics and Geotechnical Engineering, 2020, 12(4): 821-849.

Zeng, A., Chen, M., Zhang, L., et al. Are transformers effective for time series forecasting? Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(9): 11121-11128.