Review of Different Error Metrics: A Case of Solar Forecasting
Main Article Content
Abstract
Renewable energy systems (RES) are no longer confined to being used as a stand-alone entity in the modern era. These RES, especially solar panels are also used with the grid power systems to supply electricity. However, precise forecasting of solar irradiance is necessary to ensure that the grid operates in a balanced and planned manner. Various solar forecasting models (SFM) are presented in the literature to produce an accurate solar forecast. Nevertheless, each model has gone through the step of evaluation of its accuracy using some error measures. Many error measures are discussed in the literature for deterministic as well as probabilistic solar forecasting. But, each study has its own selected error measure which sometimes landed on a wrong interpretation of results if not selected appropriately. As a result, this paper offers a critical assessment of several common error metrics with the goal of discussing alternative error metrics and establishing a viable set of error metrics for deterministic and probabilistic solar forecasting. Based on highly cited research from the last three years (2019-2021), error measures for both types of forecasting are presented with their basic functionalities, advantages & limitations which equipped the reader to pick the required compatible metrics
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
AJSE contents are under the terms of the Creative Commons Attribution License. This permits anyone to copy, distribute, transmit and adapt the worknon-commercially provided the original work and source is appropriately cited.
References
[2] R. Blaga, A. Sabadus, N. Stefu, C. Dughir, M. Paulescu, and V. Badescu, “A current perspective on the accuracy of incoming solar energy forecasting,” Prog. Energy Combust. Sci., vol. 70, pp. 119–144, Jan. 2019, doi: 10.1016/J.PECS.2018.10.003.
[3] A. H. Elsheikh, S. W. Sharshir, M. Abd Elaziz, A. E. Kabeel, W. Guilan, and Z. Haiou, “Modeling of solar energy systems using artificial neural network: A comprehensive review,” Sol. Energy, vol. 180, no. January, pp. 622–639, 2019, doi: 10.1016/j.solener.2019.01.037.
[4] M. N. Akhter, S. Mekhilef, H. Mokhlis, and N. M. Shah, “Review on forecasting of photovoltaic power generation based on machine learning and metaheuristic techniques,” IET Renew. Power Gener., vol. 13, no. 7, pp. 1009–1023, May 2019, doi: 10.1049/IET-RPG.2018.5649.
[5] R. Ahmed, V. Sreeram, Y. Mishra, and M. D. Arif, “A review and evaluation of the state-of-the-art in PV solar power forecasting: Techniques and optimization,” Renew. Sustain. Energy Rev., vol. 124, no. June 2019, p. 109792, 2020, doi: 10.1016/j.rser.2020.109792.
[6] P. Choudhary and R. K. Srivastava, “Sustainability perspectives- a review for solar photovoltaic trends and growth opportunities,” J. Clean. Prod., vol. 227, pp. 589–612, Aug. 2019, doi: 10.1016/J.JCLEPRO.2019.04.107.
[7] F. Antonanzas-Torres, R. Urraca, J. Polo, O. Perpiñán-Lamigueiro, and R. Escobar, “Clear sky solar irradiance models: A review of seventy models,” Renew. Sustain. Energy Rev., vol. 107, pp. 374–387, Jun. 2019, doi: 10.1016/J.RSER.2019.02.032.
[8] A. Mellit, A. M. Pavan, E. Ogliari, S. Leva, and V. Lughi, “Advanced methods for photovoltaic output power forecasting: A review,” Appl. Sci., vol. 10, no. 2, p. 487, 2020, doi: 10.3390/app10020487.
[9] M. Guermoui, F. Melgani, K. Gairaa, and M. L. Mekhalfi, “A comprehensive review of hybrid models for solar radiation forecasting,” J. Clean. Prod., vol. 258, p. 120357, 2020, doi: 10.1016/j.jclepro.2020.120357.
[10] A. R. Pazikadin, D. Rifai, K. Ali, M. Z. Malik, A. N. Abdalla, and M. A. Faraj, “Solar irradiance measurement instrumentation and power solar generation forecasting based on Artificial Neural Networks (ANN): A review of five years research trend,” Sci. Total Environ., vol. 715, p. 136848, May 2020, doi: 10.1016/J.SCITOTENV.2020.136848.
[11] P. Singla, M. Duhan, and S. Saroha, “A comprehensive review and analysis of solar forecasting techniques,” Front. Energy, pp. 1–37, Mar. 2021, doi: 10.1007/s11708-021-0722-7.
[12] M. J. Mayer and G. Gróf, “Extensive comparison of physical models for photovoltaic power forecasting,” Appl. Energy, vol. 283, p. 116239, Feb. 2021, doi: 10.1016/j.apenergy.2020.116239.
[13] P. Kumari and D. Toshniwal, “Extreme gradient boosting and deep neural network based ensemble learning approach to forecast hourly solar irradiance,” J. Clean. Prod., vol. 279, p. 123285, Jan. 2021, doi: 10.1016/j.jclepro.2020.123285.
[14] M. Guermoui, K. Gairaa, J. Boland, and T. Arrif, “A Novel Hybrid Model for Solar Radiation Forecasting Using Support Vector Machine and Bee Colony Optimization Algorithm: Review and Case Study,” J. Sol. Energy Eng., vol. 143, no. 2, Apr. 2021, doi: 10.1115/1.4047852.
[15] S. Das, “Short term forecasting of solar radiation and power output of 89.6kWp solar PV power plant,” Mater. Today Proc., vol. 39, pp. 1959–1969, Jan. 2021, doi: 10.1016/J.MATPR.2020.08.449.
[16] E. Jumin, F. B. Basaruddin, Y. B. M. Yusoff, S. D. Latif, and A. N. Ahmed, “Solar radiation prediction using boosted decision tree regression model: A case study in Malaysia,” Environ. Sci. Pollut. Res. 2021 2821, vol. 28, no. 21, pp. 26571–26583, Jan. 2021, doi: 10.1007/S11356-021-12435-6.
[17] P. Kumari and D. Toshniwal, “Long short term memory–convolutional neural network based deep hybrid approach for solar irradiance forecasting,” Appl. Energy, vol. 295, p. 117061, Aug. 2021, doi: 10.1016/J.APENERGY.2021.117061.
[18] G. Narvaez, L. F. Giraldo, M. Bressan, and A. Pantoja, “Machine learning for site-adaptation and solar radiation forecasting,” Renew. Energy, vol. 167, pp. 333–342, Apr. 2021, doi: 10.1016/J.RENENE.2020.11.089.
[19] T. Peng, C. Zhang, J. Zhou, and M. S. Nazir, “An integrated framework of Bi-directional long-short term memory (BiLSTM) based on sine cosine algorithm for hourly solar radiation forecasting,” Energy, vol. 221, p. 119887, Apr. 2021, doi: 10.1016/j.energy.2021.119887.
[20] B. K. Puah et al., “A regression unsupervised incremental learning algorithm for solar irradiance prediction,” Renew. Energy, vol. 164, pp. 908–925, Feb. 2021, doi: 10.1016/J.RENENE.2020.09.080.
[21] M. O. Moreira, P. P. Balestrassi, A. P. Paiva, P. F. Ribeiro, and B. D. Bonatto, “Design of experiments using artificial neural network ensemble for photovoltaic generation forecasting,” Renew. Sustain. Energy Rev., vol. 135, p. 110450, Jan. 2021, doi: 10.1016/J.RSER.2020.110450.
[22] P. Li, K. Zhou, X. Lu, and S. Yang, “A hybrid deep learning model for short-term PV power forecasting,” Appl. Energy, vol. 259, p. 114216, Feb. 2020, doi: 10.1016/j.apenergy.2019.114216.
[23] H. Zang, L. Cheng, T. Ding, K. W. Cheung, Z. Wei, and G. Sun, “Day-ahead photovoltaic power forecasting approach based on deep convolutional neural networks and meta learning,” Int. J. Electr. Power Energy Syst., vol. 118, p. 105790, Jun. 2020, doi: 10.1016/j.ijepes.2019.105790.
[24] H. Sharadga, S. Hajimirza, and R. S. Balog, “Time series forecasting of solar power generation for large-scale photovoltaic plants,” Renew. Energy, vol. 150, pp. 797–807, May 2020, doi: 10.1016/j.renene.2019.12.131.
[25] F. Wang et al., “A minutely solar irradiance forecasting method based on real-time sky image-irradiance mapping model,” Energy Convers. Manag., vol. 220, p. 113075, Sep. 2020, doi: 10.1016/J.ENCONMAN.2020.113075.
[26] Y. Jung, J. Jung, B. Kim, and S. U. Han, “Long short-term memory recurrent neural network for modeling temporal patterns in long-term power forecasting for solar PV facilities: Case study of South Korea,” J. Clean. Prod., vol. 250, p. 119476, Mar. 2020, doi: 10.1016/J.JCLEPRO.2019.119476.
[27] J. Lee, W. Wang, F. Harrou, and Y. Sun, “Reliable solar irradiance prediction using ensemble learning-based models: A comparative study,” Energy Convers. Manag., vol. 208, p. 112582, Mar. 2020, doi: 10.1016/J.ENCONMAN.2020.112582.
[28] M. AlKandari and I. Ahmad, “Solar power generation forecasting using ensemble approach based on deep learning and statistical methods,” Appl. Comput. Informatics, no. xxxx, 2019, doi: 10.1016/j.aci.2019.11.002.
[29] H. Zang, L. Liu, L. Sun, L. Cheng, Z. Wei, and G. Sun, “Short-term global horizontal irradiance forecasting based on a hybrid CNN-LSTM model with spatiotemporal correlations,” Renew. Energy, vol. 160, pp. 26–41, Nov. 2020, doi: 10.1016/j.renene.2020.05.150.
[30] Z. Pang, F. Niu, and Z. O’Neill, “Solar radiation prediction using recurrent neural network and artificial neural network: A case study with comparisons,” Renew. Energy, 2020, doi: 10.1016/j.renene.2020.04.042.
[31] S. Theocharides, G. Makrides, A. Livera, M. Theristis, P. Kaimakis, and G. E. Georghiou, “Day-ahead photovoltaic power production forecasting methodology based on machine learning and statistical post-processing,” Appl. Energy, vol. 268, p. 115023, Jun. 2020, doi: 10.1016/J.APENERGY.2020.115023.
[32] M. Abdel-Nasser and K. Mahmoud, “Accurate photovoltaic power forecasting models using deep LSTM-RNN,” Neural Comput. Appl., vol. 31, pp. 2727–2740, 2019, doi: https://doi.org/10.1007/s00521-017-3225-z.
[33] L. Benali, G. Notton, A. Fouilloy, C. Voyant, and R. Dizene, “Solar radiation forecasting using artificial neural network and random forest methods: Application to normal beam, horizontal diffuse and global components,” Renew. Energy, vol. 132, pp. 871–884, Mar. 2019, doi: 10.1016/j.renene.2018.08.044.
[34] K. Wang, X. Qi, and H. Liu, “A comparison of day-ahead photovoltaic power forecasting models based on deep learning neural network,” Appl. Energy, vol. 251, Oct. 2019, doi: 10.1016/j.apenergy.2019.113315.
[35] H. Zhou, Y. Zhang, L. Yang, Q. Liu, K. Yan, and Y. Du, “Short-Term photovoltaic power forecasting based on long short term memory neural network and attention mechanism,” IEEE Access, vol. 7, pp. 78063–78074, 2019, doi: 10.1109/ACCESS.2019.2923006.
[36] S. Ghimire, R. C. Deo, N. Raj, and J. Mi, “Deep solar radiation forecasting with convolutional neural network and long short-term memory network algorithms,” Appl. Energy, vol. 253, p. 113541, Nov. 2019, doi: 10.1016/j.apenergy.2019.113541.
[37] G. M. Yagli, D. Yang, and D. Srinivasan, “Automatic hourly solar forecasting using machine learning models,” Renew. Sustain. Energy Rev., vol. 105, no. October 2018, pp. 487–498, 2019, doi: 10.1016/j.rser.2019.02.006.
[38] M. Alsharif, M. Younes, and J. Kim, “Time Series ARIMA Model for Prediction of Daily and Monthly Average Global Solar Radiation: The Case Study of Seoul, South Korea,” Symmetry (Basel)., vol. 11, no. 2, p. 240, Feb. 2019, doi: 10.3390/sym11020240.
[39] S. Ghimire, R. C. Deo, N. J. Downs, and N. Raj, “Global solar radiation prediction by ANN integrated with European Centre for medium range weather forecast fields in solar rich cities of Queensland Australia,” J. Clean. Prod., vol. 216, pp. 288–310, Apr. 2019, doi: 10.1016/j.jclepro.2019.01.158.
[40] K. Wang, X. Qi, and H. Liu, “Photovoltaic power forecasting based LSTM-Convolutional Network,” Energy, vol. 189, p. 116225, Dec. 2019, doi: 10.1016/J.ENERGY.2019.116225.
[41] A. Nespoli et al., “Day-Ahead Photovoltaic Forecasting: A Comparison of the Most Effective Techniques,” Energies, vol. 12, no. 9, p. 1621, Apr. 2019, doi: 10.3390/en12091621.
[42] G. Liu et al., “Probabilistic spatiotemporal solar irradiation forecasting using deep ensembles convolutional shared weight long short-term memory network,” Appl. Energy, vol. 300, p. 117379, Oct. 2021, doi: 10.1016/J.APENERGY.2021.117379.
[43] J. Le Gal La Salle, M. David, and P. Lauret, “A new climatology reference model to benchmark probabilistic solar forecasts,” Sol. Energy, vol. 223, pp. 398–414, Jul. 2021, doi: 10.1016/J.SOLENER.2021.05.037.
[44] B. Schulz, M. El Ayari, S. Lerch, and S. Baran, “Post-processing numerical weather prediction ensembles for probabilistic solar irradiance forecasting,” Sol. Energy, vol. 220, pp. 1016–1031, May 2021, doi: 10.1016/J.SOLENER.2021.03.023.
[45] K. Doubleday, S. Jascourt, W. Kleiber, and B. M. Hodge, “Probabilistic solar power forecasting using bayesian model averaging,” IEEE Trans. Sustain. Energy, vol. 12, no. 1, pp. 325–337, Jan. 2021, doi: 10.1109/TSTE.2020.2993524.
[46] F. Najibi, D. Apostolopoulou, and E. Alonso, “Enhanced performance Gaussian process regression for probabilistic short-term solar output forecast,” Int. J. Electr. Power Energy Syst., vol. 130, p. 106916, Sep. 2021, doi: 10.1016/J.IJEPES.2021.106916.
[47] X. Zhang, F. Fang, and J. Wang, “Probabilistic Solar Irradiation Forecasting Based on Variational Bayesian Inference with Secure Federated Learning,” IEEE Trans. Ind. Informatics, vol. 17, no. 11, pp. 7849–7859, Nov. 2021, doi: 10.1109/TII.2020.3035807.
[48] Z. Zhang et al., “Solar Radiation Intensity Probabilistic Forecasting Based on K-Means Time Series Clustering and Gaussian Process Regression,” IEEE Access, vol. 9, pp. 89079–89092, 2021, doi: 10.1109/ACCESS.2021.3077475.
[49] G. M. Yagli, D. Yang, and D. Srinivasan, “Reconciling solar forecasts: Probabilistic forecasting with homoscedastic Gaussian errors on a geographical hierarchy,” Sol. Energy, vol. 210, pp. 59–67, Nov. 2020, doi: 10.1016/J.SOLENER.2020.06.005.
[50] M. Khodayar, S. Mohammadi, M. E. Khodayar, J. Wang, and G. Liu, “Convolutional graph autoencoder: A generative deep neural network for probabilistic spatio-temporal solar irradiance forecasting,” IEEE Trans. Sustain. Energy, vol. 11, no. 2, pp. 571–583, Apr. 2020, doi: 10.1109/TSTE.2019.2897688.
[51] C. Huang, Z. Zhao, L. Wang, Z. Zhang, and X. Luo, “Point and interval forecasting of solar irradiance with an active Gaussian process,” IET Renew. Power Gener., vol. 14, no. 6, pp. 1020–1030, Apr. 2020, doi: 10.1049/IET-RPG.2019.0769.
[52] Q. Huang and S. Wei, “Improved quantile convolutional neural network with two-stage training for daily-ahead probabilistic forecasting of photovoltaic power,” Energy Convers. Manag., vol. 220, p. 113085, Sep. 2020, doi: 10.1016/J.ENCONMAN.2020.113085.
[53] F. von Loeper, P. Schaumann, M. de Langlard, R. Hess, R. Bäsmann, and V. Schmidt, “Probabilistic prediction of solar power supply to distribution networks, using forecasts of global horizontal irradiation,” Sol. Energy, vol. 203, pp. 145–156, Jun. 2020, doi: 10.1016/J.SOLENER.2020.04.001.
[54] D. van der Meer, D. Yang, J. Widén, and J. Munkhammar, “Clear-sky index space-time trajectories from probabilistic solar forecasts: Comparing promising copulas,” J. Renew. Sustain. Energy, vol. 12, no. 2, p. 026102, Apr. 2020, doi: 10.1063/1.5140604.
[55] R. Alonso-Suárez, M. David, V. Branco, and P. Lauret, “Intra-day solar probabilistic forecasts including local short-term variability and satellite information,” Renew. Energy, vol. 158, pp. 554–573, Oct. 2020, doi: 10.1016/J.RENENE.2020.05.046.
[56] H. Quan and D. Yang, “Probabilistic solar irradiance transposition models,” Renew. Sustain. Energy Rev., vol. 125, p. 109814, Jun. 2020, doi: 10.1016/J.RSER.2020.109814.
[57] G. M. Yagli, D. Yang, and D. Srinivasan, “Ensemble solar forecasting using data-driven models with probabilistic post-processing through GAMLSS,” Sol. Energy, vol. 208, pp. 612–622, Sep. 2020, doi: 10.1016/J.SOLENER.2020.07.040.
[58] H. Panamtash, Q. Zhou, T. Hong, Z. Qu, and K. O. Davis, “A copula-based Bayesian method for probabilistic solar power forecasting,” Sol. Energy, vol. 196, pp. 336–345, Jan. 2020, doi: 10.1016/J.SOLENER.2019.11.079.
[59] M. Sun, C. Feng, and J. Zhang, “Probabilistic solar power forecasting based on weather scenario generation,” Appl. Energy, vol. 266, p. 114823, May 2020, doi: 10.1016/J.APENERGY.2020.114823.
[60] K. Doubleday, V. Van Scyoc Hernandez, and B. M. Hodge, “Benchmark probabilistic solar forecasts: Characteristics and recommendations,” Sol. Energy, vol. 206, pp. 52–67, Aug. 2020, doi: 10.1016/J.SOLENER.2020.05.051.
[61] H. He, N. Lu, Y. Jie, B. Chen, and R. Jiao, “Probabilistic solar irradiance forecasting via a deep learning-based hybrid approach,” IEEJ Trans. Electr. Electron. Eng., vol. 15, no. 11, pp. 1604–1612, Nov. 2020, doi: 10.1002/TEE.23231.
[62] W. Liu and Y. Xu, “Randomised learning-based hybrid ensemble model for probabilistic forecasting of PV power generation,” IET Gener. Transm. Distrib., vol. 14, no. 24, pp. 5816–5822, Dec. 2020, doi: 10.1049/IET-GTD.2020.0625.
[63] J. Le Gal La Salle, J. Badosa, M. David, P. Pinson, and P. Lauret, “Added-value of ensemble prediction system on the quality of solar irradiance probabilistic forecasts,” Renew. Energy, vol. 162, pp. 1321–1339, Dec. 2020, doi: 10.1016/J.RENENE.2020.07.042.
[64] J. Munkhammar, D. van der Meer, and J. Widén, “Probabilistic forecasting of high-resolution clear-sky index time-series using a Markov-chain mixture distribution model,” Sol. Energy, vol. 184, pp. 688–695, May 2019, doi: 10.1016/j.solener.2019.04.014.
[65] B. Chen and J. Li, “Combined probabilistic forecasting method for photovoltaic power using an improved Markov chain,” IET Gener. Transm. Distrib., vol. 13, no. 19, pp. 4364–4373, Oct. 2019, doi: 10.1049/IET-GTD.2018.6860.
[66] K. Bakker, K. Whan, W. Knap, and M. Schmeits, “Comparison of statistical post-processing methods for probabilistic NWP forecasts of solar radiation,” Sol. Energy, vol. 191, pp. 138–150, Oct. 2019, doi: 10.1016/j.solener.2019.08.044.
[67] M. Abuella and B. Chowdhury, “Forecasting of solar power ramp events: A post-processing approach,” Renew. Energy, pp. 1380–1392, Apr. 2019, doi: 10.1016/j.renene.2018.09.005.
[68] C. A. Gueymard, “A review of validation methodologies and statistical performance indicators for modeled solar radiation data: Towards a better bankability of solar projects,” Renew. Sustain. Energy Rev., vol. 39, pp. 1024–1034, Nov. 2014, doi: 10.1016/J.RSER.2014.07.117.
[69] T. E. Hoff, R. Perez, J. Kleissl, D. Renne, and J. Stein, “Reporting of irradiance modeling relative prediction errors,” Prog. Photovoltaics Res. Appl., vol. 21, no. 7, pp. 1514–1519, Nov. 2013, doi: 10.1002/PIP.2225.