Model predictive control of super-mild hybrid electric vehicle based on Markov chain and Q-Learning

doi:10.3969/j.issn.1674-8484.2021.04.015

Abstract

Abstract:

A model predictive control energy management strategy for super-light hybrid electric vehicles (HEV) was proposed to take into account the global optimality of the energy management strategy and the real-time operation at the same time based on Markov chain and Q-Learning algorithm. The multi-step Markov model was used to predict the acceleration change process to calculate the future required power of HEV. An energy management strategy optimization model was established by taking the minimum equivalent fuel consumption and the local balance of the state of charge (SOC) of power battery as the objective function. The Q-learning algorithm was used to solve the optimization problem in the prediction time domain to obtain the optimal torque distribution sequence. The simulation analysis was carried out under the ECE_EUDC+UDDS cycle conditions on the base of MATLAB / Simulink platform. The results show that the control strategy solved by the Q-Learning solution reduces the simulation time by 4 s under the same fuel economy condition, comparing with the control strategy based on the dynamic programming (DP) solution. The proposed control strategy can significantly improve the operating efficiency and has better real-time performance.

Key words: super-light hybrid electric vehicles, model predictive control, Markov chain, Q-learning algorithm, multi-step Markov model, energy management

CLC Number:

U469.72

YIN Yanli, MA Yongjuan, ZHOU Yawei, WANG Ruixin, ZHAN Sen, MA Shenpeng, HUANG Xuejiang, ZHANG Xinxin. Model predictive control of super-mild hybrid electric vehicle based on Markov chain and Q-Learning[J]. Journal of Automotive Safety and Energy, 2021, 12(4): 557-569.

Figures/Tables 14

References 35

[1]	Kim H, Kum D. Comprehensive design methodology of input and output-split hybrid electric vehicles: In search of optimal configuration[J], *IEEE/ASME Trans Mechatronic*, 2016, 21:2912-2923. doi: 10.1109/TMECH.2016.2579646 URL
[2]	Kang J. Enabling localized peer-to-peer electricity trading among plug-in hybrid electric vehicles using consortium blockchains[J]. *IEEE Trans Ind Info*, 2017, 13:245-346.
[3]	苏岭, 曾育平, 秦大同. 插电式混合动力汽车能量管理策略研究现状和发展趋势[J]. 重庆大学学报, 2017, 40(2):10-15.
	SU Ling, ZENG Yuping, QING Datong. Research status and development trends in plug-in hybrid automobile energy management strategies[J]. J Chongqing Univ, 2017, 40(2):10-15. (in Chinese)
[4]	周能辉, 赵春明, 辛明华, 等. 插电式混合动力轿车整车控制策略的研究[J]. 汽车工程, 2013, 35(2):99-104.
	ZHOU Nenghui, ZHAO Chunming, XIN Minghua, et al. Research on vehicle control strategy of plug - in hybrid electric vehicle[J]. Autom Engi, 2013, 35(2):99-104. (in Chinese)
[5]	ZHAO Guangyao, CHEN Zeyu, et al. Energy management strategy for series hybrid electric vehicle[J]. *J Northeastern Univ*, 2013, 34:583-587.
[6]	LI S G, Sharkh S M, Walsh F C, et al. Energy and battery management of a plug-In series hybrid electric vehicle using fuzzy logic[J]. *IEEE Trans Vehi Tech*, 2011, 60(8):3571-3585.
[7]	PENG Jiankun, HE Hongwen, XIONG Rui. Rule based energy management strategy for a series-parallel plug-in hybrid electric bus optimized by dynamic programming[J]. *Appl Energ, 2017, 185*:224-256.
[8]	Jihun Han, Dongsuk Kum, Youngjin Park. Synjournal of predictive equivalent consumption minimization strategy for hybrid electric vehicles based on closed form solution of optimal equivalence factor[J]. *IEEE Trans Vehi Tech*, 2017, 66:5604-5616.
[9]	QIN Datong, ZHAN Seng, SU Ling. Management strategy of hybrid electrical vehicle based on driving style recognition[J]. *J Mech Engi*, 2016, 52:162-169.
[10]	LIN Xinyou, FENG Qigao, MO Liping, et al. Optimal adaptation equivalent factor of energy management strategy for plug-in CVT HEV[J]. *Proc IMECHE Part D; J Autom Engi, 2019, 233*:877-889.
[11]	王慧璇. 基于动态规划的 Plug-in混合动力汽车能量管理策略优化研究[D]. 济南:山东大学, 2012.
	WANG Huixuan. Research on plug-in hybrid electric vehicles energy management strategy based on dynamic planning[D]. Jinnan: Shan Dong Univ, 2012. (in Chinese)
[12]	ZHANG Shuo, XIONG Rui, SUN Fengchun, et al. Model predictive control for power management in a plug-in hybrid electric vehicle with hybrid energy storage system[J]. *Appl Energ, 2017, 185*:1654-1662. doi: 10.1016/j.apenergy.2015.12.035 URL
[13]	GUO Jinquan. A novel MPC-based adaptive energy management strategy in plug-in hybrid electric vehicles[J]. *Energy. 2019, 175*:378-392. doi: 10.1016/j.energy.2019.03.083
[14]	LI Liang, YOU Sixiong, YANG Chao, et al. Driving-behavior-aware stochastic model predictive control for plug-in hybrid electric buses[J]. *Appl Energ, 2016, 162*:868-879. doi: 10.1016/j.apenergy.2015.10.152 URL
[15]	曾祥瑞, 黄开胜, 孟凡博. 具有实时运算潜力的并联混合动力汽车模型预测控制[J]. 汽车安全与节能学报, 2012, 3(2):65-172.
	ZENG Xiangrui, HUANG Kaisheng, MENG Fanbo. Model predictive control for parallel hybrid electric vehicles with potential real-time capability[J]. J Autom Safe Energ, 2012, 3(2):165-172. (in Chinese)
[16]	赵韩, 吴迪. 基于随机模型预测控制的并联式混合动力汽车控制策略研究[J]. 汽车工程, 2014, 36(11):1289-1294.
	ZHAO Han, WU Di. A research on the control strategy of a parallel HEV based on stochastic model predictive control[J]. Autom Engi, 2014, 36(11):1289-1294. (in Chinese)
[17]	钱立军, 荆红娟, 邱利宏. 基于随机模型预测控制的四驱混合动力汽车能量管理[J]. 中国机械工程, 2018, 29(11):1342-1348.
	QIAN Lijun, JING Hongjuan, QIU Lihong. Energy management of a 4WD HEV based on SMPC[J]. Chin Mech Eng, 2018, 29(11):1342-1348. (in Chinese)
[18]	SUN Chao, Moura S J, HU Xiaosong, et al. Dynamic traffic feedback data enabled energy management in plug-in hybrid electric vehicles[J]. *IEEE Trans Contr Syst Tech*, 2015, 23(3):1075-1086. doi: 10.1109/TCST.2014.2361294 URL
[19]	孙超. 混合动力汽车预测能量管理研究[D]. 北京: 北京理工大学, 2016.
	SUAN Chao. Predictive energy management of hybrid electric vehicle powertrains[D]. Beijing:Beijing Institute of Technology, 2016. (in Chinese)
[20]	Johannesson L, Murgovski N, Jonasson E, et al. Predictive energy management of hybrid long-haul trucks[J]. *Contr Engi Pract*, 2015, 41:83-97.
[21]	钱立军, 邱利宏, 司远, 等. 智能网联环境下的混合动力汽车分层能量管理[J]. 汽车工程, 2017, 39(6):621-629.
	QIAN Lijun, QIU Lihong, SI Yuan, et al. Hierarchical energy management of hybrid electric vehicles under the environment of intelligent and connected vehicles[J]. Autom Engi, 2017, 39(6):621-629. (in Chinese)
[22]	邱利宏, 钱立军, 杜志远, 等. 车联网环境下车辆最优车速闭环快速模型预测控制[J]. 中国机械工程, 2017, 28(10):1245-1252.
	QIU Lihong, QIAN Lijun, DU Zhiyuan, et al. A closed-loop FMPC of optimal velocities for connected vehicles[J]. Chin Mech Engi, 2017, 28(10):1245-1252. (in Chinese)
[23]	ZENG Xiangrui, WANG Junming. A parallel hybrid electric vehicle energy management strategy using stochastic model predictive control with road grade preview[J]. *IEEE Trans Contr Syst Tech*, 2015, 23(6):2416-2423. doi: 10.1109/TCST.2015.2409235 URL
[24]	孙延秋. 基于模型预测控制的HEV能量管理策略的研究[D]. 大连: 大连理工大学, 2017.
	SUN Yanqiu. Energy management strategy of HEV based on model predictive control[D]. Dalian:Dalian University of Technology, 2017. (in Chinese)
[25]	Cairano S D, Bernardini D, Bemporad A, et al. Stochastic MPC with learning for driver-predictive vehicle control and its application to HEV energy management[J]. *IEEE Trans Contr Syst Tech*, 2014, 22(3):1018-1031. doi: 10.1109/TCST.2013.2272179 URL
[26]	秦大同, 秦岭. 基于显示随机模型预测控制的功率分流式混合动力汽车能量管理策略[J]. 华南理工大学学报 (自然科学版), 2019, 47(7):112-120.
	QING Datong, QING Ling . Power shunt hybrid electric vehicle energy management strategy based on display random model prediction control[J]. J Huanan Univ Tech (Nat Sci Edit), 2019, 47(7):112-120. (in Chinese)
[27]	YU Kaijiang, YANG Haizhu, TAN Xingguo, et al. Model predictive control for hybrid electric vehicle platooning using slope information[J]. *IEEE Trans Intel Transport Syst*, 2016, 17(7):1894-1909. doi: 10.1109/TITS.2015.2513766 URL
[28]	ZHANG Jiangyan, SHEN Tielong. Real-time fuel economy optimization with nonlinear MPC for PHEVs[J]. *IEEE Trans Contr Syst Techn*, 2016, 24(6):2167-2175. doi: 10.1109/TCST.2016.2517130 URL
[29]	GUO Lulu, GAO Bingzhao, GAO Ying, et al. Optimal energy management for HEVs in eco-driving applications using bi-level MPC[J]. *IEEE Trans Intel Transport Syst*, 2017, 18(8):2153-2162. doi: 10.1109/TITS.2016.2634019 URL
[30]	XIE Shaobo, HU Xiaosong, XIN Zongke, et al. Pontryagin’s minimum principle based model predictive control of energy management for a plug-in hybrid electric bus[J]. *Appl Energ, 2019, 236*:893-905. doi: 10.1016/j.apenergy.2018.12.032
[31]	WU Jinda, HE Hongwen,, PENG Jiankun, et al. Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus[J]. *Appl Energ*, 2018, 22:799-811.
[32]	QI Xuewei, LUO Yadan. Deep reinforcement learning enabled self-learning control for energy efficient driving[J]. *Transport Res Part C: Emerg Tech*, 2019, 99:67-81. doi: 10.1016/j.trc.2018.12.018 URL
[33]	LIU Teng, WANG Bo, YANG Chenglang. Online Markov chain-based energy management for a hybrid tracked vehicle with speedy Q Learning[J]. *Energy, 2018, 160*:544-555. doi: 10.1016/j.energy.2018.07.022 URL
[34]	YUAN Jingni, YANG Lin, CHEN Qu. Intelligent energy management strategy based on hierarchical approximate global optimization for plug-in fuel cell hybrid electric vehicles[J]. *Int’l J Hydro Energ*, 2018, 43:8063-8078.
[35]	陈虹. 模型预测控制[M]. 北京: 科学出版社, 2013: 1-407.
	CHENG Hong. Model Prediction Control[M]. Beijing: Science Press, 2013: 1-407. (in Chinese)

部件	参数名称	数值
发动机	发动机类型额定功率峰值转矩	JL472Q1 (49 ±2.45) kW (82 ± 4.1) Nm
电动机	额定功率额定转速	5 kW 1 000 r/min
电池	额定容量额定电压	6.5 A·h 80 V
变速器	最大速比最小速比主减速器传动比	0.498 4.04 4.99
其他参数	整车质量迎风面积轮胎半径空气阻力系数滚动阻力系数	1 190 kg 1.7 m² 0.274 m 0.32 0.014

部件	参数名称	数值
发动机	发动机类型额定功率峰值转矩	JL472Q1 (49 ±2.45) kW (82 ± 4.1) Nm
电动机	额定功率额定转速	5 kW 1 000 r/min
电池	额定容量额定电压	6.5 A·h 80 V
变速器	最大速比最小速比主减速器传动比	0.498 4.04 4.99
其他参数	整车质量迎风面积轮胎半径空气阻力系数滚动阻力系数	1 190 kg 1.7 m² 0.274 m 0.32 0.014

预测时域p中n_p	均方根误差Re
3	1.858 6
5	3.135 5
7	4.488 1
9	5.892 1

预测时域p中n_p	均方根误差Re
3	1.858 6
5	3.135 5
7	4.488 1
9	5.892 1