基于等效因子的Q学习燃料电池汽车能量管理策略

doi:10.3969/j.issn.1674-8484.2022.04.020

摘要/Abstract

摘要：

为提高燃料电池混合动力汽车 (FCHEV) 燃料经济性以及维持蓄电池能量平衡，该文提出了基于等效因子的Q-learning算法的能量管理策略。构建等效耗氢量最小与维持蓄电池荷电状态（SOC）平衡的目标函数，建立FCHEV动力源能量流转化平衡模型，通过能量转化平衡机理得到耗氢量的等效因子；在城市循环+全球轻型汽车测试循环（UDDS+WLTC）工况下，对需求功率的转移概率矩阵进行求解，利用Q-learning算法离线优化燃料电池和蓄电池的输出功率；基于MATLAB/Simulink平台建立了前向仿真模型，进行整车性能的仿真试验。结果表明：在WLTC循环工况下，该策略的100 km等效耗氢量为0.730 kg，接近基于动态规则（DP）控制策略的耗氢量，且SOC保持在合理的范围内，验证了该策略的有效性；在西宁市实际工况下，验证了本文所提控制策略的适应性。

关键词: 燃料电池混合动力汽车（FCHEV）, 等效因子, Q-learning算法, 能量管理

Abstract:

An energy management strategy based on equivalent factor Q-learning algorithm was proposed to improve the fuel economy of fuel cell hybrid electric vehicles (FCHEVs) and maintain the battery energy balance. The objective function of minimizing equivalent hydrogen consumption and maintaining battery state of charge (SOC) was constructed to establish the energy flow conversion balance model of FCHEVs power source, and the equivalent factor of hydrogen consumption was obtained through the energy conversion balance mechanism. The transfer probability matrix of required power was solved under urban dynamometer driving schedule + world light vehicle test cycle (UDDS+WLTC) conditions, and the output power of fuel cells and batteries was optimized offline by Q-learning algorithm. The forward simulation model was established based on MATLAB/Simulink platform, and the vehicle simulation was carried out under different working conditions. The results show that the equivalent of the hydrogen consumption per 100 km is 0.730 kg under WLTC cycle conditions in the proposed strategy, which is close to that based on the dynamic programming (DP) control strategy, and the SOC is kept within a reasonable range, which verifies the effectiveness of the proposed strategy. And the adaptability of the proposed control strategy is also verified in the actual working conditions of Xining City.

Key words: fuel cell hybrid electric vehicle (FCHEV), equivalent factor, Q-learning algorithm, energy management

中图分类号:

U469.7

尹燕莉, 张鑫新, 潘小亮, 詹森, 黄学江, 王福振. 基于等效因子的Q学习燃料电池汽车能量管理策略[J]. 汽车安全与节能学报, 2022, 13(4): 785-795.

YIN Yanli, ZHANG Xinxin, PAN Xiaoliang, ZHAN Shen, HUANG Xuejiang, WANG Fuzhen. Equivalent factor of energy management strategy for fuel cell hybrid electric vehicles based on Q-Learning[J]. Journal of Automotive Safety and Energy, 2022, 13(4): 785-795.

图/表 18

图1 燃料电池汽车动力系统结构

整车	整车质量	1.399 t
	迎风面积	2.0 m2
	风阻因数	0.335
	滚动阻力因数	0.008 6
	车轮半径	0.282 m
传动系统	主减速器速比	8.879
传动系统	传动效率	0.90
燃料电池	最大功率	50 kW
燃料电池	DC/DC转换器效率	0.94
蓄电池	额定容量	12 Ah
蓄电池	额定转速	10⁴ r/min
电机	最大功率	75 kW
电机	最大转矩	272 Nm

表1 整车主要参数

图2 电动机效率MAP图

图3 燃料电池效率和耗氢率曲线

图4 Rint等效电路

图5 蓄电池放电时能量流等效模型

图6 蓄电池充电时能量流等效模型

图7 不同车速的需求功率转移概率矩阵

图8 基于Q学习算法的燃料电池汽车能量管理策略逻辑框图

图9 Q学习算法流程图

图10 不同车速下蓄电池和燃料电池功率优化结果

图11 WLTC循环工况

图12 WLTC工况下燃料电池和蓄电池不同控制策略的输出功率对比

控制策略	100 km耗氢量 g	最终SOC %
DP	0.686	60.8
QL	0.730	59.4
RB	0.798	57.8

表2 WLTP工况下燃料经济性对比

图13 WLTC工况下的SOC变化轨迹图

图14 实际工况下不同控制策略蓄电池和燃料电池输出功率对比

策略	耗氢量 kg	100 km耗氢量 kg	与DP算法对比 %
λ_var -DP	0.181	0.886	-
λ_var -Q	0.195	0.953	+7.52
λ_cv-Q	0.199	0.972	+9.68

表3 西宁某路段实际工况下燃料经济性对比

图15 不同控制策略SOC变化曲线

参考文献 18

[1]	Li Siguang, Sharkh S M, Walsh F C, et al. Energy and battery management of a plug-in series hybrid electric vehicle using fuzzy logic[J]. *IEEE Trans Vehi Tech*, 2011, 60(8): 3571-3585.
[2]	罗国鹏, 罗禹贡, 李克强. 基于最佳电能使用的插电式混合动力客车控制策略[J]. 汽车工程, 2012, 34(6): 475-478.
	LUO Guopeng, LUO Yugong, LI Keqiang. Control strategy for plug-in hybrid electric bus based on optimal electric energy usage[J]. *Autom Engi*, 2012, 34(6): 475-478. (in Chinese)
[3]	高建平, 赵金宝, 杨松, 等. 基于驾驶意图的插电式混合动力公交车控制策略[J]. 机械工程学报, 2016, 52(24): 107-114. doi: 10.3901/JME.2016.24.107
	GAO Jianping, ZHAO Jingbao, YANG Song, et al. Control strategy of plug-in hybrid electric bus based on driving intention[J]. *J Mech Engi*, 2016, 52(24): 107-114. (in Chinese)
[4]	ZHOU Yuan, HOU Shijie, HAN Erliang, et al. Dynamic programming-based energy management strategy optimization for hybrid electric commercial vehicle[J]. *Autom Engi*, 2012, 34(08): 663-668
[5]	巴特, 高印寒, 王庆年, 等. 基于动态规划算法的并联混合动力客车控制策略优化[J]. 汽车工程, 2015, 37(12): 1359-1365.
	BA Te, GAO Yinghan, WANG Qingnian, et al. Control strategy optimization of parallel hybrid electric bus based on dynamic programming algorithm[J]. *Autom Engi*, 2015, 37(12): 1359-1365. (in Chinese)
[6]	解少博, 刘通, 李会灵, 等. 基于马尔科夫链的并联PHEB预测型能量管理策略研究[J]. 汽车工程, 2018, 40(8): 871-877+911.
	XIE Shaobo, LIU Tong, Li Huiling, et al. Research on predictive energy management strategy for parallel PHEB based on Markov chain[J]. *Autom Engi*, 2018, 40(8): 871-877+911. (in Chinese)
[7]	ZHENG Chunhua, LI Weimin, LIANG Quan. An energy management strategy of hybrid energy storage systems for electric vehicle applications[J]. *IEEE Trans Sustain Energ, 201*,89(4): 1180-1188.
[8]	SUN Haochen, FU Zhumu, TAO Fazhan, et al. Data-driven reinforcement-learning-based hierarchical energy management strategy for fuel cell/battery/ultracapacitor hybrid electric vehicles[J]. *J Power Sources, 2020, 455*(C): 227964-227964.
[9]	WANG Yujie, SUN Zhendong, CHEN Zonghai. Rule-based energy management strategy of a lithium-ion battery, supercapacitor and PEM fuel cell system[J]. *Energy Procedia, 2019, 158*: 2555-2560. doi: 10.1016/j.egypro.2019.02.003
[10]	XU Liangfei, Mueller C D, LI Jianqiu, et al. Multi-objective component sizing based on optimal energy management strategy of fuel cell electric vehicles[J]. *Appl Energ, 2015, 157*(1): 664-674.
[11]	WU Xiaohua, HU Xiaosong, LIN Xiaofei, et al. Convex programming energy management and components sizing of a plug-in fuel cell urban logistics vehicle[J]. *J Power Sources, 2019, 423*: 358-366. doi: 10.1016/j.jpowsour.2019.03.044
[12]	林歆悠, 夏玉田, 李雪凡, 等. 燃料电池汽车行驶里程自适应ECMS策略[J]. 汽车工程, 2019(7): 750-756.
	LIN Xinyou, XIA Yutian, LI Xuefan, et al. Vehicle mileage adaptive ECMS strategy for fuel cell vehicles[J]. *Autom Engi*, 2019(7): 750-756. (in Chineses)
[13]	Barsali S, Miulli C, Possenti A. A control strategy to minimize fuel consumption of series hybrid electric vehicles[J]. *IEEE Trans Energ Conv*, 2004, 19(1): 187-195. (in Chinese)
[14]	ZHENG Qing, YUAN Haorui, WU Jinzhu, et al. Equivalent consumption minimization strategy based on dynamic programming for plug-in hybrid electric vehicle[J]. *IFAC-Papers On Line*, 2018, 51(31): 612-617.
[15]	Wipke K B, Cuddy M R, Burch S D. ADVISOR 2.1: A user-friendly advanced powertrain simulation using a combined backward/forward approach[J]. *IEEE Trans Vehi Tech*, 1999, 48(6): 1751-1761.
[16]	H Lee, Cha S W. Energy management strategy of fuel cell electric vehicles using model-based reinforcement learning with data-driven model update[J]. *IEEE Access*, 2021(99): 59244-59254
[17]	Fares D, Chedid R, F Panik, et al. Dynamic programming technique for optimizing fuel cell hybrid vehicles[J]. *Int’l J Hydro Energ*, 2015, 40(24): 7777-7790.
[18]	SONG Ke, CHEN Huan, WEN Peiming, et al. A comprehensive evaluation framework to evaluate energy management strategies of fuel cell electric vehicles[J]. *Electrochimica Acta, 2018, 292*: 960-973.