ARTICLES

Application of support vector machines plus to regression analysis for pressure-relief valves leaking

W. Sun, G. X. Meng, Q. Ye, H. L. Jin and J. Z. Zhang

School of Mechanical Engineering Shanghai Jiaotong University, Shanghai 200240, P.R. China. elec_soft@hotmail.com

---

Abstract - Carrying out regression analysis for gas leakage of pressure-relief valve (PRV) to get accurate leakage flow and changing trend of leakage will be helpful in assessing the reliability of PRV. Classic support vector regression (SVR) is an excellent regression model, and has been widely used in various fields. However, standard SVR model does regression only using leakage data without elements closely related to the leakage considered. In this paper a regression model based on support vector regression plus (SVR+) is put forward to perform leakage regression of PRV, in which particle swarm optimization (PSO) is used to select optimum parameters of SVR+, termed PSO_SVR+. The experimental results demonstrate that the proposed model taking the difference of inlet pressure and outlet pressure of PRV as hidden information can access a more favorable regression precision than SVR can provide. Meanwhile this article also investigates effects of PSO and Genetic Algorithm on the performance of regression model (SVR+ or SVR).

Keywords - Gas Leakage; Regression Modeling; Support Vector Regression; Hidden Information; Particle Swarm Optimization.

---

 

I. INTRODUCTION

Pressure-relief valve (PRV) does overpressure protection to the entire pneumatic circuit by adjusting of the inlet pressure and exporting a stable outlet pressure. The uncontrolled delivery of gas flow, due to fit clearances or sealing elements, which occurs between the inside and the outside of PRV or among cavities within PRV is gas leakage. In some cases, even the smaller amount of leakage can also result in severe damage to the outlet pressure and subsequent valve failure (Orchard, 2006). Gas leakage will inevitably result in a potential security risk (Bukowski and Goble, 2008). Basing on the existence and severity of the leakage of PRV, ISO 19973-4:2007 provides both test procedures and failure criteria for assessing the reliability of PRV (ISO, 2007). Hence investigation on the actual value and changing trend of gas leakage of PRV has realistic significance and better practical value in engineering, and attracts the attention of scholars. References (Sharif and Grosvenor, 1999; Instanes and Pedersen, 2008; Kaewwaewnoi et al., 2010) discuss the leakage by using acoustic emission sensor and obtain relatively accurate leakage flow, whereas none of them show changing trend. By analyzing influence of working condition and structure parameters of valves on gas leakage, references (Xu et al., 2009; Tang et al., 2005) build mathematical simulation model of leakage and present changing trend of leakage. But the mathematical simulation model can only acquire an approximate solution, which has a huge difference from actual leakage value. This paper handles with these problems using statistical method to offset these disadvantages.

Artificial neural network (ANN) is capable of fitting any nonlinear function with certain generalization ability and is used widely in engineering, such as (Figueroa and Cousseau, 2008) for time series prediction, (Montandon et al., 2008) for nonlinear model predictive control and (Fonseca et al., 2004) for production of sorbitol and gluconic acid. However, ANN can not guarantee its regression precision for small training samples since over-fitting problem or local minimum problem often happens. Due to perfect theoretical foundation, support vector regression (SVR) is more robust than ANN and other regression models, and has being gained popularity on engineering regression problems (Gunn, 1998; Vapnik, 1999). Taking gas leakage as input vectors, SVR can give a regression model with a higher precision. As (Xu et al., 2009; Tang et al., 2005) indicates, leakage of PRV has close relation with working condition and structure parameters of PRV, but SVR tends to only use leakage data without these elements considered.

In 2009, V. Vapnik proposes an advanced learning model support vector regression plus (SVR+; Vapnik, 2006; Vapnik et al., 2009), which adds hidden information based on SVR. The introduction of the hidden information, which is some correlation inside input vectors, and which is available only for the training examples and not be available (hidden) for the test examples, further boosts the generalization ability of SVR and makes SVR+ have a preferable application prospect.

Sun et al. (2011) discusses a new time series regression method based on SVR+, in which Genetic Algorithm (GA) is used to determine parameters of SVR+, and shows that SVR+ is more suitable for regression than ANN.

To attempt to get accurate leakage flow and reflect changing trend of leakage simultaneously, this article puts forward a regression analysis model based on SVR+ and particle swarm optimization (PSO), termed PSO_SVR+, which introduces SVR+ into the gas leakage analysis, and which takes working condition of PRV into account. Additionally this article employs PSO to select optimal parameters of SVR+ and investigates effects of different optimization algorithms on the performance of SVR+ and SVR.

II. SUPPORT VECTOR REGRESSION WITH HIDDEN INFORMATION

A. A Brief Review of SVR (Vapnik, 1999)

Given a set of data , , , where x_i denotes the input feature vectors, y_i stands for the target value, n is the number of dimension of x_i, and N is the total number of data. For regression purpose, Eq. (1) is defined to make approximation in a feature space, where the input x is mapped into a vector K(x) by using a proper kernel function.

(1)

where w is a vector in the feature space, b is a bias term. Then solving the optimization problem below can tackle this problem perfectly:

(2)

subject to:

	(3)
	(4)

where C is the regularization parameter and controls the trade-off between complexity and the number of non-separable data, ε is insensitive loss function, ξ and ξ^* are slack variables regarding ε - insensitive tube.

The decision function is

(5)

where the parameters a, a ^* are solutions to the following optimization problem:

(6)

where a, a^*, β, β^* are Lagrange multipliers.

B. SVR+

Given a set of samples , , , , where x_i denotes the input feature vector, is the hidden information vector, and y_i is the target value. is some correlation inside x_i. In fact for almost all machine learning problems there exists some sort of hidden information (Vapnik et al., 2009). In general the information is available only for the training examples and it will not be available (hidden) for the test examples (Vapnik et al., 2009).

SVM+ regression simultaneously maps x and x^* into K space and K^* space by utilizing two kernel functions which may be different or may be same with different kernel parameter. The goal is also to find the regression function Eq. (1) in K space:

Two nonnegative slack variables explain the hidden information

	(7)
	(8)

Similarly, this regression problem is also transformed into an optimization problem:

(9)

subject to:

	(10)
	(11)

where ? adjusts the relative weight of decision function capacity and correcting function capacity. Hence the decision function and the two correcting functions are defined respectively by the dual space solution:

	(12)
	(13)
	(14)

where the parameters a, a^*, β, β^* are solutions to the following optimization problem:

(15)

III. APPROACHES FOR SELECTING FREE PARAMETERS

A. A Brief Introduction to Particle Swarm Optimization

PSO is a population based stochastic optimization technique developed by Kennedy and Eberhart (1995), inspired by social behavior of bird flocking. To discover the optimal solution in a multi-dimensional space, a swarm of particles (called potential solutions) fly through the problem space by following the current optimum particles. The optimum solution is found if the number of iteration reaches the pre-determined maximum number of iteration, or if a termination condition is met.

B. A Controversial Issue

The earliest method of selecting parameters for SVR is parallel grid search (Chang et al., 2001). It has been demonstrated that parallel grid search is more time consuming and less effective than GA (Sun et al., 2011) or PSO (Fei et al., 2009; Prasad et al., 2010; Ren and Bai, 2010). However, with respect to GA and PSO, there is still a controversial issue about which of them is better suited for selecting parameters for support vector machines. Fei et al. (2009) and Prasad et al. (2010) prove GA is better than PSO, while Ren and Bai (2010) hold the opposite.

IV. REGRESSION MODEL FOR LEAKAGE OF PRV USING SVM+

The purpose of this part is to build a regression model with excellent generalization performance by training the example dataset.

A. Acquisition of the Leakage

To get the leakage of PRV, the standard pneumatic circuit specified by ISO 19973-4:2007 is employed, as shown in Fig. 1. When acquiring, with the supply pressure applied at the inlet port of PRV, and with shut-off valves 4 and 7 on, 5 and 11 off, the value of flow gauge 6 is the leakage flow. Otherwise, with shut-off valves 4 and 7 off, 5 and 11 on, PRV works with 2.5Hz of switching frequency of directional control valve 3.

Fig. 1: Acquisition circuit of leakage, inlet pressure and outlet pressure of PRV.

B. Selection and Acquisition of Hidden Information

The leakage of PRV is closely related to fit clearances between valve core and valve, sealing elements, inlet pressure, outlet pressure, structure parameters of PRV (Xu et al., 2009; Tang et al., 2005). Over a period of working time, the sealing status and fit clearances will change a lot, so it is very difficult to accurately measure their exact numerical changes especially when PRV is working. The primal structure parameters are available in the data sheet of PRV, whereas their minute changes caused by wearing and tearing have no significant influence on the leakage. So the difference of inlet pressure and outlet pressure, whose acquisition is feasible and easy in operation when the pneumatic circuit is running, are taken as hidden information.

As shown in Fig. 1, pressure gauges 9 and 13 are used to measure inlet pressure and outlet pressure respectively. Thus the hidden information at the sampling time of the leakage is available.

C. Data Preprocessing

Before training the model, it is necessary to normalize all the leakage data and all the hidden information , which can improve the generalization performance of SVR+ and can be helpful to have a more accurate regression result. Correspondingly, the output result of the model, which is not a true leakage data, is required to be de-normalized to get an actual leakage data.

D. Performance of the Model

The regression precision of model is evaluated by mean squared error Eq. (16) and squared correlation coefficient Eq. (17).

	(16)
	(17)

where y_i stands for the actual leakage value, is the regression leakage value and N is the total number of the predicted leakage data.

E. Determining the Parameters for the Model

Tuning unknown parameters of the model is a key to achieve the best regression precision. A total of six parameters in the model need to be tuned. Compared with SVR, besides C, ε and σ₁ (σ₁ is kernel parameter regarding the input feature vector), SVR+ has two additional parameters: weight coefficient ? and kernel parameter σ₂ of the hidden information vector (in this paper two RBF kernel functions with different kernel parameters are employed). In this study, 5-fold cross-validation via PSO is used to determine the optimum parameters of the model. Figure 2 shows the process of searching the better combination of these parameters.

Fig. 2: The flow chart of PSO optimizing parameters of SVR+.

In 5-fold cross-validation, the training data set is randomly separated to 5 subsets of equal size and a subset serves as the validation set and other subsets are used for training sets. Then, repeat this procedure 5 times with each subset taken as the validation set in turn. The average performance regarding 5 times validations is the particle's performance.

V. EXPERIMENT AND DISCUSSION

A. Experimental Records

In order to obtain the leakage flow, inlet pressure data and outlet pressure data, experiments are conducted in accordance with ISO 19973-4:2007. Three identical PRVs provided by SMC, whose model is AR20, are tested with the same air source, as shown in Fig. 3 based on the test circuit in Fig. 1. The three PRVs are sampled at intervals of 48 hours, and total 80 effective sample records are collected. Figure 4 is inlet pressure and outlet pressure of No.1 PRV, and their difference is Fig. 5. Figure 6 is leakage flow No.1 PRV. For clarity, the experimental records of No.2 PRV and No.3 PRV are not shown here.

Fig. 3: The pneumatic circuit for testing PRVs.

Fig. 4: Inlet pressure and Outlet pressure of No.1 PRV.

Fig. 5: Pressure difference of No.1 PRV.

Fig. 6: Leakage flow of No.1 PRV.

B. Analysis on Regression Precision

Based on Section IV, PSO_SVR+ models for the three PRVs are trained. GA_SVR+ models are also trained for comparison purpose. At the same time, GA and PSO are both utilized to search the best regression precision of standard SVR (GA_SVR and PSO_SVR), aiming to show differences between SVR and SVR+ (GA_SVR+ and PSO_SVR+). All models are implemented in MATLAB 7 (Eq. (6) and Eq. (15) are solved using the routine "fmincon" in MATLAB) and run on an Intel Xeon W3580 Workstation with 24G RAM.

Table 3 is parameters of SVR+ and SVR of No.1 PRV. For clarity, parameters of No.2 PRV and No.3 PRV are not listed here.

Table 1: Comparison of regression precision of SVR+ models and SVR models.

Table 2: Regression precision obtained by LIBSVM

Table 3: Parameters of SVR+ and SVR of No.1 PRV

Table 1 lists MSE and r² of PSO_SVR+ and GA_SVR+, and their regression results regarding the three PRVs are depicted in Figs.7-9 respectively. For No.1 PRV and No.3 PRV, r² of GA_SVM+ is slightly smaller than that of PSO_SVM+. For No.2 PRV, r² of both models are almost equal. Whereas for all PRVs, MSE of GA_SVR+ is inferior obviously to that of PSO_SVR+. Table 1 also lists MSE and r² of PSO_SVR and GA_SVR. The differences between PSO_SVR and GA_SVR in r² are not apparent, while in MSE the former is better than the latter.

Fig. 7: Regression results regarding No.1 PRV (SVR and SVR+ curves are coincident).

Fig. 8: Regression results regarding No.2 PRV

Fig.. 9: Regression results regarding No.3 PRV

To ensure accuracy of SVR models in this paper, we also use LIBSVM (Chang et al., 2001) to perform regression analysis of valves. Table 2 shows their MSE and r², and proves SVR models developed in MATLAB work as well as LIBSVM.

In Table 1, PSO_SVR+ outperforms PSO_SVR prominently except No.1 PRV. Their average MSE of all PRVs is 1.7968 and 2.1614, and their average r² are 0.8864 and 0.8603. Similarly, GA_SVR+ is also superior to GA_SVR. So the difference (hidden information of SVR+) of inlet pressure and outlet pressure has a positive impact on regression accuracy. Table.1 demonstrates the superiority of the PSO_SVR+.

As for No.1 PRV, both PSO_SVR and PSO_SVR+ have almost identical performance in Table 1, meanwhile GA_SVR and GA_SVR+ arrive at the same conclusion. Actually in certain cases, solutions of SVM+ include solutions of SVM. Calculating the partial derivative to Eq. (15) regarding and , Eqs. (18) and.(19) are got.

	(18)
	(19)

When all and hold, Eqs. (13) and (14) are equal to ξ and ξ^* of the standard SVM respectively. No.1 PRV is the case.

C. Comparison of Time Consumed

Now we compare time consumed of SVR and SVR+, with specified parameters and without regard to the regression precision. Table 4 shows time consumed of both models taking different size of the dataset as input vectors. Obviously, for each PRV, runtime of SVR+ is much longer than that of SVR.

Table 4: Comparison of time consumed of SVR+ models and SVR models.

Note: because models are implemented using M language in Matlab not C++ language, their runtime is longer.

VI. CONCLUSIONS

This paper performs gas leakage regression of PRV, which simultaneously provides assessing the reliability of PRV with accurate leakage flow and changing trend of leakage. In this paper a regression model PSO_SVR+, which perform leakage regression not only with leakage considered but also with elements closely related to the leakage considered, is proposed. With introduction of hidden information, the regression accuracy of SVR+ is further improved. The experimental results demonstrate the superiority of the model proposed. Additionally this article also investigates effects of PSO and GA on the performance of regression model (SVR+ or SVR). Squared correlation coefficients of both algorithms are almost equal, while mean squared error with PSO is superior obviously to that with GA.

Although the model in this study gains a favorable regression accuracy and is a more promising one, the calculation effort brought by introduction of hidden information can not be ignored, which should deserve an in-depth research in the future.

REFERENCES
1. Bukowski, J.V. and W.M. Goble, "Statistical analysis of pressure relief valve proof test data: Findings and implications," Proc. 2008 AIChE Spring National Meeting, New Orleans, LA, United states, (2008).         [ Links ]
2. Chang, C.-C. and C.-J. Lin, "LIBSVM: a library for support vector machines," Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm (2001).         [ Links ]
3. Fei, S.-W., C.-L. Liu, and Y.-B. Miao, "Support vector machine with genetic algorithm for forecasting of key-gas ratios in oil-immersed transformer," J. Expert Systems with Applications, 36, 6326-6331 (2009).         [ Links ]
4. Figueroa, J.L. and J.E. Cousseau, "Dynamical functional artificial neural network: use of efficient piecewise linear functions," Latin Am. Appl. Res., 38, 187-193 (2008).         [ Links ]
5. Fonseca, E.F., T.L.M. Alves, E.L. Lima and M.B. De Souza Jr., "A hybrid neural model for the production of sorbitol and gluconic acid using immobilized Zymomonas mobilis cells," Latin Am. Appl. Res., 34, 187-193 (2004).         [ Links ]
6. Gunn, S.R., Support Vector Machines for Classification and Regression, Technical Report, University of Suthoampton, School of Electronics and Computer Science (1998).         [ Links ]
7. Instanes, G. and A. Pedersen, "Acoustic leak monitoring on subsea valves," SPE Asia Pacific Oil and Gas Conference and Exhibition 2008 - Gas Now: Delivering on Expectations, Perth, WA, Australia, 1, 27-31 (2008).         [ Links ]
8. ISO, ISO 19973- 4: 2007 ISO/TC131, Pneumatic fluid power - assessment of component reliability by testing - Part 4: pressure regulators[S], (2007).         [ Links ]
9. Kaewwaewnoi, W., A. Prateepasen and P. Kaewtrakulpong, "Investigation of the relationship between internal fluid leakage through a valve and the acoustic emission generated from the leakage," J. Measurement: Journal of the International Measurement Confederation, 43, 274-282 (2010).         [ Links ]
10. Kennedy, J. and R. Eberhart, "Particle swarm optimization," IEEE International Conference on Neural Networks, Perth, Australia, 1942-1948 (1995).         [ Links ]
11. Montandon, A.G., R.M. Borges and H.M. Henrique, "Experimental application of a neural constrained model predictive controller based on reference system," Latin Am. Appl. Res., 38, 51-62 (2008).         [ Links ]
12. Orchard, B., "Taking a soft seated approach to preventing valve leakage," J. World Pumps, 478, 20-22 (2006).         [ Links ]
13. Prasad, Y., K.K. Biswas and C.K. Jain., "SVM classifier based feature selection using GA, ACO and PSO for siRNA design," 1th Advances in Swarm Intelligence., Beijing, China, 304-317 (2010).         [ Links ]
14. Ren, Y. and G. Bai, "Determination of optimal SVM parameters by using GA/PSO," J. Journal of Computers, 5, 1160-1168 (2010).         [ Links ]
15. Sharif, M. A. and R.I. Grosvenor, "Internal valve leakage detection using an acoustic emission measurement system," J. Transactions of the Institute of Measurement and Control, 20, 233-242 (1999).         [ Links ]
16. Sun, W., G.-X. Meng and Q. Ye, "A new time series regression method based on support vector machine plus and genetic algorithm," J. Advanced Materials Research, 201-203, 2277-2280 (2011)         [ Links ]
17. Tang, F.-M., Z.-D.Wang and M.-Y.Chen, "Multiclass classification methods for support vector machines," J. Kongzhi yu Juece/Control and Decision, 20, 746-749 (2005).         [ Links ]
18. Vapnik, V.N., "An overview of statistical learning theory," IEEE Transactions on Neural Networks, 10, 988-999 (1999).         [ Links ]
19. Vapnik, V., Estimation of Dependences Based on Empirical Data: Empirical Inference Science. Information Science and Statistics, Springer, New York (2006).         [ Links ]
20. Vapnik, V., A. Vashist and N. Pavlovitch, "Learning using hidden information (learning with teacher)," International Joint Conference on Neural Networks, Atlanta, GA, United states, 3188-3195 (2009).         [ Links ]
21. Xu, Z.-P., X.-Y. Wang and Y.-X. Luo, "Clearance leakage in high pressure pneumatic relief valve with slide pilot," J. Zhejiang Daxue Xuebao(Gongxue Ban)/Journal of Zhejiang University (Engineering Science), 43, 1789-1793 (2009).         [ Links ]

Received: November 16, 2010.
Accepted: June 15, 2011.
Recommended by Subject Editor Jorge Solsona.