Current Issue

25 August 2012, Volume 33 Issue 8

Previous Issue    Next Issue

Academic paper

Cooperative transmission design combining network coding and superposition coding

Hai-rong WANG,Yong-ming HUANG,Lv-xi YANG

2012, 33(8):  1-9.  doi:1000-436X(2012)08-0001-09

Asbtract ( 0 )   HTML ( 0)   PDF (753KB) ( 223 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A joint network coding (NC) and superposition coding scheme was proposed to realize a cooperative transmission.The use of NC was able to improve spectral efficiency over classical decode and forward relaying strategy by relaxing the orthogonality constraint between different data streams and mitigating the resultant interference through precoding.In contrast to using the CSI based beamforming scheme which could achieve near optimal performance but involves huge computational complexity,designing the beamforming vector using the form of superposition coding was proposed,such that CSI at the transmitters were not required and the involved computation was greatly reduced.The theoretical analysis and simulation results reveal that the proposed scheme shows performances close to the near-optimal CSIT based scheme.

Resource allocation algorithms in the reconfigurable network based on network centrality and topology potential

Zi-hou WANG,Yan-ni HAN,Tao LIN,Yue-mei XU,Hui TANG

2012, 33(8):  10-20.  doi:1000-436X(2012)08-0010-11

Asbtract ( 0 )   HTML ( 0)   PDF (763KB) ( 516 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

In reconfigurable network environment,a major challenge is how to map virtual networks to the substrate network efficiently,network centrality and topology potential were introduced into the allocation of virtual resources.Two algorithms were proposed that consider the embedding problem with the attributes of network topology.Simulation experiments show that the proposed algorithms increase the acceptance ratio and the revenue,while decreasing the provisioning cost in the long term.

Research on resource management and service mapping method for reconfigurable flexible network

Bao-jin WANG,Bin-qiang WANG,Zhi-ming WANG

2012, 33(8):  21-34.  doi:1000-436X(2012)08-0021-14

Asbtract ( 0 )   HTML ( 0)   PDF (940KB) ( 495 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Reconfigurable flexible network (RFNet) based on resource re-composition was proposed.The network service in RFNet was composed of special components,and could construct the new network service to satisfy a coming traffic demand by component composition technology in both software and hardware area.After introducing the architecture of RFNet,a process flow mechanism of resource management,carraying network mapping and network service mapping was proposed.By simulation evaluation,the mechanism was confirmed to provide the support of heterogeneous networks construction and coexistence,and especially the reconfigurable service carrying network mapping(RSCNM)algorithm based on load balance had an advantage over network successful construction ratio(SCR),maximum node stress,average link utilization rate and average revenue.

Hardware reconfigurable coding and evolution algorithm based on evolvable hardware

Ting WANG,Ju-long LAN,Jun-ting WU

2012, 33(8):  35-41.  doi:1000-436X(2012)08-0035-07

Asbtract ( 115 )   HTML ( 1)   PDF (706KB) ( 541 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A planar mapping function increments chromosome coding method based on FPGA platform with SRAM-architecture was proposed.The method could improve hardware reconfiguration efficiency by coding mapping realized by double platform mapping of binary configurable file string.Meanwhile,a betterment difference evolution algorithm was proposed based on local optimal mechanism introduced.The algorithm could promote convergence rate and whole efficiency.Finally,the result of the algorithm emulation shows that:MDE improves disadvantage of difference evolution algorithm with local optimal and approaches actual optimization outcome.

Design and implementation of packet forwarding engine for reconfigurable routers

Yi-jiao CHEN,Ze-xin LU,Zhi-gang SUN,Tao LI,Gao-feng LV

2012, 33(8):  42-51.  doi:1000-436X(2012)08-0042-10

Asbtract ( 110 )   HTML ( 4)   PDF (786KB) ( 396 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

The packet forwarding engine should provide not only the basic packet switching function,but also the reconfigurable ability for supporting the network processing functions evolving with time and space.Aiming at the above requirements,a component reconfiguration model of packet forwarding engine for the reconfigurable routers was proposed,and the program/circuit component running environment consisting of FPGA and network processers was designed and implemented based on the Pass-Through mode.The system implementation and application experimental results show that the packet forwarding engine for reconfigurable routers can support flexibly reconfiguration and mapping for the diverse service components,and it can provide high throughput and low-latency packet forwarding ability.

Self-healing mechanism for reconfigurable service overlay networks

Yu-ting MIAO,Chun-ming WU,Qiang YANG,Ming JIANG

2012, 33(8):  52-61.  doi:1000-436X(2012)08-0052-10

Asbtract ( 58 )   HTML ( 1)   PDF (750KB) ( 339 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A fast self-healing algorithm of reconfigurable service overlay networks against the physical substrate faults was presented.The suggested algorithm converted the self-healing problem to the multi-commodity flow problem.Extensive simulation experiments and the numerical results show that the proposed fast self-healing algorithm can further improve the healing success ratio and efficiency of the service overlay networks in comparison with the conventional overall re-mapping approach,whilst significantly reduces the load across the physical network.

Channel estimation based on cyclic orthogonal training sequence for relay-assisted cooperative system

Zhi-peng ZHANG,Jie ZHONG,Min-jian ZHAO

2012, 33(8):  62-58.  doi:1000-436X(2012)08-0062-07

Asbtract ( 57 )   HTML ( 1)   PDF (739KB) ( 212 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A practical channel estimation algorithm based on cyclic orthogonal training sequence for relay-assisted cooperative system was presented.According to this algorithm,the source node transmitted a cyclic orthogonal training sequence,and the relay nodes circularly shifted the received training sequences before retransmitted them.Although the relays retransmitted the training sequences to the destination node simultaneously,the correlation among them was fully eliminated.The corresponding least square (LS) and linear minimum mean square (LMMSE) channel estimators were derived.Simulation results confirm that the proposed algorithm has high accuracy,high efficiency and low complexity.

Context-aware link scheduling in static wireless networks with successive interference cancellation

Shao-he LV,Xiao-dong WANG,Xing-ming ZHOU

2012, 33(8):  69-76.  doi:1000-436X(2012)08-0069-08

Asbtract ( 62 )   HTML ( 0)   PDF (821KB) ( 249 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Greedy link scheduling based on the physical model was studied in a static wireless network with SIC.In general,there were two major stages in a greedy scheduling scheme,link selection,i.e.,to decide which link was scheduled next,and time slot selection,i.e.,to decide which slot was allocated to a given link.Most available schemes took a first-fit policy in the second stage and became less efficient when SIC was available.For the time slot selection stage,tolerance margin was defined to measure the saturation of a link set and two heuristic policies were presented:one was to schedule a link to a slot such that the resulting set of links had a maximum tolerance margin; the other was to choose a slot such that the decrease of tolerance margin was minimum.Simulation results show that the performance of the proposed schemes is much better than the first-fit policy.

Blind extraction of underdetermined mixtures via time-frequency mask

Ming XIAO,Feng GAO,Gong-xian SUN,Sheng-li XIE

2012, 33(8):  77-84.  doi:1000-436X(2012)08-0077-08

Asbtract ( 63 )   HTML ( 0)   PDF (952KB) ( 493 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

For blind source estimation problem of underdetermined and non-completely sparse signals,a novel blind extraction algorithm was proposed.The algorithm combined the non-completely sparse blind extraction algorithm with the DUET approach.It computed firstly the extraction vector of source and the normal vector of source direction,and then renewed the mixtures and the mixing matrix using a linear transform,and recoverd the source using time-frequency mask method finally.Several speech signal experiments demonstrate its performance and practical.

Algebraic fault attack on PRESENT

Ke-hui WU,Xin-jie ZHAO,Tao WANG,Shi-ze GUO,Hui-ying LIU

2012, 33(8):  85-92.  doi:1000-436X(2012)08-0085-08

Asbtract ( 105 )   HTML ( 5)   PDF (867KB) ( 691 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A new fault analysis method on PRESENT—algebraic fault attack was proposed.This attack combined conventional algebraic cryptanalysis with fault attack,firstly built equivalent Boolean algebraic equations of cipher encryption by algebraic cryptanalysis method; secondly got information of fault cryptograph by fault attack technique,and transformed differential of fault and cryptograph into additional algebraic equations; finally utilized Crypto Mini SAT solver to solve the equations and recover key.Experiments demonstrate that after injecting 4-bit fault to the 29^th round of PRESENT-80,the fault location and fault value are unknown,only 2 injectings can recover 64-bit last whitening key in 50 seconds that reduce master key of PRESENT-80 searching space to 2¹⁶,then recover the master key after 1 minute brute-force-search on average; compared with previous fault attack on PRESENT,the amount of this attack sample is the smallest;meanwhile,the analysis method proposed can be applied into the algebraic fault attack of other block ciphers.

Service-oriented dynamic resource configuration optimization scheme in RFID system

Jian-hua LIU,Wei-qin TONG

2012, 33(8):  93-105.  doi:1000-436X(2012)08-0093-13

Asbtract ( 40 )   HTML ( 1)   PDF (745KB) ( 405 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Aiming at the defect that the rate between tag and reader was not match and their resource-constrained problem in the radio frequency identification(RFID)systems,the collaboration communication context was analyzed theoretically.Service-oriented dynamic resource configuration optimization scheme of RFID system was proposed which used the component quality of service(QoS) information of tag and reader to establish closed-loop control framework and developed three control strategies to adjust QoS parameter according to the state and type of its resources.The scheme was analyzed and validated for performance through experiments.The numerical experiments results show that the optimization mechanism make throughput of RFID systems improve,use resources efficiently and reduce the energy consumption and interference of reader-to-reader.

Wi-Fi co-channel interference suppression approach in LTE uplink

Wan-zhi MA,Hong-zhi ZHAO,Hui-yue YU,Jun WANG,You-xi TANG

2012, 33(8):  106-112.  doi:1000-436X(2012)08-0106-07

Asbtract ( 44 )   HTML ( 0)   PDF (720KB) ( 992 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Considering the Wi-Fi co-channel interference in the 3rd generation mobile communication long term evolution (LTE) uplink with multi-received antennas,a new combining approach named interference reconstruction rejection Combining was proposed.Firstly,as the covariance matrix approximation of the interference,the covariance matrix of the received signal was estimated.Secondly,the desired signal was reconstructed by interference rejection combining.Then,the interference was reconstructed by removing the desired signal.At last,the covariance matrix of the interference was estimated again for interference rejection combining.Simulation results over multipath fading channels show that the IRRC approach based on the minimum mean square error outperformed the conventional IRC approach about 2dB when the signal in LTE uplink is modulated by QPSK with 1 transmit antenna and 2 receive antennas,and the jam to signal ratio is 0dB.

Multiple symbol soft-output non-coherent detection of serial concatenated CPM

You-fu YANG,Jian-wei LIU,Qi-shan ZHANG,Chun-di XIU

2012, 33(8):  113-118.  doi:1000-436X(2012)08-0113-06

Asbtract ( 55 )   HTML ( 0)   PDF (686KB) ( 320 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A non-coherent detection algorithm for CPM was proposed.Several sequential symbols were used as the observation window to calculate the prior probability for codeword in CPM soft demodulation process,and the more observation symbols were used,the better performance the algorithm achieved.Non-coherent receiver architecture was given,in which expansion method of Walsh basis function was utilized to reduce the calculating amount in matched filtering.Non-coherent detection performances of convolutional encoding CPM system adopting the proposed algorithm,with different CPM modulation parameters,were compared through simulation.The results indicate that proposed non-coherent detection algorithm brings significant robustness.As to 2CPM,4CPM serial concatenated encoding modulation,when the observation length gets to 4 symbols,system with the proposed algorithm gets nearly 6dB performance improvement than with the single-symbol non-coherent detection algorithm.

Set reconciliation based on counting Bloom filters

Xiao-mei TIAN,Da-fang ZHANG,Kun XIE,Can HU,Xiao-bo YANG,Chang-qiong SHI

2012, 33(8):  119-127.  doi:1000-436X(2012)08-0119-09

Asbtract ( 88 )   HTML ( 0)   PDF (723KB) ( 656 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A new set reconciliation algorithm was presented,which called counting-Bloom-filter based set reconciliation(CBFSR).This method represented sets S_A and S_B as counting Bloom filters,subtracts S_A's counting Bloom filter from S_B's counting Bloom filter,the differences denoted CBF(S_B)?CBF(S_A),then determined S_B?S_A elements in S_B with CBF(S_B)?CBF(S_A),and finally performed union operation on S_B?S_A and S_A.Simulation resdts show counting-Bloom-filter based set reconciliation combines the advantages of exact set reconciliation and approximate set reconciliation,besides gaining all S_B?S_A elements with single-round messgage exchanges,it can apply to update-intensive distributed systems because counting Bloom filters support element deletion operation.

Research on diversity receive technology on wireless optical communication using subcarrier modulation

Dan CHEN,Xi-zheng KE

2012, 33(8):  128-133.  doi:1000-436X(2012)08-0128-06

Asbtract ( 72 )   HTML ( 4)   PDF (686KB) ( 371 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Based on the log-normal model under the turbulence channel,the BPSK subcarrier modulated FSO system with spatial diversity was established.Under different scintillation distribution and different detectors,the error performance and diversity gain among three linear combining technologies were compared.The simulation results show that the performance improved of system by MRC is the best,followed by EGC,and SelC is poor.Spatial diversity is efficient to improve the performance and have strong ability on resistance to atmospheric channel decline.But on short link FSO system experiencing weak irradiance fluctuation,SelC spatial diversity would not be recommended.

Gray images of the linear codes and their dual codes over F₄+uF ₄

Xiu-sheng LIU,Hua-lu LIU

2012, 33(8):  134-137.  doi:1000-436X(2012)08-0134-04

Asbtract ( 65 )   HTML ( 0)   PDF (496KB) ( 359 )   Knowledge map   

References | Related Articles | Metrics

The linear codes over ring F₄＋uF₄ and field F₄ were studied.By using the Gray weight w_G ,the Gray distance d_G of codes C over F₄＋uF₄,and the Gray map ψ from (F₄＋uF₄)ⁿ to $$F_4^{2n}$$,the Gray image ψ(C)of linear codes C over F₄＋uF₄ were linear codes over F ₄ as well,d _H(ψ(C))＝d _G(C),and the proposition that linear codes ψ(C )⊥ were the dual codes of ψ(C) were showed.Furthermore,the Gray image ψ(C)of cyclic codes C over F₄＋uF₄ were the 2-quasi cyclic codes over F₄.

Distributed multicell beamforming scheme with limited intercell communication

Yong-ming HUANG,Ying-gang DU,Hui YU,Lv-xi YANG

2012, 33(8):  138-146.  doi:1000-436X(2012)08-0138-09

Asbtract ( 47 )   HTML ( 0)   PDF (762KB) ( 123 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Coordinated downlink beamforming optimization was studied for the time division duplexed(TDD)multicell system with limited inter-BS communication.With the objective to reach the particular point on the Pareto boundary of user rates with the maximum worst-user rate,a near-optimal two-step centralized solution was first proposed.Based on that,a distributed solution was then developed,in which only limited number of positive scalars were needed to exchange between the BSs.Numerical simulation results show that the proposed distributed solution achieves a rate performance close to the centralized solution.

Technical Report

Multilevel access control for video database

Jin-bo XIONG,Zhi-qiang YAO,Jian-feng MA,Qi LI

2012, 33(8):  147-154.  doi:1000-436X(2012)08-0147-08

Asbtract ( 51 )   HTML ( 0)   PDF (593KB) ( 588 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

In regards to the critical issues of classification protecting sensitive information of the data stored in video database,a multilevel access control model for the video database was proposed.A series of algorithms was designed and developed in the model where a method of distinguishing user identity and an algorithm to ensure good identity strength were presented.The result of them was then used as the input to user's security level membership function.The user's security level membership degree was subsequently acquired from the result of this function,and it is then,together with the security level membership degree of video data,used as input to the comparison function of security level membership degree in authorization rule.As a result,the multilevel access control can be implemented neatly through combining the result of the comparison function with time element.Compared with the existing access control models,the most outstanding characteristics of the proposed model are realizations of dynamic authorization and video data classification protection.

UGC³:a covert communication method defense against censorship

Qing-feng TAN,Pei-peng LIU,Jin-qiao SHI,Xiao WANG,Li GUO

2012, 33(8):  155-161.  doi:1000-436X(2012)08-0155-07

Asbtract ( 88 )   HTML ( 2)   PDF (667KB) ( 350 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

A UGC-based covert communication method: UGC³was presented,which used fixed infrastructure (UGC sites) to form a fully distributed overlay.To exchange information between sender and receiver,an efficient resource discovery algorithm was proposed to negotiate the rendezvous; and then the targeted resource was published in the corresponding user-generated content hosts using information hiding method.Finally,the security properties were discussed.

Research on orthogonal multi-carrier underwater acoustic communication system based on fractional Fourier transform

Yi-lin WANG,Yun CHEN,Jing-wei YIN,Ping CAI,Yi-meng ZHANG

2012, 33(8):  162-170.  doi:1000-436X(2012)08-0162-09

Asbtract ( 66 )   HTML ( 0)   PDF (884KB) ( 522 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Aimed at the traditional OFDM communication system fails when the channel was frequency-selective,rapidly fading and had the Doppler effects,resulting in inter-carrier interference,a orthogonal multi-carrier system for underwater acoustic communication was proposed,which employed orthogonal signal basis of the chirp type corresponding to the fractional Fourier transform (FRFT) signal basis.The scheme used the orthogonal LFM signal as sub-carrier to modulate,which had same chirp rate but different center frequency with overlapped bandwidth,and used fractional Fourier transform for modulation and demodulation method to transmit information.For suppression of the multi-paths channel effect,the guard interval (GI) was used between signal data frame.And the position correction in fractional domain method combining with QDPSK was proposed to suppress the Doppler effects,which simplified the system complexity.The communication scheme had the characteristic of high communication rate,and the maximum rate was up to 3 600bit/s.The effectiveness and reliability of this system was verified through a large number of simulation and lake experiments.

Research on reconfigurable service carrying network active protection algorithms

Ning QI,Bin-qiang WANG,Zhi-ming WANG

2012, 33(8):  171-179.  doi:1000-436X(2012)08-0171-09

Asbtract ( 48 )   HTML ( 1)   PDF (673KB) ( 212 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

To construct reconfigurable service carrying network (RSCN) in reconfigurable flexible network (RFNet) infrastructure could provide traffic specific communication services for different end users.This could effectively solve the puzzle faced by traditional Internet infrastructure.Because of the blight of network failure,active protection problems of RSCN were discussed.Mathematics model of RSCN active protection issues were established.To avoid enormous influence because of important resource failure,a resource stress factor (RSF) awareness main link construction algorithm named RSF-aware MLCA was proposed.To improve success running ratio and reduce network link failure loss of RSCN,a RSCN protection link construction algorithm named RPLCA was implemented.Considering of the two sub-algorithms,RSCN active protection algorithm named RAPA was designed.The efficiency of algorithms was evaluated by emulation experiments according to RSCN success running ratio and main link utilization ratio and average network link failure loss under several scenarios.

Overview

Research on protection technology for location privacy in VANET

Jian-ming ZHANG,Yu-juan ZHAO,Hao-bin JIANG,Xue-dan JIA,Liang-min WANG

2012, 33(8):  180-189.  doi:1000-436X(2012)08-0180-10

Asbtract ( 80 )   HTML ( 2)   PDF (705KB) ( 840 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

Location-based services in VANET solved the problem of road safety,and provided the convenient services to drivers and passengers.But it also brought new problems of privacy protection.The content of privacy protection was analyzed,and put emphasis on two kinds of privacy protection technology based on pseudonyms and signature respectively.Pseudonym-based schemes were classified as special-area based,silence-period,cryptographic mix-zones and mix-zones communication proxy based schemes.Signature-based schemes were also divided into two classes:group-signature and ring-signature.The evaluation methods for privacy protection level were surveyed,in which anonymity set,entropy metric,mathematical analysis theory and formal analysis were discussed and compared.

Academic communication

Extending structural subsumption reasoning algorithms for ontology matching

Jian-xin LIAO,Xiu-lei LIU,Xiao-min ZHU,Hai-feng SUN,Jing-yu WANG

2012, 33(8):  190-199.  doi:1000-436X(2012)08-0190-10

Asbtract ( 45 )   HTML ( 0)   PDF (784KB) ( 458 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

For exploiting semantics implied in constructors and axioms in an ontology,a structural subsumption reasoning algorithm was extended to match ontologies.The implied semantics was first enabled to be read-off easily by rephrasing entities in an ontology into normal forms; then the syntactical structure of the normal forms were compared to infer correspondences between entities,one from each ontology.The prototype was evaluated over four in use ontologies and compared against twelve methods.The experiments show that its performance is higher than other solutions on average.

Research on mapping method of logical carrying network across multiple domains

Min ZHANG,Chun-ming WU,Bin WANG,Ming JIANG

2012, 33(8):  200-227.  doi:1000-436X(2012)08-0200-08

Asbtract ( 48 )   HTML ( 0)   PDF (638KB) ( 631 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

In order to solve the mapping problem of logical carrying network across multiple domains,a hierarchical linear program model was presented,which aims to minimize the cost of mapping.Based on this model,a mapping algorithm for logical carrying network across multiple domains was designed using primal decomposition and subgradient optimization technique.The correctness of this method was proved by theoretical analysis,and the validity of the proposed method was assessed by a collection of numerical simulation experiments through the performance of convergence and runtime,as well as mapping request acceptance ratio and revenue in a dynamical network environment.

Threshold traceability anonymous authentication scheme without trusted center for ad hoc network

Fang-bin LIU,Kun ZHANG,Hai LI,Hong ZHANG

2012, 33(8):  208-213.  doi:1000-436X(2012)08-0208-06

Asbtract ( 50 )   HTML ( 0)   PDF (590KB) ( 447 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

In order to provide anonymous authentication for ad hoc network,democratic group signatures with centerless secret sharing scheme and proposed a threshold traceability anonymous authentication scheme without trusted center for ad hoc network were combined.The scheme was centerless and self-organized,which was well suited to the characters of ad hoc network and had solved the problem that those schemes of traditional network were not suited to the ad hoc network.The prover of the scheme was anonymous,traceable,and complete,which well satisfied the security of anonymous authentication.

Preliminary study on the connotation of flexibility in dynamically reconfigurable networks

Dong-nian CHENG,Bin-qiang WANG,Bao-jin WANG,Jian-hui ZHANG

2012, 33(8):  214-222.  doi:1000-436X(2012)08-0214-09

Asbtract ( 64 )   HTML ( 0)   PDF (560KB) ( 431 )   Knowledge map   

Figures and Tables | References | Related Articles | Metrics

From the microscopic and quantitative viewpoints,the connotation of the structural flexibility of reconfigurable networks was explored.More accurately,a key feature,i.e.,the dynamic data channel varying with service demands,of reconfigurable networks was first exhibited and explained.Four dominant implications of the flexibility were then revealed as gradually mapping to demands,keeping global performance good,implicit isolation,and self-driven control.Finally,related quantitative models and methods for both determining the stable deviation of outcomes from demands and computing the volume of resource dynamically reconfigured were investigated by using the exponentially moving average,the n:m voting scheme,Markov decision processes (MDP) and reinforcement learning,respectively.