This dissertation provides an in depth analysis of the existing security threats which are call disruption attacks and flooding attacks in SIP(Session Initiation Protocol) based VoIP(Voice over IP) systems. Also it discusses the goals and requirements of detection schemes for reliable SIP based VoIP systems. This dissertation presents various enhanced detection schemes against such attacks as the systems degrade QoS(Quality of Service). The first scheme is detection of SIP flooding attacks based on the upper bound of the possible number of SIP messages, which is an effective detection method for SIP flooding attacks in order to deal with the problems of conventional schemes. We derive the upper bound of the possible number of SIP messages, considering not only the network congestion status but also the different properties of individual SIP messages such as INVITE, BYE and CANCEL. This method can be easily extended to detect flooding attacks by other SIP messages. However, such attacks cannot be easily classified the attack signatures, since they may be frequently modified and newly created. That is, it makes difficult to provide the additional countermeasure scheme after detecting the attacks. Therefore, we also propose bloom filter based SIP flooding attack detection scheme. This scheme utilizes bloom filter for classifying SIP flooding attacks whose attack signatures is defined according to the modulated message pattern. In case of call disruption attacks, we propose an effective detection method for those attack(CANCEL, BYE or REGISTER attack) without authentication or encryption schemes. In order to achieve that, Extended INFO method is utilized to deal with the security threats and can be applied in both pre-call and mid-call VoIP mobility environments without additional functions or systems. The performance of our proposed schemes is evaluated in terms of attack detection time, system resource cost(Memory, CPU consumption and so on) and effectiveness of the schemes both in simulation and analytically.
