We propose high order conforming and nonconforming immersed hybridized difference (IHD) methods in two and three dimensions for elliptic interface problems. Introducing the virtual to real transformation (VRT), we could obtain a systematic and unique way of deriving arbitrary high order methods in principle. The optimal number of collocating points for imposing interface conditions is proved, and a unique way of constructing the VRT is suggested. Numerical experiments are performed in two and three dimensions. Numerical results achieving up to the 6th order convergence in the L2-norm are presented for the two dimensional case, and a three dimensional example with a 4th order convergence is presented.