Abstract: In order to improve the spectral efficiency and energy efficiency of asymmetry, random-polar and very minimum amplitude and phase (ARP-VMAP) modulated signal simultaneously, an M-ary position and three-valued VMAP modulation was presented, and the receiver based on BP neural network or support vector machine demodulator was also designed. The contrast and analysis of the power spectrum, the ?60 dB bandwidth, the spectral efficiency, and the demodulation performance among modulations as the ARP-VMAP, the three-valued VMAP, the M-ary position ARP-VMAP, and the M-ary position and three-valued VMAP, were carried out. Simulation results show that the new modem not only raise spectral efficiency, but also improve demodulation performance significantly.