Disclosed herein is a Hybrid Coaxial Fiber (HCF) network-based high-speed Quality of Service (QoS) transmission system for IP broadcasting service. In the system, a L3 switch unit, a CMTS and a coupler are provided on an ISP side, and an optical distributor, a cable modem, and IP set-top box are provided on an Internet service subscriber side. The transmission system includes one or more IP-QAM transmitters and an IP-QAM modem. The IP-QAM transmitters are connected between the L3 switch unit and coupler on the ISP side in parallel with the CMTS, and modulate the phases and amplitudes of IP broadcasting service data transmission signals into carrier signals different from Internet service data transmission signals and cable TV signals. The IP-QAM modem is connected to the coupler in parallel with the cable modem, and demodulates the IP broadcasting service data and transfer the demodulated IP broadcasting service data to the IP set-top box.