Data Warehouse Appliance - Appliance Technology

Appliance Technology

Most DW appliance vendors use massively parallel processing (MPP) architectures to provide high query performance and platform scalability. MPP architectures consist of independent processors or servers executing in parallel. Most MPP architectures implement a "shared-nothing architecture" where each server operates self-sufficiently and controls its own memory and disk. Shared-nothing architectures have a proven record for high scalability and little contention. DW appliances distribute data onto dedicated disk storage units connected to each server in the appliance. This distribution allows DW appliances to resolve a relational query by scanning data on each server in parallel. The divide-and-conquer approach delivers high performance and scales linearly as new servers are added into the architecture. Other DW appliance vendors use specialized hardware and advanced software, instead of MPP architectures. This approach is able to achieve MPP performance in a much smaller form factor. The first vendor to market with a data warehouse appliance featuring specialized SQL hardware was Netezza in 2003 through leveraging FPGA technology as sophisticated projection and restriction filters, minimizing data movement and I/O within the system. Kickfire followed in 2008 with what they deem a dataflow "sql chip".

MPP database architectures have a long pedigree. Teradata, Tandem, Britton Lee, and Sequent offered MPP SQL-based architectures in the 1980s. Open source and commodity components have aided a re-emergence of MPP data warehouses. Advances in technology have reduced costs and improved performance in storage devices, multi-core CPUs and networking components. Open-source RDBMS products, such as Ingres and PostgreSQL, reduce software-license costs and allow DW-appliance vendors to focus on optimization rather than providing basic database functionality. Open-source Linux provides a stable, well-implemented operating system for DW appliances.