Database - History - Evolution of Database and DBMS Technology

Evolution of Database and DBMS Technology

See also Database management system#History

The introduction of the term database coincided with the availability of direct-access storage (disks and drums) from the mid-1960s onwards. The term represented a contrast with the tape-based systems of the past, allowing shared interactive use rather than daily batch processing.

In the earliest database systems, efficiency was perhaps the primary concern, but it was already recognized that there were other important objectives. One of the key aims was to make the data independent of the logic of application programs, so that the same data could be made available to different applications.

In the period since the 1970s database technology has kept pace with the increasing resources becoming available from the computing platform: notably the rapid increase in affordable capacity and speed of disk storage, and of main memory. This has enabled ever larger databases and higher throughput to be achieved.

The first generation of general-purpose database systems were navigational, applications typically accessed data by following pointers from one record to another. The two main data models at this time were the hierarchical model, epitomized by IBM's IMS system, and the Codasyl model (Network model), implemented in a number of products such as IDMS.

The relational model, first proposed in 1970 by Edgar F. Codd, departed from this tradition by insisting that applications should search for data by content, rather than by following links. This was considered necessary to allow the content of the database to evolve without constant rewriting of links and pointers. The relational model is made up of ledger-style tables, each used for a different type of entity. Data may be freely inserted, deleted and edited in these tables, with the DBMS (DataBase Management System) doing whatever maintenance needed to present a table view to the application/user. The relational part comes from entities referencing other entities in what is known as one-to-many relationship, like a traditional hierarchical model, and many-to-many relationship, like a navigational (network) model. Thus, a relational model can express both hierarchical and navigational models, as well as its native tabular model, allowing for pure or combined modeling in terms of these three models, as the application requires.

The earlier expressions of the relational model did not make relationships between different entities explicit in the way practitioners were used to back then, but as primary keys and foreign keys. These keys, though, can be also seen as pointers in their own right, stored in tabular form. This use of keys rather than pointers conceptually obscured relations between entities, at least the way it was presented back then. Thus, the wisdom at the time was that the relational model emphasizes search rather than navigation, and that it was a good conceptual basis for a query language, but less well suited as a navigational language. As a result, another data model, the entity-relationship model which emerged shortly later (1976), gained popularity for database design, as it emphasized a more familiar description than the earlier relational model. Later on, entity-relationship constructs were retrofitted as a data modeling construct for the relational model, and the difference between the two have become irrelevant.

Earlier relational system implementations lacked the sophisticated automated optimizations of conceptual elements and operations versus their physical storage and processing counterparts, present in modern DBMSs (DataBase Management Systems), so their simplistic and literal implementations placed heavy demands on the limited processing resources at the time. It was not until the mid 1980s that computing hardware became powerful enough to allow relational systems (DBMSs plus applications) to be widely deployed. By the early 1990s, however, relational systems were dominant for all large-scale data processing applications, and they remain dominant today (2012) except in niche areas. The dominant database language is the standard SQL for the Relational model, which has influenced database languages for other data models.

The rigidity of the relational model, in which all data are held in related tables with a fixed structure of rows and columns, has increasingly been seen as a limitation when handling information that is richer or more varied in structure than the traditional 'ledger-book' data of corporate information systems. These limitations come to play when modeling document databases, engineering databases, multimedia databases, or databases used in the molecular sciences.

Most of that rigidity, though, is due to the need to represent new data types other than text and text-alikes within a relational model. Examples of unsupported data types are:

• graphics (and operations such as pattern-matching and OCR)
• Multidimensional constructs such as 2D (geographical), 3D (geometrical), and multidimensional hypercube models (data analysis).
• XML (an hierarchical data modeling technology evolved from EDS and HTML), used for data interchange among dissimilar systems.

More fundamental conceptual limitations came with Object Oriented methodologies, with their emphasis on encapsulating data and processes (methods), as well as expressing constructs such as events or triggers. Traditional data modeling constructs emphasize the total separation of data from processes, though modern DBMS do allow for some limited modeling in terms of validation rules and stored procedures.

Various attempts have been made to address this problem, many of them banners such as post-relational or NoSQL. Two developments of note are the object database and the XML database. The vendors of relational databases have fought off competition from these newer models by extending the capabilities of their own products to support a wider variety of data types.