Design | Prototype | Development | Testing | Staging | Production | Optimization
Administrators often create a prototype before starting development. Usually, administrators create a smaller test database, and then create Omnidex indexes and run sample queries. Omnidex indexes can also be created on the production database if this is preferable. Prototypes enable businesses to gauge the performance they will see and insure that they have a good indexing strategy.
Prototyping an Omnidex application has four steps:
Administrators usually create a small test database, often using an Omnidex Snapshot. This test database is useful for optimizing queries and predicting performance. Since Omnidex Snapshots reside in flat files, they are highly portable and can be moved to development servers. They do not require licenses for the relational database software, nor do they need the support of database administrators. This allows Omnidex prototyping to occur quickly and easily, without affecting the main data server.
Omnidex Snapshots can be built on a subset of the database. At this stage of analyzing queries, the database does not need more than a few thousand rows. A smaller database allows quick changes to the indexing structure as queries are tuned, avoiding the complications of indexing the main database. Omnidex Snapshots can be built using the Omnidex Administrator. For more information on Omnidex Snapshots, refer to the article on Omnidex Snapshots.
The production database can be indexed directly as long as the development and testing process does not interfere with regular operations. If the database contains over 20 million rows, it is a candidate for an Omnidex Grid; therefore, a smaller database would be appropriate for prototyping.
Omnidex databases are described in an Omnidex Environment. Omnidex Environments store information about the underlying database, the Omnidex indexing and other configuration settings. Omnidex Environments can then be connected to a variety of tools and API's.
Omnidex Environments are created interactively using the Omnidex Administrator, or are created with SQL statements such as CREATE ENVIRONMENT, CREATE DATABASE, CREATE TABLE and CREATE INDEX. Both of these approaches will create an XML file that contains all of the meta-data describing an Omnidex-enhanced database.
The Omnidex Environments section of the documentation discusses how to create and connect to Omnidex Environments.
Omnidex indexes are first declared in the Omnidex Environment, and then all indexes are built at the same time. This approach allows Omnidex to build indexes substantially faster than relational databases. With relational databases, the CREATE INDEX statement declares the index and starts a sequential scan of the table in order to build the index. When creating a large number of indexes, this approach causes an excessive load on the database by re-reading the table for each index. With Omnidex, indexes are declared without being built and then all indexes are built with one sequential scan of the table.
Omnidex indexes can be declared interactively using the Omnidex Administrator, or can be declared using the CREATE INDEX statement.
Omnidex indexes can be built interactively using the Omnidex Administrator, or can be built using the UPDATE INDEXES statement.
The documentation on Index Creation discusses how to create and build Omnidex indexes.
Omnidex applications remain fast when the majority of queries are well optimized. This allows resources to be quickly used and then released for the next query. If enough queries run for a long time, they can consume enough CPU, memory and disk access to degrade the overall performance of the machine. The goal at this stage is to insure that most, if not all, of the queries are well optimized.
Queries are optimized by running them against a test database and evaluating the query plan. The query plan shows each access to an Omnidex index and shows how each aspect of the query is processed. The query plan also shows warnings and notes indicating areas where optimization can be improved.
Each time the indexing strategy is changed, it is worth repeating this process to insure that all queries remain well optimized. Again, this points to the value of doing this process on a small version of the database.
Query optimization is done using the Omnidex Administrator or using OdxSQL. Administrators should learn how to analyze query plans, and are encouraged to read the article on Optimizing Queries.
Once the queries are optimized, a broader prototype can be created that engages any client-side functionality. Omnidex is most commonly accessed using standard ODBC and JDBC drivers. Omnidex provides its own Network Services optimized to Omnidex's needs.
Application architectures often contain many other layers such as analytic tools, bus interfaces, intermediate languages, middleware tools, load balancers, connection poolers, etc. Omnidex will integrate by simply pointing to the Omnidex ODBC or JDBC driver rather than the relational database's ODBC or JDBC driver. Some tailoring may be needed in the application layers to take advantage of Omnidex's extended functionality, such as fuzzy searches or geographic searches.
The Application Integration section of the documentation discusses the process of connecting Omnidex to the client-side tools and integrating with some of the more popular analytic tools.
See also: