Omnidex Grids are straightforward to create. Each grid node is a separate Omnidex environment, with an environment file, indexes and data. For convenience, there are methods to share a single Omnidex environment file across multiple nodes. There are also methods to share data files across multiple nodes.
Before creating an Omnidex Grid, you must first have a partitioning scheme and a distribution plan. These will be needed during the steps of creating an Omnidex Grid.
Omnidex Grids are straightforward to administer. Each grid node is a separate Omnidex environment, with an environment file, indexes and data. For convenience, there are methods to share a single Omnidex environment file across multiple nodes. There are also methods to share data files across multiple nodes.
Omnidex Grids needs an environment that points to the original database so that the data can be partitioned. This environment is a standard environment that accesses the underlying database.
If the database will be logically partitioned, then create the relational views that correlate with the partitioning strategy. If the database will be physically partitioned using an Omnidex Snapshot, then partition the database using the PARTITION statement.
The PARTITION statement is available in the Omnidex SQL language, and is callable from the OdxSQL application. Since the statement is part of the Omnidex SQL language, it may also be called directly from ODBC and JDBC.
The PARTITION statement allows the table to be partitioned one of three ways, 1) into a specific number of equally sized partitions, 2) into separate partitions based on a partition qualifier consisting of a column or a SQL expression, or 3) a combination of these two steps. In the third case, Omnidex will evaluate the cardinality of the partition qualifier and group values together to comprise the specific number of partitions in as equal size as possible. The syntax of the PARTITION statement is:
PARTITION table [INTO n [PARTITIONS]] [BY column]
[IN path]
[WITH options]
An example of a statement to partition a table is:
PARTITION LIST BY “STATE || SUBSTRING(ZIP from 1 for 2)” IN D:\grid\data
If the database will be logically partitioned, then make sure each grid server has access to the database. If the database will be physically partitioned, then distribute both the partitioned and unpartitioned data to the grid nodes. If each node will be in its own directory, then the data files can be copied or symbolically linked to the correct locations. It is also possible for multiple nodes to share the same physical directories as long as the partitioned data can be differentiated.
The environment file for the Omnidex Grid is a normal environment file; however, it has additional information that describes the locations of the nodes and the partitioning scheme. There are changes that need to be made for the ENVIRONMENT statement, the DATABASE statement and the TABLE statements. Note that older versions of Omnidex Grids had a RULE statement at the bottom, and that is no longer necessary.
The CREATE ENVIRONMENT statement is used to declare the grid nodes and their connection information. The NODE declarations in the CREATE ENVIRONMENT statement are show below. Consult the documentation on the CREATE ENVIRONMENT statement for the complete syntax.
CREATE ENVIRONMENT environment
[MAX_THREADS n]
[NODE node
[<PARTITIONED | UNPARTITIONED>]
[HOST <LOCALHOST | host> [PORT n]]
[ENVIRONMENT “filename”]]
Here is an example of a CREATE ENVIRONMENT statement for an Omnidex Grid.
create environment "list_env" maxthreads 4 node NODE01 partitioned node NODE02 partitioned node NODE03 partitioned node NODE04 partitioned ...
The CREATE DATABASE statement is used to declare the database characteristics for each node. The NODE declarations in the CREATE DATABASE statement are show below. Consult the documentation on the CREATE DATABASE statement for the complete syntax.
CREATE DATABASE database
[NODE node]
TYPE type
[SUBTYPE subtype]
[PHYSICAL "physical"]
[USER “user”]
[PASSWORD “password”]
[INDEX_DIRECTORY “directory”]
[INDEX_MAINTENANCE index_maintenance]
...
Here is an example of a CREATE DATABASE statement for an Omnidex Grid.
create database "list"
node NODE01
type flatfile
index_directory "idx/node01/LIST_"
node NODE02
type flatfile
index_directory "idx/node02/LIST_"
node NODE03
type flatfile
index_directory "idx/node03/LIST_"
node NODE04
type flatfile
index_directory "idx/node04/LIST_"
The CREATE TABLE statement is used to declare the table characteristics for each node for each partitioned table in the grid. The NODE declarations in the CREATE TABLE statement are show below. Consult the documentation on the CREATE TABLE statement for the complete syntax.
CREATE TABLE table
[NODE node]
[OPTIONS “options”]
[PHYSICAL “physical”]
[AUTOFILTER “criteria”]
[DATA_CACHING <DYNAMIC | NONE>]
[PARTITION_BY “criteria”]
[INDEX_MAINTENANCE index_maintenance]
...
Here is an example of a CREATE TABLE statements for an Omnidex Grid.
create table "HOUSEHOLDS"
node NODE01
physical "dat/node01/pro*.dat"
partition_by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
'NY','DE','PA','DC','MD','VA','WV') and
ZIP between '00000' and '26999'"
node NODE02
physical "dat/node02/pro*.dat"
partition_by "STATE in ('NC','SC','GA','FL','AL','MS',
'TN','KY','OH') and
ZIP between '27000' and '45999'"
node NODE03
physical "dat/node03/pro*.dat"
partition_by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
'WI','IL','KS','MO','NE','AR','LA',
'OK','TX') and
ZIP between '46000' and '79999'"
node NODE04
physical "dat/node04/pro*.dat"
partition_by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
'CA','HI','AK','OR','WA') and
ZIP between '80000' and '99999'"
As discussed in the partition scheme, it is necessary to partition tables so that referential integrity is maintained. Typically, the parent is partitioned using criteria, as shown for HOUSEHOLDS above, and the child is partitioned along the foreign key. The PARTITION BY clause for a child shows the join criteria, as shown below:
create table "INDIVIDUALS"
node NODE01
physical "dat/node01/pro*.dat"
partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
node NODE02
physical "dat/node02/pro*.dat"
partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
node NODE03
physical "dat/node03/pro*.dat"
partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
node NODE04
physical "dat/node04/pro*.dat"
partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
It is recommended, though not required, that each node be indexed the same. This reduces the likelihood of queries taking too long because of a lack of indexing on one node.
Indexes can be built in parallel. For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.
Omnidex network services must be started on every machine that contains grid nodes. No special options are required.
See how to configure and start the Omnidex Network Services.
Applications should connect to the grid controller as though it was a standard Omnidex environment.