These instructions show how to create Omnidex Grids using environment files that are compiled with OACOMP. This is an approach that pre-dated the use of SQL statements such as CREATE DATABASE or CREATE TABLE. Environment files that are compiled with OACOMP are still supported for backward compatability. These instructions are for Omnidex Version 5.1 and later. Click here for instructions for Omnidex Version 5.0.

Omnidex Grids are straightforward to administer. Each grid node is a separate Omnidex environment, with an environment file, indexes and data. For convenience, there are methods to share a single Omnidex environment file across multiple nodes. There are also methods to share data files across multiple nodes.

Before creating an Omnidex Grid, you must first have a partitioning scheme and a distribution plan. These will be needed during the steps of creating an Omnidex Grid.

These instructions describe how to create an Omnidex Grid using Environment Rules. This was the original method to create Omnidex Grids. This method has now been replace by use of the Omnidex Administrator or SQL statements such as the CREATE DATABASE and CREATE TABLE.

Omnidex Grids needs an environment that points to the original database so that the data can be partitioned. This environment is a standard environment that accesses the underlying database.

If the database will be logically partitioned, then create the relational views that correlate with the partitioning strategy. If the database will be physically partitioned using an Omnidex Snapshot, then partition the database using the PARTITION command.

The PARTITION command is available in the Omnidex SQL language, and is callable from the OdxSQL application. Since the command is in the Omnidex SQL language, it may also be called directly from ODBC and JDBC.

The PARTITION command allows the table to be partitioned one of three ways, 1) into a specific number of equally sized partitions, 2) into separate partitions based on a partition qualifier consisting of a column or a SQL expression, or 3) a combination of these two steps. In the third case, Omnidex will evaluate the cardinality of the partition qualifier and group values together to comprise the specific number of partitions in as equal size as possible. The syntax of the PARTITION command is:

PARTITION table [INTO n [PARTITIONS]] [BY column] 
                [IN path] 
                [WITH options]

An example of a command to partition a table is:

PARTITION LIST BY “STATE || SUBSTRING(ZIP from 1 for 2)” 
  IN D:\grid\data

If the database will be logically partitioned, then make sure each grid server has access to the database. If the database will be physically partitioned, then distribute both the partitioned and unpartitioned data to the grid nodes. If each node will be in its own directory, then the data files can be copied or symbolically linked to the correct locations. It is also possible for multiple nodes to share the same physical directories as long as the partitioned data can be differentiated.

The environment file for the Omnidex Grid is a normal environment file; however, it has additional information that describes the locations of the nodes and the partitioning scheme. There are changes that need to be made for the ENVIRONMENT statement, the DATABASE statement and the TABLE statements. Note that older versions of Omnidex Grids had a RULE statement at the bottom, and that is no longer necessary.

The ENVIRONMENT statement is expanded to declare the grid nodes and their connection information. The syntax for NODE declarations in the ENVIRONMENT statement is:

ENVIRONMENT environment
  [NODE node]
     [<PARTITIONED | UNPARTITIONED>]
     [HOST "host"]
     [PORT n]
     [ENVIRONMENT "filename"]
     [OPTIMIZATION "options"]

  
The logical name for the node.

A partitioned node is one in which some of the tables have been partitioned.  This implies that this node does not have a complete copy of the database.

HOST “host”

The name of the host containing this node.  If the node is on the current host, this may be omitted, or "localhost" may be used.

PORT “host”

The port number used to connect to this host using Omnidex Network Services.  If using the default port of 7555, this may be omitted.

ENVIRONMENT “filename”

The name of the Environment File for this node.  If the node is on the current host, this may be omitted.

OPTIMIZATION “options”

The options used in optimizing this node.  At present, no options are supported.

Here is an example of an ENVIRONMENT declaration with several nodes:

environment  "list_env"
  maxthreads  4
  node NODE01 partitioned
  node NODE02 partitioned
  node NODE03 partitioned 
  node NODE04 partitioned

The DATABASE statement is expanded to show the database characteristics for each node. The syntax for the NODE declarations for the DATABASE statement is:

[DATABASE database
  [NODE node]                   
    TYPE database_type_spec
    [VERSION “version”]
    [PHYSICAL “filespec”]
    [INDEXPREFIX “filespec”]
    [OPTIONS “options”]
    [USERCLASS database_access_definition [, database_access_definition ...]]

Note that environment variables are sometimes allowed in the syntax. The following example below uses the automatic environment variable “OMNIDEX_NODE” in the index prefix for the database. The variable $OMNIDEX_NODE will be replaced by the node name.

database       "list"
  node NODE01
    type         flatfile
    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
  node NODE02
    type         flatfile
    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
  node NODE03
    type         flatfile
    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
  node NODE04
    type         flatfile
    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"

It is recommended, though not required, that each node be indexed the same. This reduces the likelihood of queries taking too long because of a lack of indexing on one node.

Indexes can be built in parallel. For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.

The environment file for the grid controller has the same database and table layout as the grid nodes. The physical clauses for the tables can be omitted on the grid controller since the controller only access the tables through the nodes.

At the bottom of the environment file, include a rule that describes the Omnidex Grid. This rule declares the nodes and partitioning scheme used in the Omnidex grid, as well as naming the columns that are guaranteed to have distinct values on each node (discussed earlier).

The syntax for the grid rule is:

  RULE dbname_GRID
  {
    MAX_CONCURRENT_NODES 	n
    UNIQUE_TO_NODE 		“table.column, table.column, …”
    NODE node_name 		[HOST name] [PORT n] [ENVIRONMENT filename] 
      DATABASE database_name
          TABLE table_name PARTITION BY “partition_clause”
  }

An example of a grid rule is:

  rule LIST_GRID
  {
    max_concurrent_nodes = 4
    unique_to_node "HOUSEHOLDS.HOUSEHOLD"
    unique_to_node "HOUSEHOLDS.STATE"
    unique_to_node "HOUSEHOLDS.ZIP"
    unique_to_node "INDIVIDUALS.HOUSEHOLD"
    unique_to_node "INDIVIDUALS.INDIVIDUAL"
    unique_to_node "INDIVIDUALS.SSN"

    node node01
      database LIST 
        table HOUSEHOLDS 
          partition by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
                                  'NY','DE','PA','DC','MD','VA','WV') and 
                        ZIP between '00000' and '26999'"
        table INDIVIDUALS 
          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"

    node node02
      database LIST 
        table HOUSEHOLDS 
          partition by "STATE in ('NC','SC','GA','FL','AL','MS',
                                  'TN','KY','OH') and 
                        ZIP between '27000' and '45999'"
        table INDIVIDUALS 
          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"

    node node03
      database LIST 
        table HOUSEHOLDS 
          partition by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
                                  'WI','IL','KS','MO','NE','AR','LA',
                                  'OK','TX') and 
                        ZIP between '46000' and '79999'"
        table INDIVIDUALS 
          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
  
    node node04
      database LIST 
        table HOUSEHOLDS 
          partition by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
                                  'CA','HI','AK','OR','WA') and 
                        ZIP between '80000' and '99999'"
        table INDIVIDUALS 
          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
  } 

Omnidex network services must be started on every machine that contains grid nodes. No special options are required.

See how to configure and start the Omnidex Network Services (OdxNet).

Applications should connect to the grid controller as though it was a standard Omnidex environment.