Differences

This shows you the differences between two versions of the page.

--- admin:grid:creation:oacomp [2009/11/13 18:00]
els
+++ admin:grid:creation:oacomp [2012/10/26 14:27] (current)
@@ Line 1: / Line 1: @@
+<html><div align="center"><span style="color:red">DRAFT</span></div></html>
 ====== Creating an Omnidex Grid using OACOMP ======
@@ Line 4: / Line 6: @@
 :!: //These instructions show how to create Omnidex Grids using environment files that are compiled with OACOMP.  This is an approach that pre-dated the use of SQL statements such as CREATE DATABASE or CREATE TABLE.  Environment files that are compiled with OACOMP are still supported for backward compatability.  These instructions are for Omnidex Version 5.1 and later.  Click [[admin:grid:creation:rules|here]] for instructions for Omnidex Version 5.0.//
+----
+\\
 Omnidex Grids are straightforward to administer.  Each grid node is a separate Omnidex environment, with an environment file, indexes and data.  For convenience, there are methods to share a single Omnidex environment file across multiple nodes.  There are also methods to share data files across multiple nodes.
@@ Line 10: / Line 14: @@
 These instructions describe how to create an Omnidex Grid using Environment Rules.  This was the original method to create Omnidex Grids.  This method has now been replace by use of the [[admin:grid:creation:odxadmin| Omnidex Administrator]] or [[admin:grid:creation:sql | SQL statements]] such as the CREATE DATABASE and CREATE TABLE.
-===== Steps to create an Omnidex Grid using Environment Rules =====
+===== Steps to create an Omnidex Grid using OACOMP =====
 ==== 1. Create an environment pointing to the original database ====
@@ Line 43: / Line 47: @@
 === The ENVIRONMENT statement ===
-The ENVIRONMENT statement is expanded to declare the grid nodes and their connection information.  The syntax for NODE declarations in the ENVIRONMENT statement is:
+The ENVIRONMENT statement is expanded to declare the grid nodes and their connection information.  The NODE declarations in the ENVIRONMENT statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#environment_statement | ENVIRONMENT statement]] for the complete syntax.
 <code>
@@ Line 52: / Line 56: @@
      [PORT n]
      [ENVIRONMENT "filename"]
-     [OPTIMIZATION "options"]
+     [OPTIONS "options"]
 </code>
-  NODE node
+Here is an example of an ENVIRONMENT declaration on an Omnidex Grid.
-    The logical name for the node.
-  <PARTITIONED | UNPARTITIONED>
-    A partitioned node is one in which some of the tables have been partitioned.  This implies that this node does not have a complete copy of the database.
-  HOST "host"
-    The name of the host containing this node.  If the node is on the current host, this may be omitted, or "localhost" may be used.
-  PORT "host"
-    The port number used to connect to this host using Omnidex Network Services.  If using the default port of 7555, this may be omitted.
-  ENVIRONMENT "filename"
-    The name of the Environment File for this node.  If the node is on the current host, this may be omitted.
-  OPTIMIZATION "options"
-    The options used in optimizing this node.  At present, no options are supported.
-Here is an example of an ENVIRONMENT declaration with several nodes:
 <code>
@@ Line 94: / Line 73: @@
 === The DATABASE statement ===
-The DATABASE statement is expanded to show the database characteristics for each node.   The syntax for the NODE declarations for the DATABASE statement is:
+The DATABASE statement is expanded to show the database characteristics for each node. The NODE declarations in the DATABASE statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#database_statement | DATABASE statement]] for the complete syntax.
 <code>
@@ Line 108: / Line 87: @@
 </code>
-:!: Note that environment variables are sometimes allowed in the syntax.  The following example below uses the automatic environment variable "OMNIDEX_NODE" in the index prefix for the database.  The variable $OMNIDEX_NODE will be replaced by the node name.
+Here is an example of a DATABASE statement on an Omnidex Grid.
 <code>
@@ Line 114: / Line 93: @@
   node NODE01
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node01/LIST_}"
   node NODE02
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node02/LIST_}"
   node NODE03
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node03/LIST_}"
   node NODE04
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node04/LIST_}"
 </code>
+=== The TABLE statement ===
-==== 4. Install and build Omnidex indexes on each node. ====
+The TABLE statement is expanded for each partitioned table to show the table characteristics for each node. The NODE declarations in the TABLE statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#table_statement | TABLE statement]] for the complete syntax.
-It is recommended, though not required, that each node be indexed the same.  This reduces the likelihood of queries taking too long because of a lack of indexing on one node.
+<code>
+[TABLE table
+  [NODE node]
+    [TYPE table_type_spec]
+    [PHYSICAL "filespec"]
+    [OPTIONS "options"]
+    [CARDINALITY n]
+    [INDEXMAINTENANCE <AUTOMATIC | MANUAL | NONE>]
+    [DATA CACHING <NONE | DYNAMIC>]
+    [PARTITIONED BY "sql_predicates"]
-Indexes can be built in parallel.  For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.
+</code>
+Here is an example of TABLE statements on an Omnidex Grid.
-==== 5. Create an environment file for the grid controller. ====
+<code>
+table            "HOUSEHOLDS"
+  node NODE01
+    physical     "{dat/node01/pro*.dat}"
+    partition by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
+                            'NY','DE','PA','DC','MD','VA','WV') and
+                  ZIP between '00000' and '26999'"
-The environment file for the grid controller has the same database and table layout as the grid nodes.  The physical clauses for the tables can be omitted on the grid controller since the controller only access the tables through the nodes.
+  node NODE02
+    physical     "{dat/node02/pro*.dat}"
+    partition by "STATE in ('NC','SC','GA','FL','AL','MS',
+                            'TN','KY','OH') and
+                  ZIP between '27000' and '45999'"
+  node NODE03
+    physical     "{dat/node03/pro*.dat}"
+    partition by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
+                            'WI','IL','KS','MO','NE','AR','LA',
+                            'OK','TX') and
+                  ZIP between '46000' and '79999'"
-At the bottom of the environment file, include a rule that describes the Omnidex Grid.  This rule declares the nodes and partitioning scheme used in the Omnidex grid, as well as naming the columns that are guaranteed to have distinct values on each node (discussed earlier).
+  node NODE04
+    physical     "{dat/node04/pro*.dat}"
+    partition by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
+                            'CA','HI','AK','OR','WA') and
+                  ZIP between '80000' and '99999'"
+</code>
-The syntax for the grid rule is:
+As discussed in the [[admin:grid:partitions#how_are_referential_constraints_handled_such_as_primary_and_foreign_keys| partition scheme]], it is necessary to partition tables so that referential integrity is maintained.  Typically, the parent is partitioned using criteria, as shown for HOUSEHOLDS above, and the child is partitioned along the foreign key.  The PARTITION BY clause for a child shows the join criteria, as shown below:
 <code>
-  RULE dbname_GRID
+table            "INDIVIDUALS"
-  {
+  node NODE01
-    MAX_CONCURRENT_NODES 	n
+    physical     "{dat/node01/pro*.dat}"
-    UNIQUE_TO_NODE 		“table.column, table.column, …”
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    NODE node_name 		[HOST name] [PORT n] [ENVIRONMENT filename]
-      DATABASE database_name
+  node NODE02
-          TABLE table_name PARTITION BY “partition_clause”
+    physical     "{dat/node02/pro*.dat}"
-  }
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-</code>
-An example of a grid rule is:
+  node NODE03
+    physical     "{dat/node03/pro*.dat}"
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-<code>
+  node NODE04
+    physical     "{dat/node04/pro*.dat}"
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
+</code>
-  rule LIST_GRID
-  {
-    max_concurrent_nodes = 4
-    unique_to_node "HOUSEHOLDS.HOUSEHOLD"
-    unique_to_node "HOUSEHOLDS.STATE"
-    unique_to_node "HOUSEHOLDS.ZIP"
-    unique_to_node "INDIVIDUALS.HOUSEHOLD"
-    unique_to_node "INDIVIDUALS.INDIVIDUAL"
-    unique_to_node "INDIVIDUALS.SSN"
-    node node01
+==== 5. Install and build Omnidex indexes on each node. ====
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
-                                  'NY','DE','PA','DC','MD','VA','WV') and
-                        ZIP between '00000' and '26999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node02
+It is recommended, though not required, that each node be indexed the same.  This reduces the likelihood of queries taking too long because of a lack of indexing on one node.
-      database LIST
-        table HOUSEHOLDS
+Indexes can be built in parallel.  For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.
-          partition by "STATE in ('NC','SC','GA','FL','AL','MS',
-                                  'TN','KY','OH') and
-                        ZIP between '27000' and '45999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node03
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
-                                  'WI','IL','KS','MO','NE','AR','LA',
-                                  'OK','TX') and
-                        ZIP between '46000' and '79999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node04
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
-                                  'CA','HI','AK','OR','WA') and
-                        ZIP between '80000' and '99999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-  }
-</code>
 ==== 6. Start Omnidex Network Services on each server. ====
@@ Line 210: / Line 185: @@
 Omnidex network services must be started on every machine that contains grid nodes.  No special options are required.
-See how to [[admin:network:overview | configure and start the Omnidex Network Services (OdxNet)]].
+See how to configure and start the [[admin:network:overview | Omnidex Network Services]].

Back to top

admin/grid/creation/oacomp.1258135233.txt.gz · Last modified: 2012/10/26 14:26 (external edit)

Sitemap Recent Changes