Differences

This shows you the differences between two versions of the page.

--- admin:grid:creation:oacomp [2009/11/13 18:49]
els
+++ admin:grid:creation:oacomp [2012/10/26 14:27] (current)
@@ Line 1: / Line 1: @@
+<html><div align="center"><span style="color:red">DRAFT</span></div></html>
 ====== Creating an Omnidex Grid using OACOMP ======
@@ Line 12: / Line 14: @@
 These instructions describe how to create an Omnidex Grid using Environment Rules.  This was the original method to create Omnidex Grids.  This method has now been replace by use of the [[admin:grid:creation:odxadmin| Omnidex Administrator]] or [[admin:grid:creation:sql | SQL statements]] such as the CREATE DATABASE and CREATE TABLE.
-===== Steps to create an Omnidex Grid using Environment Rules =====
+===== Steps to create an Omnidex Grid using OACOMP =====
 ==== 1. Create an environment pointing to the original database ====
@@ Line 45: / Line 47: @@
 === The ENVIRONMENT statement ===
-The [[admin:environments:create:oacomp:syntax#environment_statement | ENVIRONMENT statement]] is expanded to declare the grid nodes and their connection information.  The syntax for NODE declarations in the ENVIRONMENT statement is:
+The ENVIRONMENT statement is expanded to declare the grid nodes and their connection information.  The NODE declarations in the ENVIRONMENT statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#environment_statement | ENVIRONMENT statement]] for the complete syntax.
 <code>
@@ Line 57: / Line 59: @@
 </code>
-Here is an example of an ENVIRONMENT declaration with several nodes:
+Here is an example of an ENVIRONMENT declaration on an Omnidex Grid.
 <code>
@@ Line 71: / Line 73: @@
 === The DATABASE statement ===
-The DATABASE statement is expanded to show the database characteristics for each node.   The syntax for the NODE declarations for the DATABASE statement is:
+The DATABASE statement is expanded to show the database characteristics for each node. The NODE declarations in the DATABASE statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#database_statement | DATABASE statement]] for the complete syntax.
 <code>
@@ Line 85: / Line 87: @@
 </code>
-:!: Note that environment variables are sometimes allowed in the syntax.  The following example below uses the automatic environment variable "OMNIDEX_NODE" in the index prefix for the database.  The variable $OMNIDEX_NODE will be replaced by the node name.
+Here is an example of a DATABASE statement on an Omnidex Grid.
 <code>
@@ Line 91: / Line 93: @@
   node NODE01
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node01/LIST_}"
   node NODE02
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node02/LIST_}"
   node NODE03
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node03/LIST_}"
   node NODE04
     type         flatfile
-    indexprefix  "{idx/$OMNIDEX_NODE/LIST_}"
+    indexprefix  "{idx/node04/LIST_}"
 </code>
+=== The TABLE statement ===
-==== 4. Install and build Omnidex indexes on each node. ====
+The TABLE statement is expanded for each partitioned table to show the table characteristics for each node. The NODE declarations in the TABLE statement are show below.  Consult the documentation on the [[admin:environments:create:oacomp:syntax#table_statement | TABLE statement]] for the complete syntax.
-It is recommended, though not required, that each node be indexed the same.  This reduces the likelihood of queries taking too long because of a lack of indexing on one node.
+<code>
+[TABLE table
+  [NODE node]
+    [TYPE table_type_spec]
+    [PHYSICAL "filespec"]
+    [OPTIONS "options"]
+    [CARDINALITY n]
+    [INDEXMAINTENANCE <AUTOMATIC | MANUAL | NONE>]
+    [DATA CACHING <NONE | DYNAMIC>]
+    [PARTITIONED BY "sql_predicates"]
-Indexes can be built in parallel.  For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.
+</code>
+Here is an example of TABLE statements on an Omnidex Grid.
-==== 5. Create an environment file for the grid controller. ====
+<code>
+table            "HOUSEHOLDS"
+  node NODE01
+    physical     "{dat/node01/pro*.dat}"
+    partition by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
+                            'NY','DE','PA','DC','MD','VA','WV') and
+                  ZIP between '00000' and '26999'"
-The environment file for the grid controller has the same database and table layout as the grid nodes.  The physical clauses for the tables can be omitted on the grid controller since the controller only access the tables through the nodes.
+  node NODE02
+    physical     "{dat/node02/pro*.dat}"
+    partition by "STATE in ('NC','SC','GA','FL','AL','MS',
+                            'TN','KY','OH') and
+                  ZIP between '27000' and '45999'"
+  node NODE03
+    physical     "{dat/node03/pro*.dat}"
+    partition by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
+                            'WI','IL','KS','MO','NE','AR','LA',
+                            'OK','TX') and
+                  ZIP between '46000' and '79999'"
-At the bottom of the environment file, include a rule that describes the Omnidex Grid.  This rule declares the nodes and partitioning scheme used in the Omnidex grid, as well as naming the columns that are guaranteed to have distinct values on each node (discussed earlier).
+  node NODE04
+    physical     "{dat/node04/pro*.dat}"
+    partition by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
+                            'CA','HI','AK','OR','WA') and
+                  ZIP between '80000' and '99999'"
+</code>
-The syntax for the grid rule is:
+As discussed in the [[admin:grid:partitions#how_are_referential_constraints_handled_such_as_primary_and_foreign_keys| partition scheme]], it is necessary to partition tables so that referential integrity is maintained.  Typically, the parent is partitioned using criteria, as shown for HOUSEHOLDS above, and the child is partitioned along the foreign key.  The PARTITION BY clause for a child shows the join criteria, as shown below:
 <code>
-  RULE dbname_GRID
+table            "INDIVIDUALS"
-  {
+  node NODE01
-    MAX_CONCURRENT_NODES 	n
+    physical     "{dat/node01/pro*.dat}"
-    UNIQUE_TO_NODE 		“table.column, table.column, …”
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    NODE node_name 		[HOST name] [PORT n] [ENVIRONMENT filename]
-      DATABASE database_name
+  node NODE02
-          TABLE table_name PARTITION BY “partition_clause”
+    physical     "{dat/node02/pro*.dat}"
-  }
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-</code>
-An example of a grid rule is:
+  node NODE03
+    physical     "{dat/node03/pro*.dat}"
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-<code>
+  node NODE04
+    physical     "{dat/node04/pro*.dat}"
+    partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
+</code>
-  rule LIST_GRID
-  {
-    max_concurrent_nodes = 4
-    unique_to_node "HOUSEHOLDS.HOUSEHOLD"
-    unique_to_node "HOUSEHOLDS.STATE"
-    unique_to_node "HOUSEHOLDS.ZIP"
-    unique_to_node "INDIVIDUALS.HOUSEHOLD"
-    unique_to_node "INDIVIDUALS.INDIVIDUAL"
-    unique_to_node "INDIVIDUALS.SSN"
-    node node01
+==== 5. Install and build Omnidex indexes on each node. ====
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('MA','ME','NH','NY','PR','RI','VT',
-                                  'NY','DE','PA','DC','MD','VA','WV') and
-                        ZIP between '00000' and '26999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node02
+It is recommended, though not required, that each node be indexed the same.  This reduces the likelihood of queries taking too long because of a lack of indexing on one node.
-      database LIST
-        table HOUSEHOLDS
+Indexes can be built in parallel.  For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible.
-          partition by "STATE in ('NC','SC','GA','FL','AL','MS',
-                                  'TN','KY','OH') and
-                        ZIP between '27000' and '45999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node03
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('IN','MI','IA','MN','MT','ND','SD',
-                                  'WI','IL','KS','MO','NE','AR','LA',
-                                  'OK','TX') and
-                        ZIP between '46000' and '79999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-    node node04
-      database LIST
-        table HOUSEHOLDS
-          partition by "STATE in ('AZ','CO','ID','NM','NV','UT','WY',
-                                  'CA','HI','AK','OR','WA') and
-                        ZIP between '80000' and '99999'"
-        table INDIVIDUALS
-          partition by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD"
-  }
-</code>
 ==== 6. Start Omnidex Network Services on each server. ====
@@ Line 187: / Line 185: @@
 Omnidex network services must be started on every machine that contains grid nodes.  No special options are required.
-See how to [[admin:network:overview | configure and start the Omnidex Network Services (OdxNet)]].
+See how to configure and start the [[admin:network:overview | Omnidex Network Services]].

Back to top

admin/grid/creation/oacomp.1258138199.txt.gz · Last modified: 2012/10/26 14:26 (external edit)

Sitemap Recent Changes