* [[topbar |Table of Contents]] * [[intro:overview:what|Introduction]] * [[install:home|Installation]] * [[install:editions|Product Editions]] * [[install:requirements|System Requirements]] * [[install:platforms|Supported Platforms]] * [[install:guides:home|Installation Guides]] * [[install:licensing:home|Licensing]] * [[install:releases:home|Release Notes]] * [[admin:home|Administration]] * [[admin:admin:home|Administration Basics]] * [[admin:admin:architecture:home|Omnidex Architecture]] * [[admin:admin:applications:home|Building Applications]] * [[admin:basics:home|Omnidex Fundamentals]] * [[admin:basics:environments:home|Omnidex Environments]] * [[admin:basics:sqlengine:home|Omnidex SQL Engine]] * [[admin:basics:updates:home|Omnidex Updates]] * [[admin:basics:network:home|Network Services]] * [[admin:indexing:home|Omnidex Indexing]] * [[admin:indexing:concepts:home|Indexing Concepts]] * [[admin:indexing:indexes:home|Indexing Options]] * [[admin:indexing:creation:home|Index Creation]] * [[admin:indexing:strategies:home|Indexing Strategies]] * [[admin:indexing:activecounts:home|ActiveCounts]] * [[admin:indexing:powersearch:home|PowerSearch]] * [[admin:indexing:autocomplete:home|AutoComplete]] * [[admin:indexing:text:home|Omnidex Text]] * [[admin:features:home|Omnidex Features]] * [[admin:features:snapshots:home|Omnidex Snapshots]] * [[admin:features:grids:home|Omnidex Grids]] * [[admin:features:rollups:home|Rollup Tables]] * [[admin:features:geo:home|Geographic Searches]] * [[admin:features:expressioncols:home|Expression Columns]] * [[admin:features:segments:home|Omnidex Segments]] * [[admin:features:attach:home|Attaching Data]] * [[admin:features:export:home|Exporting Data]] * [[admin:optimization:home|Optimizing Queries]] * [[admin:optimization:plans:home|Query Plans]] * [[admin:optimization:caches:home|Dynamic Caches]] * [[admin:optimization:config:home|Configuring Omnidex]] * [[integration:home|Integration]] * [[integration:rdbms:home|Relational Databases]] * [[integration:rdbms:oracle:home|Oracle]] * [[integration:rdbms:sqlserver:home|SQL Server]] * [[integration:rdbms:mysql:home|MySQL]] * [[integration:rdbms:odbc:home|Generic ODBC]] * [[integration:rawdata:home|Raw Data Files]] * [[integration:rawdata:fixed:home|Fixed-length Files]] * [[integration:rawdata:delimited:home|Delimited Files]] * [[integration:rawdata:ost:home|Standalone Tables]] * [[dev:home|Development]] * [[dev:sql:home|Omnidex SQL]] * [[dev:sql:overview|Overview]] * [[dev:sql:statements:home|Statements]] * [[dev:sql:functions:home|Functions]] * [[dev:sql:examples:home|Examples]] * [[dev:connections:home|Connection Strings]] * [[dev:odbc:home|ODBC Interface]] * [[dev:jdbc:home|JDBC Interface]] * [[dev:debugging:home|Logging/Debugging]] * [[programs:home|Programs]] * [[programs:odxsql:home|OdxSQL]] * [[programs:odxnet:home|OdxNet]] * [[appendix:home|Appendix]] * [[dev:sql:home|Omnidex SQL]] * [[dev:sql:statements:home|Statements]] * [[dev:sql:functions:home|Functions]] * [[dev:sql:examples:home|Examples]] * [[http://www.omnidex.com/download/|Downloads]] * [[support:home|Support]]

~~NOTOC~~ {{page>:top_add&nofooter&noeditbtn}} ====== Administration: Omnidex Features ====== ===== Omnidex Grids ===== [[admin:features:grids:home|Overview]] | [[admin:features:grids:partitions|Partitioning Scheme]] | [[admin:features:grids:distribution|Distribution Plan]] | **[[admin:features:grids:creation|Grid Creation]]** ---- ==== Creating an Omnidex Grid ==== Omnidex Grids are straightforward to create. Each grid node is a separate Omnidex environment, with an environment file, indexes and data. For convenience, there are methods to share a single Omnidex environment file across multiple nodes. There are also methods to share data files across multiple nodes. Before creating an Omnidex Grid, you must first have a [[admin:features:grids:partitions | partitioning scheme]] and a [[admin:features:grids:distribution | distribution plan]]. These will be needed during the steps of creating an Omnidex Grid. Omnidex Grids are straightforward to administer. Each grid node is a separate Omnidex environment, with an environment file, indexes and data. For convenience, there are methods to share a single Omnidex environment file across multiple nodes. There are also methods to share data files across multiple nodes. ==== Steps to create an Omnidex Grid using SQL Statements ==== === 1. Create an environment pointing to the original database === Omnidex Grids needs an environment that points to the original database so that the data can be partitioned. This environment is a standard environment that accesses the underlying database. === 2. Partition the database === If the database will be logically partitioned, then create the relational views that correlate with the partitioning strategy. If the database will be physically partitioned using an Omnidex Snapshot, then partition the database using the PARTITION statement. The PARTITION statement is available in the Omnidex SQL language, and is callable from the OdxSQL application. Since the statement is part of the Omnidex SQL language, it may also be called directly from ODBC and JDBC. The PARTITION statement allows the table to be partitioned one of three ways, 1) into a specific number of equally sized partitions, 2) into separate partitions based on a partition qualifier consisting of a column or a SQL expression, or 3) a combination of these two steps. In the third case, Omnidex will evaluate the cardinality of the partition qualifier and group values together to comprise the specific number of partitions in as equal size as possible. The syntax of the PARTITION statement is: PARTITION table [INTO n [PARTITIONS]] [BY column] [IN path] [WITH options] An example of a statement to partition a table is: PARTITION LIST BY “STATE || SUBSTRING(ZIP from 1 for 2)” IN D:\grid\data === 3. Distribute the partitioned data across the grid. === If the database will be logically partitioned, then make sure each grid server has access to the database. If the database will be physically partitioned, then distribute both the partitioned and unpartitioned data to the grid nodes. If each node will be in its own directory, then the data files can be copied or symbolically linked to the correct locations. It is also possible for multiple nodes to share the same physical directories as long as the partitioned data can be differentiated. === 4. Create the environment file for the Omnidex Grid. === The environment file for the Omnidex Grid is a normal environment file; however, it has additional information that describes the locations of the nodes and the partitioning scheme. There are changes that need to be made for the ENVIRONMENT statement, the DATABASE statement and the TABLE statements. Note that older versions of Omnidex Grids had a RULE statement at the bottom, and that is no longer necessary. == The CREATE ENVIRONMENT statement == The CREATE ENVIRONMENT statement is used to declare the grid nodes and their connection information. The NODE declarations in the CREATE ENVIRONMENT statement are show below. Consult the documentation on the [[dev:sql:statements:create_environment:home | CREATE ENVIRONMENT]] statement for the complete syntax. <code> CREATE ENVIRONMENT environment [MAX_THREADS n] [NODE node [<PARTITIONED | UNPARTITIONED>] [HOST <LOCALHOST | host> [PORT n]] [ENVIRONMENT “filename”]] </code> Here is an example of a CREATE ENVIRONMENT statement for an Omnidex Grid. <code> create environment "list_env" maxthreads 4 node NODE01 partitioned node NODE02 partitioned node NODE03 partitioned node NODE04 partitioned ... </code> == The CREATE DATABASE statement == The CREATE DATABASE statement is used to declare the database characteristics for each node. The NODE declarations in the CREATE DATABASE statement are show below. Consult the documentation on the [[dev:sql:statements:create_database:home | CREATE DATABASE]] statement for the complete syntax. <code> CREATE DATABASE database [NODE node] TYPE type [SUBTYPE subtype] [PHYSICAL "physical"] [USER “user”] [PASSWORD “password”] [INDEX_DIRECTORY “directory”] [INDEX_MAINTENANCE index_maintenance] ... </code> Here is an example of a CREATE DATABASE statement for an Omnidex Grid. <code> create database "list" node NODE01 type flatfile index_directory "idx/node01/LIST_" node NODE02 type flatfile index_directory "idx/node02/LIST_" node NODE03 type flatfile index_directory "idx/node03/LIST_" node NODE04 type flatfile index_directory "idx/node04/LIST_" </code> == The CREATE TABLE statement == The CREATE TABLE statement is used to declare the table characteristics for each node for each partitioned table in the grid. The NODE declarations in the CREATE TABLE statement are show below. Consult the documentation on the [[dev:sql:statements:create_table:home | CREATE TABLE]] statement for the complete syntax. <code> CREATE TABLE table [NODE node] [OPTIONS “options”] [PHYSICAL “physical”] [AUTOFILTER “criteria”] [DATA_CACHING <DYNAMIC | NONE>] [PARTITION_BY “criteria”] [INDEX_MAINTENANCE index_maintenance] ... </code> Here is an example of a CREATE TABLE statements for an Omnidex Grid. <code> create table "HOUSEHOLDS" node NODE01 physical "dat/node01/pro*.dat" partition_by "STATE in ('MA','ME','NH','NY','PR','RI','VT', 'NY','DE','PA','DC','MD','VA','WV') and ZIP between '00000' and '26999'" node NODE02 physical "dat/node02/pro*.dat" partition_by "STATE in ('NC','SC','GA','FL','AL','MS', 'TN','KY','OH') and ZIP between '27000' and '45999'" node NODE03 physical "dat/node03/pro*.dat" partition_by "STATE in ('IN','MI','IA','MN','MT','ND','SD', 'WI','IL','KS','MO','NE','AR','LA', 'OK','TX') and ZIP between '46000' and '79999'" node NODE04 physical "dat/node04/pro*.dat" partition_by "STATE in ('AZ','CO','ID','NM','NV','UT','WY', 'CA','HI','AK','OR','WA') and ZIP between '80000' and '99999'" </code> As discussed in the [[admin:features:grids:partitions#how_are_referential_constraints_handled_such_as_primary_and_foreign_keys| partition scheme]], it is necessary to partition tables so that referential integrity is maintained. Typically, the parent is partitioned using criteria, as shown for HOUSEHOLDS above, and the child is partitioned along the foreign key. The PARTITION BY clause for a child shows the join criteria, as shown below: <code> create table "INDIVIDUALS" node NODE01 physical "dat/node01/pro*.dat" partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD" node NODE02 physical "dat/node02/pro*.dat" partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD" node NODE03 physical "dat/node03/pro*.dat" partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD" node NODE04 physical "dat/node04/pro*.dat" partition_by "HOUSEHOLDS.HOUSEHOLD = INDIVIDUALS.HOUSEHOLD" </code> === 5. Install and build Omnidex indexes on each node. === It is recommended, though not required, that each node be indexed the same. This reduces the likelihood of queries taking too long because of a lack of indexing on one node. Indexes can be built in parallel. For maximum performance, it is recommended that the temporary directory, indicated by the TMPDIR environment variable, be different between builds whenever possible. === 6. Start Omnidex Network Services on each server. === Omnidex network services must be started on every machine that contains grid nodes. No special options are required. See how to configure and start the [[admin:network:overview | Omnidex Network Services]]. === 7. Perform queries against the controller. === Applications should connect to the grid controller as though it was a standard Omnidex environment. ==== ==== **[[admin:features:grids:distribution|Prev]]** ====== Additional Resources ====== {{page>:admin:features:see_also&nofooter&noeditbtn}} {{page>:bottom_add&nofooter&noeditbtn}}

 
Back to top
admin/features/grids/creation.txt · Last modified: 2016/06/28 22:38 (external edit)
 
 
chimeric.de = chi`s home Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0