Differences

This shows you the differences between two versions of the page.

--- admin:lifecycle:design [2009/11/29 18:14]
els
+++ — (current)
@@ Line 1: / Line 1: @@
-{{page>:top_add&nofooter&noeditbtn}}
-<html><div align="center"><span style="color:red">DRAFT</span></div></html>
-====== The Omnidex Application Lifecycle ======
-===== Design =====
-As with most applications, this is the most important step of the entire application lifecycle.  During the design step, key decisions are made about how to approach Omnidex queries and updates in the application.  These decisions can lead to high-performing applications, and they can also lead to poor performance.  It is worth spending the needed time on this step.
-This article focuses on the design steps for an Omnidex Application.  It does not focus on the steps for application design in general.  These steps should be interwoven with the design steps needed for the overall application.  Designing an Omnidex Application has six important steps:
-  - [[#Deciding_on_an_architecture|Deciding on an architecture]]
-  - [[#Understanding_the_data_model|Understanding the data model]]
-  - [[#Designing_an_indexing_strategy|Designing an indexing strategy]]
-  - [[#Prototyping_on_the_data_server|Prototyping on the data server]]
-  - [[#Optimizing_queries|Optimizing queries]]
-  - [[#Prototyping_the_application|Prototyping the application]]
-==== Deciding on an Architecture ====
-An Omnidex architecture is a plan for where the various components of the application will reside and how they will talk to each other.  In the simplest applications, everything resides on a single machine.  There may not even be a web server or an application server.  In more traditional applications, there is a web server, an application server and a data server, often on three separate machines.
-Omnidex adds three new concepts that affect the application architecture: Omnidex Snapshots,  Omnidex Grids and Omnidex Index Servers.  These concepts add a great deal of flexibility to the application architecture and are worth considering before embarking on a full application.
-In brief, Omnidex Snapshots are simple copies of the database stored in flat files that can be indexed and queried as independent databases.  Omnidex Snapshots are quite convenient because they can be heavily indexed and then easily distributed to different servers.  Many businesses direct much of their query traffic to Omnidex Snapshots, gaining performance in their application while reducing the load on their relational database. This assures the highest performance without requiring changes to the data model.
-Omnidex Grids allow large databases to be partitioned to improve performance and scalability.  Databases with over 20 million rows are candidates for Omnidex Grids, and there are several strategies for distributing the nodes of the grid to achieve the greatest flexibility and performance. Omnidex Grids are also a common way to incorporate large amounts to new data.  Large volumes of data can be added to a new node in the grid without having to affect the entire database.
-Omnidex Index Servers are similar to Omnidex Snapshots, but they only distribute the Omnidex indexes rather than a full copy of the database.  In many applications, Omnidex resolves most of the queries using only the Omnidex indexes.  This is especially true in applications that rely heavily on obtaining counts or aggregations.  Applications may direct these types of queries to Omnidex Index Servers, improving performance and easing the load on the data server.
-Architects would benefit from a deeper understanding of these concepts, and are encouraged to ready the article on [[admin:architecture:home|Omnidex Architecture]].
-==== Understanding the Data Model ====
-A good indexing strategy requires an understanding of the data model.  Specifically, it is important to understand the database schema, the table and column cardinalities and the pattern of queries.
-== Database Schema ==
-A basic database schema shows all the tables, their respective columns and datatypes, and their respective primary and foreign constraints. This schema provides an understanding of the table relationships and will be used to optimize table joins.  The schema also provides a list of columns for identifying likely candidates for indexing.
-Database schemas can be obtained from the underlying relational database, or can be obtained using the Omnidex Administrator program.
-== Table and Column Cardinalities ==
-Table cardinalities show the number of rows in each table.  Column cardinalities show the number of distinct values in each column.  For example, a table containing name and addresses about people may have a table cardinality of 300 million, meaning that there are 300 million rows in the table.  The GENDER column may have a column cardinality of two, meaning that there are two distinct values in the column. These cardinalities are important to predicting query performance and to tuning queries.
-Cardinalities can be obtained from the underlying relational database, or can be obtained using the Omnidex Administrator after an UPDATE STATISTICS statement has been completed on a table.
-== Sample SQL Queries ==
-Sample SQL queries show the patterns of queries that are expected in an application.  In turn, these patterns of queries are key to determining an indexing strategy.  When an application is first being prototyped, designers often index everything using Omnidex, but this can result in over-indexing, and it also doesn't insure that the correct indexing options are used on each column.  It also does not insure that table joins and aggregations will be properly optimized.  A review of the query patterns will insure the best performance for the least cost.
-Sample queries can be logged in most relational databases.  Omnidex can also log queries; however, that requires that Omnidex is already integrated into the application.
-When analyzing SQL queries, there are several patterns to recognize:
-  * **Table Joins and Nested Queries** - It is important to identify which tables are being accessed and how they are being joined together.  Omnidex frequently optimizes table joins.  Also note any nested queries, including the select item of the inner query and the criteria column in the outer query.
-  * **Criteria Columns** - Columns found in the WHERE clause of a SQL statement usually correlate with Omnidex indexes.  The literal values in the criteria are not particularly important, unless they contain LIKE operators and wildcards.  These indicate a likely use for Omnidex's text indexing features.
-  * **Aggregations** - Queries that use the COUNT, SUM, AVERAGE, MIN or MAX functions indicate  opportunities for optimization in Omnidex.  In these cases, note the columns being aggregated and the columns referenced in the GROUP BY clause.
-  * **Ordering** - Queries that have ORDER BY clauses indicate opportunities for optimization in Omnidex.  In these cases, note the columns use in the ORDER BY clause.  Note that at the present time, only ascending ORDER BY clauses are optimized.
-  * **Select Items** - In some situations, Omnidex may satisfy the select items from its own indexes.  Note any patterns of queries that return the same few columns for a large number of queries.
-====  Designing an indexing strategy ====
-  - [[#Prototyping_on_the_data_server|Prototyping on the data server]]
-  - [[#Optimizing_queries|Optimizing queries]]
-  - [[#Prototyping_the_application|Prototyping the application]]
-{{page>:bottom_add&nofooter&noeditbtn}}

Back to top

admin/lifecycle/design.1259518464.txt.gz · Last modified: 2012/10/26 14:25 (external edit)

Sitemap Recent Changes