This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
admin:optimization:plans:home [2012/02/01 23:41] doc |
admin:optimization:plans:home [2016/06/28 22:38] (current) |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| {{page>:top_add&nofooter&noeditbtn}} | {{page>:top_add&nofooter&noeditbtn}} | ||
| - | ====== Administration ====== | + | ====== Administration: Optimizing Queries ====== |
| - | ===== Optimizing Queries ===== | + | ===== Query Plans ===== |
| **[[admin:optimization:plans:home|Overview]]** | | **[[admin:optimization:plans:home|Overview]]** | | ||
| - | [[admin:optimization:plans:sections|Sections]] | | + | [[admin:optimization:plans:reference|Anatomy of a Query Plan]] | |
| - | [[admin:optimization:plans:steps|Steps]] | | + | [[admin:optimization:plans:example|Reading a Query Plan]] | |
| - | [[admin:optimization:plans:optimized|Fully Optimized Queries]] | + | [[admin:optimization:plans:configuration|Configuring Query Plans]] | |
| + | [[admin:optimization:plans:optimization|Optimizing Queries]] | ||
| ---- | ---- | ||
| - | \\ | + | |
| ==== Overview ==== | ==== Overview ==== | ||
| - | Omnidex always optimizes a query as well as it can using the Omnidex indexes; however, if the indexes are not enough, Omnidex will complete the query without indexes, insuring the correct result. In fact, Omnidex can process queries even when no Omnidex indexes are available at all. In this way, Omnidex is first and foremost a SQL Engine for both relational and non-relational, or NoSQL, databases. | + | Omnidex always optimizes a query as well as it can using the Omnidex indexes; however, if the indexes are not enough, Omnidex will complete the query without indexes, insuring the correct result. In fact, Omnidex can process queries even when no Omnidex indexes are available. In this way, Omnidex is first and foremost a SQL Engine for both relational and non-relational, or NoSQL, databases. |
| - | Omnidex will evaluate the query and identify where indexes can be used. Omnidex evaluates the tables and their join relationships. Omnidex evaluates criteria, including nested queries, SQL functions and complex Boolean operations. Omnidex evaluates group by and order by, both to perform aggregations and to avoid unnecessary sorting of data. Omnidex even considers whether indexes can be used to return columns in the result set, avoiding accessing the data whenever possible. If there are not indexes to satisfy any of these steps, it will process then without the aid of indexing. | + | Omnidex will evaluate the query and identify where indexes can be used. Omnidex evaluates the tables and their join relationships. Omnidex evaluates criteria, including nested queries, SQL functions and complex Boolean operations. Omnidex evaluates GROUP BY and ORDER BY clauses, both to perform aggregations and to avoid unnecessary sorting of data. Omnidex even considers whether indexes can be used to return columns in the result set, avoiding access to the database whenever possible. In any of these situations, if there are not indexes to satisfy a step, it will process that part of the query without the aid of indexing. |
| The optimization plan for a query shows a sequence of steps, including table joins, processing criteria, aggregating data, and retrieving from the database. The ideal with Omnidex optimization is to avoid retrieving from the database if possible, and to fully optimize the query solely through the Omnidex indexes. If non-indexed steps are required, optimization tries to minimize these steps as much as possible. | The optimization plan for a query shows a sequence of steps, including table joins, processing criteria, aggregating data, and retrieving from the database. The ideal with Omnidex optimization is to avoid retrieving from the database if possible, and to fully optimize the query solely through the Omnidex indexes. If non-indexed steps are required, optimization tries to minimize these steps as much as possible. | ||
| - | ==== Optimizing a Query ==== | + | ==== Obtaining a Query Plan ==== |
| - | + | ||
| - | The following is a rather straightforward query that joins two tables together --- Individuals and their respective Households --- and aggregates counts of all people born since 1980 that are either in Denver, CO or in Phoenix, AZ. The result is a display of counts by Gender. Let's take a look at how Omnidex optimizes this query. | + | |
| - | + | ||
| - | <html> | + | |
| - | <table> | + | |
| - | <tr> | + | |
| - | <td valign="top"> | + | |
| - | <font face="'Courier New'"> | + | |
| - |   Select   I.GENDER, count(*) <br /> | + | |
| - |   from   HOUSEHOLDS H <br /> | + | |
| - |   join   INDIVIDUALS I <br /> | + | |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + | |
| - |    where   ((H.STATE = 'CO' and <br /> | + | |
| - |    H.CITY = 'Denver') or <br /> | + | |
| - |   (H.STATE = 'AZ' and <br /> | + | |
| - |    H.CITY = 'Phoenix')) and <br /> | + | |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + | |
| - |   group by   I.GENDER  | + | |
| - | </font> | + | |
| - | </td> | + | |
| - | </tr> | + | |
| - | </table> | + | |
| - | </html> | + | |
| - | + | ||
| - | === Step 1: Determine the processing order === | + | |
| - | + | ||
| - | Omnidex must first determine the best processing order for the query. This is determined based on analyzing the table relationships and the select items. Generally speaking, Omnidex needs to end up in the table where the select items or aggregations reside, and must process the other tables first. In this example, Omnidex will recognize that the HOUSEHOLDS table is the parent table, and the query will aggregate data in the INDIVIDUALS table. It will process the HOUSEHOLDS criteria first. | + | |
| - | + | ||
| - | <html> | + | |
| - | <table> | + | |
| - | <tr> | + | |
| - | <td valign="top"> | + | |
| - | <font face="'Courier New'"> | + | |
| - |   Select   <font color=red>I.GENDER, count(*) </font><br /> | + | |
| - |   from   <font color=red>HOUSEHOLDS H <br /> | + | |
| - |   join   INDIVIDUALS I <br /> | + | |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD </font><br /> | + | |
| - |    where   ((H.STATE = 'CO' and <br /> | + | |
| - |    H.CITY = 'Denver') or <br /> | + | |
| - |   (H.STATE = 'AZ' and <br /> | + | |
| - |    H.CITY = 'Phoenix')) and <br /> | + | |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + | |
| - |   <font color=red>group by   I.GENDER </font> | + | |
| - | </font> | + | |
| - | </td> | + | |
| - | <td valign="top"> | + | |
| - | <font color=red> | + | |
| - | <-- <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | + | ||
| - | </font> | + | |
| - | </td> | + | |
| - | <td valign="top"> | + | |
| - | <font color=red> | + | |
| - | Since this query will aggregate the INDIVIDUALS table, <br /> | + | |
| - | Omnidex will need to process the HOUSEHOLDS first <br /> | + | |
| - | and then end in the INDIVIDUALS table. <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | + | ||
| - | </font> | + | |
| - | </td> | + | |
| - | </tr> | + | |
| - | </table> | + | |
| - | </html> | + | |
| - | + | ||
| - | + | ||
| - | === Step 2: Process the HOUSEHOLD table's criteria === | + | |
| - | + | ||
| - | Omnidex will perform index qualifications to process the criteria in the HOUSEHOLDS table. If will do pay attention to Boolean operators and parentheses to insure the correct result. Once this is done, Omnidex will have isolated the rows that meet this criteria, identified by a temporary file containing index pointer. | + | |
| - | <html> | + | Query plans are produced using the EXPLAIN command in OdxSQL. A plan can be obtain by preceding the SELECT statement with the word EXPLAIN, as shown in the example below: |
| - | <table> | + | |
| - | <tr> | + | |
| - | <td valign="top"> | + | |
| - | <font face="'Courier New'"> | + | |
| - |   Select   I.GENDER, count(*) <br /> | + | |
| - |   from   HOUSEHOLDS H <br /> | + | |
| - |   join   INDIVIDUALS I <br /> | + | |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + | |
| - |    where   <font color=red>((H.STATE = 'CO' and <br /> | + | |
| - |    H.CITY = 'Denver') or <br /> | + | |
| - |   (H.STATE = 'AZ' and <br /> | + | |
| - |    H.CITY = 'Phoenix'))</font> and <br /> | + | |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + | |
| - |   <font color=red>group by   I.GENDER </font> | + | |
| - | </font> | + | |
| - | </td> | + | |
| - | <td valign="top"> | + | |
| - | <font color=red> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <-- <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | </font> | + | <code> |
| - | </td> | + | > explain select name, phone, email from individuals; |
| - | <td valign="top"> | + | ----------------------------------- SUMMARY ----------------------------------- |
| - | <font color=red> | + | Select NAME, |
| - | <br /> | + | PHONE, |
| - | <br /> | + | |
| - | <br /> | + | from INDIVIDUALS; |
| - | <br /> | + | |
| - | Omnidex will use the STATE and CITY indexes to process <br /> | + | |
| - | the HOUSHOLDS table's criteria, paying attention to <br /> | + | |
| - | Boolean operators and parentheses. <br /> | + | |
| - | <br /> | + | |
| - | <br /> | + | |
| - | </font> | + | Version: 5.2.01 (Compiled Jan 23 2012 17:27:25) |
| - | </td> | + | Warnings: SEQUENTIAL_SCAN |
| - | </tr> | + | ----------------------------------- DETAILS ----------------------------------- |
| - | </table> | + | Retrieve INDIVIDUALS sequentially; |
| - | </html> | + | Return INDIVIDUALS.NAME, INDIVIDUALS.PHONE, INDIVIDUALS.EMAIL; |
| + | ------------------------------------------------------------------------------- | ||
| + | </code> | ||
| + | The EXPLAIN command can also follow a query in OdxSQL. It will provide a query plan for the query that was just executed. This approach has the advantage of having more statistics about the query; however, generating the query plan before the query is run has the advantage of checking a query before it consumes too many resources. | ||
| + | A plan consists of two sections: the Summary and the Details. The Summary section shows the SQL statement, the version of Omnidex, and any optimization warnings or suggestions about the query. The Details section shows the steps that will be executed to satisfy the query. | ||
| - | [[admin:optimization:plans:home| More >]] | + | In the simple example above, three columns are retrieved from the INDIVIDUALS table. Since no criteria was provided in a WHERE clause, the details of the plan show that the table is retrieved sequentially and the columns are returned. A simple warning indicates that a sequential scan is taking place, which in this case cannot be avoided. |
| + | |||
| ===== ===== | ===== ===== | ||
| ---- | ---- | ||
| \\ | \\ | ||
| - | **[[admin:optimization:overview:hdc|Next]]** | + | **[[admin:optimization:plans:reference|Next]]** |
| ====== Additional Resources ====== | ====== Additional Resources ====== | ||
| See also: | See also: | ||
| - | {{page>:admin:indexing:see_also&nofooter&noeditbtn}} | + | {{page>:admin:optimization:see_also&nofooter&noeditbtn}} |
| {{page>:bottom_add&nofooter&noeditbtn}} | {{page>:bottom_add&nofooter&noeditbtn}} | ||