Skip to main content

Query Tuning Tips

Query Tuning
-------------------

CO-Relations (vs) Joins (vs) Sets
++++++++++++++++++++++++++++++++++

Co-Relations/Joins/Sets are actually using for joint or disjoint collections.

1. Co-Relations sub queries are high cost expensive queries when compare to Joins.
2. However, Co-Relations are using less memory when compared to Joins.
3. Joins are always using some builting algorithms like (Hash join (Small set of rows), Merge Join (Huge set of row + Sort)) which makes the things very faster.

Find the below examples

-> CREATE TABLE TEST(T INT);
-> CREATE TABLE TEST1(T1 INT);
-> INSERT INTO TEST VALUES(GENERATE_SERIES(1,1000));
-> INSERT INTO TEST VALUES(GENERATE_SERIES(500,1499));
CO-Relation
----------- 
postgres=# EXPLAIN SELECT * FROM TEST AS OUT WHERE OUT.T IN (SELECT INN.T1 FROM TEST1 AS INN WHERE INN.T1=OUT.T);

QUERY PLAN                           

------------------------------------------------------------------

Seq Scan on test "out"  (cost=0.00..48043.00 rows=1200 width=4)

Filter: (SubPlan 1)

SubPlan 1

->  Seq Scan on test1 inn  (cost=0.00..40.00 rows=1 width=4)

Filter: (t1 = "out".t)

(5 rows)
If you want to tune this Co-Relation then use "EXISTS" instead of "IN" Operation.(Note:- Replacing EXISTS/NOT EXISTS with IN/NOT IN is depends on the subquery logic).

postgres=# EXPLAIN SELECT * FROM TEST AS OUT WHERE EXISTS (SELECT INN.T1 FROM TEST1 AS INN WHERE INN.T1=OUT.T);

QUERY PLAN                               

-------------------------------------------------------------------------

Hash Semi Join  (cost=26.50..54.25 rows=1000 width=4)

Hash Cond: ("out".t = inn.t1)

->  Seq Scan on test "out"  (cost=0.00..14.00 rows=1000 width=4)

->  Hash  (cost=14.00..14.00 rows=1000 width=4)

->  Seq Scan on test1 inn  (cost=0.00..14.00 rows=1000 width=4)

(5 rows)
Observer the cost of both queries.
Joins
----- 
postgres=# EXPLAIN SELECT * FROM TEST AS OUT INNER JOIN TEST1 AS INN ON INN.T1=OUT.T;

QUERY PLAN                               

-------------------------------------------------------------------------

Hash Join  (cost=26.50..54.25 rows=1000 width=8)

Hash Cond: ("out".t = inn.t1)

->  Seq Scan on test "out"  (cost=0.00..14.00 rows=1000 width=4)

->  Hash  (cost=14.00..14.00 rows=1000 width=4)

->  Seq Scan on test1 inn  (cost=0.00..14.00 rows=1000 width=4)

(5 rows)
Set Operators(Intersect)
------------------------ 
postgres=# EXPLAIN SELECT * FROM TEST INTERSECT SELECT * FROM TEST1;                

QUERY PLAN                                   

---------------------------------------------------------------------------------

HashSetOp Intersect  (cost=0.00..53.00 rows=1000 width=4)

->  Append  (cost=0.00..48.00 rows=2000 width=4)

->  Subquery Scan on "*SELECT* 1"  (cost=0.00..24.00 rows=1000 width=4)

->  Seq Scan on test  (cost=0.00..14.00 rows=1000 width=4)

->  Subquery Scan on "*SELECT* 2"  (cost=0.00..24.00 rows=1000 width=4)

->  Seq Scan on test1  (cost=0.00..14.00 rows=1000 width=4)
In the above three plans Set Operator's plan showing optimal result. We mayn't the same for all the cases. The execution plan always depends on the number of conditions what we are implying and the no.of rows and the row length.

In some of the cases Join Operations gives us very less cost when compares to Intersect operation.Because, Set operators always uses some internal SORT algorithms.
Inserted more records.

postgres=# SELECT COUNT(*) FROM TEST;

count

-------

32000

(1 row)



postgres=# SELECT COUNT(*) FROM TEST1;

count

-------

32000

(1 row)
Co-Relations
------------
postgres=# EXPLAIN SELECT * FROM TEST AS OUT WHERE OUT.T IN (SELECT INN.T1 FROM TEST1 AS INN WHERE INN.T1=OUT.T);

QUERY PLAN                            

--------------------------------------------------------------------

Seq Scan on test "out"  (cost=0.00..8417806.00 rows=16000 width=4)

Filter: (SubPlan 1)

SubPlan 1

->  Seq Scan on test1 inn  (cost=0.00..526.00 rows=32 width=4)

Filter: (t1 = "out".t)

(5 rows)



postgres=# EXPLAIN SELECT * FROM TEST AS OUT WHERE EXISTS (SELECT INN.T1 FROM TEST1 AS INN WHERE INN.T1=OUT.T);

QUERY PLAN                                

---------------------------------------------------------------------------

Hash Semi Join  (cost=846.00..1772.00 rows=32000 width=4)

Hash Cond: ("out".t = inn.t1)

->  Seq Scan on test "out"  (cost=0.00..446.00 rows=32000 width=4)

->  Hash  (cost=446.00..446.00 rows=32000 width=4)

->  Seq Scan on test1 inn  (cost=0.00..446.00 rows=32000 width=4)

(5 rows)
Joins
------
postgres=# EXPLAIN SELECT * FROM TEST AS OUT INNER JOIN TEST1 AS INN ON INN.T1=OUT.T;
QUERY PLAN
---------------------------------------------------------------------------
Hash Join (cost=846.00..12892.00 rows=1024000 width=8)
Hash Cond: ("out".t = inn.t1)
-> Seq Scan on test "out" (cost=0.00..446.00 rows=32000 width=4)
-> Hash (cost=446.00..446.00 rows=32000 width=4)
-> Seq Scan on test1 inn (cost=0.00..446.00 rows=32000 width=4)
(5 rows)

Set Operators
-------------
postgres=# EXPLAIN SELECT * FROM TEST INTERSECT SELECT * FROM TEST1;                                            

QUERY PLAN                                    

-----------------------------------------------------------------------------------

HashSetOp Intersect  (cost=0.00..1692.00 rows=1000 width=4)

->  Append  (cost=0.00..1532.00 rows=64000 width=4)

->  Subquery Scan on "*SELECT* 1"  (cost=0.00..766.00 rows=32000 width=4)

->  Seq Scan on test  (cost=0.00..446.00 rows=32000 width=4)

->  Subquery Scan on "*SELECT* 2"  (cost=0.00..766.00 rows=32000 width=4)

->  Seq Scan on test1  (cost=0.00..446.00 rows=32000 width=4)

(6 rows)
We need to check all the cases for fine tuning the query. Sometimes, we might get a better plan with JOINS when compares to INTERSECT.

We may get the co-related queries with DELETE/UPDATE Statements also. So, please re-write them with USING/FROM always. If possible we can go with INTERSECT also.

UPDATE
------

Actual
------
UPDATE Test AS OUT

SET T = ( SELECT T1 FROM Test1 INN WHERE INN.T=OUT.T);
Recommended
-----------
UPDATE Test AS OUT

SET T = T1

FROM Test1 AS INN Where INN.T=OUT.T;
DELETE
------
Actual
------
DELETE From Test As OUT Where NOT Exists (Select INN.T1 FROM TEST1 Where OUT.T!=INN.T1);

Recommended
----------- 
DELETE From Test As Out USING Test1 as Inn where Out.T!=Inn.T1
Equating Single Values (vs) In Single Value
-------------------------------------------

There could be slight cheper cost when compares to "=" and "IN". Find the below test case with "=" and using "IN".
postgres=# BEGIN WORK;

postgres=# DELETE FROM TEST WHERE T =1;                     

DELETE 64

Time: 65.426 ms

postgres=# ROLLBACK;



postgres=# BEGIN WORK;

postgres=# DELETE FROM TEST WHERE T IN(1);

DELETE 64

Time: 34.683 ms

postgres=# ROLLBACK;
UNION (vs) FULL OUTER JOINS
---------------------------

Actual Query with Unions.
------------------------- 
SELECT * FROM test WHERE true AND t IN (2,3,4,5) AND mp > 0

UNION

SELECT * FROM test WHERE true AND t IN (2,3,4,5) AND sp > 0

UNION

SELECT * FROM TEST WHERE TRUE AND T IN (2,3,4,5) AND TP>0;
Most of the Set operators are using Sort operations. If the sort operation is taking much more time, then following one is recommended with the indexes.
And with FullOuter Joins.
------------------------- 
SELECT coalesce(a.*,b.*,c.*) FROM

(SELECT * FROM test WHERE true AND t IN (2,3,4,5)AND mp > 0) AS A

FULL OUTER JOIN

(SELECT * FROM test WHERE true AND t IN (2,3,4,5)AND sp> 0) AS B

ON A.t=B.t and a.mp=b.mp and a.sp=b.sp and a.tp=b.tp

FULL OUTER JOIN

(SELECT * FROM test WHERE true AND t IN (2,3,4,5)AND tp> 0) AS C

ON B.t=C.t and B.mp=C.mp and C.sp=b.sp and C.tp=b.tp;
DISTINCT (vs) Group By
----------------------
DISTINCT operation also using "SORT" when it's in action. So, please use Group by instead of DISTINCT.

Actual
------ 
SELECT DISTINCT T FROM TEST;
Recommended
----------- 
SELECT T FROM TEST GROUP BY T;
Labels:

Comments

Post a Comment

Popular posts from this blog

Parallel Operations With pl/pgSQL

Hi, I am pretty sure that, there will be a right heading for this post. For now, i am going with this. If you could suggest me proper heading, i will update it :-) OK. let me explain the situation. Then will let you know what i am trying to do here, and how i did it. Situation here is, We have a table, which we need to run update on “R” no.of records. The update query is using some joins to get the desired result, and do update the table.  To process these “R” no.of records, it is taking “H” no.of hours. That too, it’s giving load on the production server. So, we planned to run this UPDATE as batch process.  Per a batch process, we took “N” no.or records. To process this batch UPDATE, it is taking “S” no.of seconds. With the above batch process, production server is pretty stable, and doing great. So, we planned to run these Batch updates parallel.  I mean, “K” sessions, running different record UPDATEs. Of-course, we can also increase the Batch size here.  But
30 comments
Read more

How To Send E-Mail From PostgreSQL

Hi , If you want to send E-Mails from PostgreSQL, then use the below Python 3.2 Script as below. I have used ActivePython 3.2 with PostgreSQL 9.1 for sending E-Mails from PostgreSQL. If you want to configure the Python 3.2 with PostgreSQL 9.1 then, please refer the below steps. http://manojadinesh.blogspot.in/2012/06/fatal-python-error-pyinitialize-unable.html Once, your Python 3.2 successful then follow the below steps to send an e-mail. Step 1 ===== postgres=# CREATE OR REPLACE FUNCTION public.send_email(_from Text,_password Text,smtp Text,port INT,receiver text, subject text, send_message text) RETURNS TEXT  LANGUAGE plpython3u AS $function$ import smtplib sender = _from receivers = receiver message = ("From: %s\nTo: %s\nSubject: %s\n\n %s"  % (_from,receiver,subject,send_message)) try:   smtpObj = smtplib.SMTP(smtp,port)   smtpObj.starttls()   smtpObj.login(_from, _password)   smtpObj.sendmail(sender, receivers,message)   print ('Successf
12 comments
Read more

::Pipelined in Oracle as well in PostgreSQL::

Pipelined Table Functions:- [ORACLE] =========================== If you want to return multiple rows to the calling environment, then piplined table functions is prefred. It will increase the dbperformance as well. Ex:- Step 1: ----------- CREATE TABLE EMP(EMPNO INT,ENAME VARCHAR2(10),SAL INT); Step 2: ----------- Insert sample data. Step 3: ----------- Create an object for the row type casting. CREATE OR REPLACE TYPE emp_row AS OBJECT ( empno INT, ename VARCHAR2(20), SAL INT ); Step 4: ----------- Create a Return Type for the pipelined function. CREATE OR REPLACE TYPE emp_table_type AS TABLE OF emp_row; Step 5: ----------- CREATE OR REPLACE FUNCTION emp_pipe_function RETURN emp_table_type PIPELINED IS BEGIN FOR rec in (select * from emp) LOOP PIPE ROW (emp_row(rec.empno,rec.ename,rec.sal)); END LOOP; RETURN; END; Step 6: ---------- SQL> select * from table(emp_pipe_function); EMPNO ENAME SAL ---------- ----
9 comments
Read more