If querying is taking lot of time, and database connections and backlogs are growing, then it is time to optimize your database. In MySQL, you can use mysqlreport command to generate report on sufficiently used database and establish a baseline for further actions. You may also be required to modify configuration file (my.cnf) to log slow queries. Identify the slow running queries and the possible causes. Usually, the causes are related to indexes like no index or improper index. This article gives a detailed description of which commands and concepts we can use to optimize MySQL.
Using indexes makes the query faster in the same way as an index in a book helps in looking for a particular word.
To know how and which indexes are used in a given query, use EXPLAIN command as a prefix to select query.
EXPLAIN SELECT emp_id FROM ORGN_DATA WHERE dept_code = 'IT';
This command returns following information:
Table - Which table(s) are used in the query
Type - JOIN type. Values can be system, const, eq_ref, ref, range, index, all
Possible Keys - All keys which can be used for indexes
Key - The key actually used for index
Key Length - Shorter keys are better
Ref - Other Columns used with key to get the results
Rows - Number of rows from where data will come
Extra - Extra information. Some possible values can be using index, using where, using temporary, using filesort
You should reconsider your table structure if Possible Keys contain NULL value.
A NULL value in Key field indicates no index is being used. Using no indexes will deter the performance if there are too many records in the table because, for any query, all the records will be searched for the given condition.
Use ALTER command to add index as given below:
ALTER TABLE table_name ADD INDEX (column_name);
You can do indexing on multiple columns using:
ALTER TABLE table_name ADD INDEX (column_name1,..,column_nameN);
MySQL uses leftmost prefixing, so automatically indexes will be added for column_name1, (column_name1,column_name2),...(column_name1,..,column_NameN-1). This is quite helpful in situations like searching based on surname so an index on (surname, firstname, middlename) will automatically add multiple indexes used in faster querying.
Points to remember:
- Choose the column(s) for indexing very carefully.
- Indexed fields should be used for searching and query should be re-framed if any calculation is performed on indexed fields.
- Index itself takes storage space.
- Each operation on database requires updating index as well.
- A smaller Key Length reported by EXPLAIN command is better. Smaller key length means lesser storage space is required by index file and also that time to search in index is less. For this, indexes can be set on part of columns. For example, INDEX(col1(10),col2(20)). Again, you have to judiciously decide what length of columns to be used for indexing.
Using ANALYZE command
Use ANALYZE command to generate key distribution for the table to be used by MySQL query optimizer to decide which indexes are best to use in a query.
ANALYZE TABLE table_name;
Using OPTIMIZE command
If there are too many inserts/deletes in a table then OPTIMIZE command must be frequently used to optimize disk head movement during retrieval. This is particularly useful if variable size columns are used like varchar, text, blob etc. OPTIMIZE command does defragmentation of the fragmentation caused by variable length fields like VARCHAR.
OPTIMIZE TABLE table_name;
Using special functions for loading data
For loading data from a file into a database table, using LOAD DATA INFILE is much faster than reading data from file iteratively and using INSERT command. The syntax for this command is:
LOAD DATA INFILE 'filedata.dat' INTO TABLE table_name (col1,col2,...,colM) FIELDS TERMINATED BY '|'");
Setting PRIORITY of commands
If there are more queries than data insertions, you may lower priority of INSERT command using:
INSERT LOW_PRIORITY or SELECT HIGH_PRIORITY
If client is not interested in results of INSERT, then it can be immediately freed using following command:
This makes the system faster by batching insertions.
DELETE vs. TRUNCATE
DELETE deletes row by row, while TRUNCATE deletes all rows at once. So, if you are not interested in number of rows deleted from a table as result of DELETE command, then use TRUNCATE with following syntax:
TRUNCATE TABLE table_name;
Lowering permission checking overhead
Simpler permissions reduce permission checking overhead. Use GRANT command to set the permissions. General syntax for GRANT is:
GRANT Oper1,...,OperN ON db_name.tb_name TO user_name@computer_name IDENTIFIED BY password
GRANT SELECT,UPDATE ON EMP_DET.SALARY_DET TO feroz@localhost IDENTIFIED BY 'abc123';
If user account 'feroz' is not there, then it will be created on localhost (although user can access data from any computer) and password 'abc123' will be assigned to it.
Use BENCHMARK() function
To know how much a given MySQL function or expression is taking time, use MySQL built-in function:
It always returns 0 but also prints time taken by the expression.
Synchrozing data types
If columns in tables contain identical information, then make their data types also same so that joins can be performed faster.
- Insert multiple rows with single SQL statement.
- Remove unnecessary paranthesis in WHERE clause.
- If SQL_SMALL_RESULT option is used, then MySQL uses temporary in-memory tables which provide faster access.
Query optimization is not a one time job. If your database is heavily used then you may have to repeat the optimization process every 5-6 months.
Syed Feroz Zainvi maintains his technical blogs at: