MySQL Reference Manual

Chapter 9. Extending MySQL

MySQL Internals

This chapter describes a lot of things that you need to know when working on the MySQL code. If you plan to contribute to MySQL development, want to have access to the bleeding-edge, in-between-versions code, or just want to keep track of development, follow the instructions in Section 2.3.4. If you are interested in MySQL internals, you should also subscribe to our internals mailing list. This list is relatively low-traffic. For details on how to subscribe, please see Section 1.6.2.1. All developers at MySQL AB are on the internals list and we help other people who are working on the MySQL code. Feel free to use this list both to ask questions about the code and to send patches that you would like to contribute to the MySQL project!

MySQL Threads

The MySQL server creates the following threads:

• The TCP/IP connection thread handles all connection requests and creates a new dedicated thread to handle the authentication and SQL query processing for each connection.

• On Windows NT there is a named pipe handler thread that does the same work as the TCP/IP connection thread on named pipe connect requests.

• The signal thread handles all signals. This thread also normally handles alarms and calls process_alarm( ) to force timeouts on connections that have been idle too long.

• If mysqld is compiled with -DUSE_ALARM_THREAD, a dedicated thread that handles alarms is created. This is only used on some systems where there are problems with sigwait( ) or if one wants to use the thr_alarm( ) code in one’s application without a dedicated signal handling thread.

• If one uses the --flush_time=# option, a dedicated thread is created to flush all tables at the given interval.

• Every connection has its own thread.

• Every different table on which one uses INSERT DELAYED gets its own thread.

• If you use --master-host, a slave replication thread will be started to read and apply updates from the master.

mysqladmin processlist only shows the connection, INSERT DELAYED, and replication threads.

MySQL Test Suite

Until recently, our main full-coverage test suite was based on proprietary customer data and for that reason has not been publicly available. The only publicly available part of our testing process consisted of the crash-me test, a Perl DBI/DBD benchmark found in the sql-bench directory, and miscellaneous tests located in the tests directory. The lack of a standardised publicly available test suite has made it difficult for our users, as well as developers, to do regression tests on the MySQL code. To address this problem, we have created a new test system that is included in the source and binary distributions starting in Version 3.23.29.

The current set of test cases doesn’t test everything in MySQL, but it should catch most obvious bugs in the SQL processing code and OS/library issues. Also, it is quite thorough in testing replication. Our eventual goal is to have the tests cover 100% of the code. We welcome contributions to our test suite. You may especially want to contribute tests that examine the functionality critical to your system, as this will ensure that all future MySQL releases will work well with your applications.

Running the MySQL test suite

The test system consist of a test language interpreter (mysqltest), a shell script to run all tests (mysql-test-run), the actual test cases written in a special test language, and their expected results. To run the test suite on your system after a build, type make test or mysql-test/mysql-test-run from the source root. If you have installed a binary distribution, cd to the install root (e.g., /usr/local/mysql) and do scripts/mysql-test-run. All tests should succeed. If not, you should try to find out why and report the problem if this is a bug in MySQL. See Section 9.1.2.3.

If you have a copy of mysqld running on the machine where you want to run the test suite, you do not have to stop it, as long as it is not using ports 9306 and 9307. If one of those ports is taken, you should edit mysql-test-run and change the values of the master and/or slave port to one that is available.

You can run one individual test case with mysql-test/mysql-test-run test_name.

If one test fails, you should test running mysql-test-run with the --force option to check if any other tests fail.

Extending the MySQL test suite

You can use the mysqltest language to write your own test cases. Unfortunately, we have not yet written full documentation for it—we plan to do this shortly. You can, however, look at our current test cases and use them as an example. The following points should help you get started:

• The tests are located in mysql-test/t/*.test.

• A test case consists of ; terminated statements and is similar to the input of the mysql command-line client. A statement by default is a query to be sent to MySQL server, unless it is recognised as an internal command (e.g., sleep).

• All queries that produce results—e.g., SELECT, SHOW, EXPLAIN, etc.—must be preceded with @/path/to/result/file. The file must contain the expected results. An easy way to generate the result file is to run mysqltest -r < t/test-case-name.test from mysql-test directory, and then edit the generated result files, if needed, to adjust them to the expected output. In that case, be very careful about not adding or deleting any invisible characters—make sure to only change the text and/or delete lines. If you have to insert a line, make sure the fields are separated with a hard tab, and there is a hard tab at the end. You may want to use od -c to make sure your text editor has not messed anything up during editing. We, of course, hope that you will never have to edit the output of mysqltest -r, as you only have to do it when you find a bug.

• To be consistent with our setup, you should put your result files in the mysql-test/r directory and name them test_name.result. If the test produces more than one result, you should use test_name.a.result, test_name.b.result, etc.

• If a statement returns an error, you should, on the line before the statement, specify with the --error error-number. The error number can be a list of possible error numbers separated with ','.

• If you are writing a replication test case, you should, on the first line of the test file, put source include/master-slave.inc;. To switch between master and slave, use connection master; and connection slave;. If you need to do something on an alternate connection, you can do connection master1; for the master, and connection slave1; for the slave.

• If you need to do something in a loop, you can use something like this:

  let $1=1000;
  while ($1)
  {
   # do your queries here
   dec $1;
  }

• To sleep between queries, use the sleep command. It supports fractions of a second, so you can do sleep 1.3;, for example, to sleep 1.3 seconds.

• To run the slave with additional options for your test case, put them in the command-line format in mysql-test/t/test_name-slave.opt. For the master, put them in mysql-test/t/test_name-master.opt.

• If you have a question about the test suite, or have a test case to contribute, email to [e-mail]. As the list does not accept attachments, you should ftp all the relevant files to: ftp://support.mysql.com/pub/mysql/Incoming/.

Reporting bugs in the MySQL test suite

If your MySQL version doesn’t pass the test suite you should do the following:

• Don’t send a bug report before you have found out as much as possible about what when wrong! When you do it, please use the mysqlbug script so that we can get information about your system and MySQL version. See Section 1.6.2.3.

• Make sure to include the output of mysql-test-run, as well as contents of all .reject files in mysql-test/r directory.

• If a test in the test suite fails, check if the test also fails when run on its own:

  cd mysql-test
  mysql-test-run --local test-name

  If this fails, you should configure MySQL with --with-debug and run mysql-test-run with the --debug option. If this also fails send the trace file var/tmp/master.trace to ftp://support.mysql.com/pub/mysql/secret so that we can examine it. Please remember to also include a full description of your system, the version of the mysqld binary, and how you compiled it.

• Try also to run mysql-test-run with the --force option to see if any other test fails.

• If you have compiled MySQL yourself, check our to learn manual for how to compile MySQL on your platform or, preferably, use one of the binaries we have compiled for you at http://www.mysql.com/downloads/. All our standard binaries should pass the test suite!

• If you get an error, like Result length mismatch, or Result content mismatch it means that the output of the test didn’t match exactly the expected output. This could be a bug in MySQL, or it could occur because your mysqld version produces slightly different results under some circumstances.

  Failed test results are put in a file with the same base name as the result file with the .reject extension. If your test case is failing, you should do a diff on the two files. If you cannot see how they are different, examine both with od -c and check their lengths.

• If a test fails totally, you should check the logs file in the mysql-test/var/log directory for hints of what went wrong.

• If you have compiled MySQL with debugging you can try to debug this by running mysql-test-run with the --gdb and/or --debug options. See Section D.1.2.

  If you have not compiled MySQL for debugging you should probably do that. Just specify the --with-debug options to configure! See Section 2.3.