WebScaleSQL RPMs for CentOS 6

Looks like this post was rather unclear. See the bottom for how to build the rpms quickly.

WebScaleSQL was announced last week. This looks like a good thing for MySQL as it provides a buildable version of MySQL which includes multiple patches from Facebook, Google, LinkedIn, and Twitter needed by large users of MySQL, patches which have not been incorporated into the upstream source tree.  Making this more visible will possibly encourage more of these patches to be brought into the code sooner.

The source is provided as a git repo at https://github.com/webscalesql/webscalesql-5.6 and as detailed at http://webscalesql.org/faq.html the documentation says there is currently no intention to provide binaries.

Instructions on building the binaries and the build requirements for WebScaleSql can be found at http://webscalesql.org/faq.html and do not look too hard. However, I prefer to install my software as rpms as this makes upgrading or removing it later much easier.

With that in mind I thought I’d try and build some webscalesql rpms.

As I’m currently using MySQL-5.6 rpms, downloaded from http://dev.mysql.com/downloads/mysql/, I wanted to build WebScaleSQL rpms which were compatible with these.  I’m aware of the Oracle-built “community” rpms which are downloadable directory from their yum repo (https://dev.mysql.com/downloads/repo/) and will probably use these when upgrading to MySQL 5.7 but moving over to that now requires changing internal infrastructure and is currently not worth the effort.

In order to build the WebScaleSQL rpms I did the following:

  • download the latest source rpm, MySQL-5.6-17-1.el6.src.rpm from http://dev.mysql.com/downloads/file.php?id=451516
  • extract the spec file: rpm -ivh MySQL-5.6.17-1.el6.src.rpm and look in the directory specified by rpm –eval ‘%{_specdir}’ for the spec file (mysql.spec)
  • install the devtools package in order to use GCC 4.7:

http://people.centos.org/tru/devtools-1.1/readme says:

sudo wget http://people.centos.org/tru/devtools-1.1/devtools-1.1.repo -O /etc/yum.repos.d/devtools-1.1.repo
sudo yum install devtoolset-1.1
  •  clone the webscalesql.git repo to any directory, let’s call it  $WSS_HOME
  • create a webscale-5.6.tar.gz tar ball and put it in the RPM SRC_DIR:
     cd $WSS_HOME/.. && tar cz --exclude-vcs -f $_sourcedir/$tarball $WSS_HOME
  • create a minimally changed webscalesql.spec file, based on mysql.spec, to include the new path for the compiler toolchain
  • build the rpm by cd’ing into the directory with the webscalesql.spec file and running:
echo "Building WebScaleSQL..."
MYSQL_BUILD_PATH=/opt/centos/devtoolset-1.1/root/usr/bin:$PATH \
MYSQL_BUILD_CC=/opt/centos/devtoolset-1.1/root/usr/bin/gcc \
MYSQL_BUILD_CXX=/opt/centos/devtoolset-1.1/root/usr/bin/c++ \
rpmbuild -ba --define "distro_specific 1" webscalesql.spec

Warning: no explict check is made for the devtools chain to be installed and compilation will break if you try to use the native gcc compiler.

  •  this built the following rpms:

[sjmudd@myhost PKG]$ ls -l webscalesql-*
-rw-rw-r-- 1 sjmudd sjmudd 30925511 Mar 30 13:21 webscalesql-5.6-0.20140317.155729.rhel6.src.rpm
-rw-rw-r-- 1 sjmudd sjmudd 19333116 Mar 30 13:21 webscalesql-client-5.6-0.20140317.155729.rhel6.x86_64.rpm
-rw-rw-r-- 1 sjmudd sjmudd  1962332 Mar 30 13:23 webscalesql-devel-5.6-0.20140317.155729.rhel6.x86_64.rpm
-rw-rw-r-- 1 sjmudd sjmudd 81725224 Mar 30 13:24 webscalesql-embedded-5.6-0.20140317.155729.rhel6.x86_64.rpm
-rw-rw-r-- 1 sjmudd sjmudd 54210064 Mar 30 13:21 webscalesql-server-5.6-0.20140317.155729.rhel6.x86_64.rpm
-rw-rw-r-- 1 sjmudd sjmudd  2062312 Mar 30 13:23 webscalesql-shared-5.6-0.20140317.155729.rhel6.x86_64.rpm
-rw-rw-r-- 1 sjmudd sjmudd 53369596 Mar 30 13:23 webscalesql-test-5.6-0.20140317.155729.rhel6.x86_64.rpm
[sjmudd@myhost PKG]$

These rpms should work for CentOS 6, RHEL 6 and other equivalent distributions.  I have not actually tried to use any of the packages except the webscalesql-server.

I have had very little time so far to play with this, but did replace the MySQL-server package with webscalesql-server on a development server and let it run for a few hours.

One thing I did notice is that the performance_schema* settings I had in /etc/my.cnf were not recognised by webscalesql-server and had to be commented out. That said performance_schema still seemed to be there.

I need to check further but guess that this may be due to differences between MySQL and webscalesql or potentially something I have not done correctly when building.

Other than that the server replicated fine and I saw no issues.

This has given me some basical rpms for testing.  I have not tested the package on anything other than CentOS 6 and it is likely that other changes are needed. I probably need to do a few other things like:

  • Clean up the package further maybe adjusting copyrights or other messages about the packages.
  • Obsolete the installed MySQL-server, … rpms so I can just do rpm -Uvh webscalesql-server …. rather than remove the MySQL-server package first.
  • Add a bit of scripting to incorporate the date of the latest webscalesql commit into the version/release settings in the spec file. This avoids having to manually change the different values and as updates happen a rerun of the build script should just build a new package transparently.

I have not yet had time to look at the patches that have been applied to WebScaleSQL. It would certainly be nice to have some sort of list of functional changes (such as the performance_schema difference I noted earlier, assuming this is not a build error) of WebScaleSQL compared to the upstream source and any new configuration settings. Perhaps that will happen later?

At least for those of you who want to run a quick test of the binaries, or look at my spec file, you can find them on my website: http://ftp.wl0.org/webscalesql/.  No guarantees of any kind as you can imagine but feedback and improvements to the current spec file or build procedure would be most welcome.

2014-04-03 Update

See: https://github.com/sjmudd/webscalesql-rpm/ which I have created as a quick helper script to do the build. It still probably needs quite a bit of work but avoids copying instructions and doing stuff by hand.

2014-04-22 Update 2

Basically all you need to do is:

Install the devtoolset-1.1 or devtoolset-2 rpms (as indicated above)
Ensure that all required rpm build directories are created.
$ git clone https://github.com/webscalesql/webscalesql-5.6.git # clone WebScaleSQL sources
$ git clone https://github.com/sjmudd/webscalesql-rpm.git      # clone my builder script repo
$ cd webscalesql-rpm
$ ./build ../webscalesql-5.6                                   # build the rpms

If you later need to build an updated version of the rpms:

$ cd ../webscalesql-5.6
$ git pull # update sources
$ cd ../webscalesql-rpm
$ git pull # update my builder script (if necessary)
$ ./build  # no need to pass the webscalesql-5.6 directory location as it is remembered.

Published by

Simon J Mudd

Born in England, I now live in Spain, but spent a few years living in the Netherlands. I previously worked in banking (financial markets) both in IT and as a broker, but IT has always had a stronger influence. Now working at booking.com as a Senior Site Reliability Engineer managing a large fleet of MySQL database servers. Other interests include photography, and travel. Simon is married, with two children and lives in Madrid.

8 thoughts on “WebScaleSQL RPMs for CentOS 6”

  1. Hi,

    this instructions go blithely on the webscalesql.spec file generation, and I was unable to find an easy tutorial.

    First I just renamed mysql.spec, and tweaked it to solve directory depth problems, as rpmbuild was looking for something in


    Then I stumped to the dbug/ and debug/ dir names, as webscalesql seems to extract in dbug/ and mysql in debug/

    Definitely I have not insights in gmake process and try and error is not and efficient solution, where can I find help ?

    1. JuAn. Just checkout the git repo somewhere, then as per the readme checkout the webscalesql repo to a separate directory, and finally type build and it should build correctly. The mysql.spec file is only there for reference, use the webscalesql.spec file as described.

      1. What is the relation between MySQL and WebScaleSQL ?

        Do they need each other ?

        (there is still few documentation in WebScaleSQL site)

        1. Juan there is no relation between the two, obviously other than the fact that webscalesql is a branch of mysql, and this branch has optimisations for heavy “MySQL” usage.

          1. Thanks Simon,

            the instructions in this blog made me think that both MySQL and WebScaleSQL sources must be packed together in the RPM as if MySQL was needed by WebScaleSQL

            Information in WSS web site, misled me

          2. No, since WSS is mysql based I just kept the “base” mysql.spec as reference, and used that to create a spec file for WebScaleSQL. Only the WebScaleSQL git source is needed.

  2. OK

    now got spec from webscalesql-rpm.git
    I put webscale sources in ~/webscalesql-5.6 and WSS_HOME points to it
    I built rpm dir structure: mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
    When I run ~/webscalesql-rpm/build if breaks with:

    CMake Error: The source directory “~/rpmbuild/BUILD/webscalesql-5.6/webscalesql-5.6” does not exist.

    Somewhere in the build script must a duplicated directory reference

    1. I think you are making this harder than needed. I assumed you had a directory structure already configured for building rpms. If not that needs resolving first.

      I will try to make the instructions clearer when I get time in the next few days, including starting from a completely clean directory tree.

Leave a Reply