One of the things that has made MySQL grow a lot is the easy setup of replication on Internet sites. This has allowed them to grow from one database server to many reasonably easily. Configuration of replication is quite simple and replication itself is extremely fast. That makes it ideal for most people.
However at a point in time you begin to notice some of the weaknesses in the current design. Here are a few comments providing ideas on what I would like to see in MySQL replication and why they are features which are important to anyone who’s database requirements grow outside of a single database instance.
My list of current issues is as follows:
- MySQL only allows replication from 1 to N servers.
- Some important aspects of MySQL Configuration are not dynamic.
- Replication is part of the mysqld server, not a separate process.
- It’s not easy to configure table replication (config issue)
- It’s not easy to filter the replication commands
- Initialisation of the databases when you first setup replication is difficult
- Replication configuration information is not in the database
MySQL only allows replication from 1 to N servers.
MySQL replicates from 1 to N servers. As mentioned in a previous blog the current replication design is very simple. Each mysqld server can get replication information from a single master and copies over the binlog from that master to the slave and replays part or all of the commands contained theerein. In MySQL up to 5.0 these are the SQL statements executed on the master, and in 5.1 and later it’s also possible to configure the master to store the rows that changed and replicate these changes on a row by row basis.
This design works well and is fast and it also scales out very well. However it does have issues. It creates a single point of failure in the master, which ends up being the weakest link, the source of all db changes. If the SELECT load is mainly distributed over the slaves then the problem is not too bad. However if the master fails all slaves will be broken.
There is however a second less obvious problem. In many companies there are various databases being used in the company. Sometimes the usage of different database servers is different and also the information which is required. However of course often there is a set of data which may be common to one or more departments. Replication is a useful way of distributing this information between the different database instances. What’s not possible is to selectively choose which databases should be replicated from a location to a location, and here the MySQL 1:N fanout is not always ideal. You can’t replicate databases from 2 different servers to a common second location. That would be very handy. It’s also not possible to replicate from multiple locations into a single dedicated “summary” database, perhaps used for global reporting. A design like this requires a little bit of care in setting up but allows independent groups of servers to be able to run separately but their results to appear in a global master system. It also helps with failures: if a branch server goes down the other satelittes offices are not affected, nor is the global reporting of the data.
I’ve used this type of replication mechanism in my last job (using Sybase) and found it to be extremely convenient. The added flexibility of being able to mix and match databases would give the DBA more freedom to be able to setup things for the business in a more intuitive manner. Of course keeping it simple (where possible) is a goal to aim for but N:1 and selective replication of sources databases to destionation servers would be very useful.
Some important aspects of MySQL Configuration are not dynamic.
With any “enterprise” system you want all changes within reason to be able to be performed while the system is running. This reduces downtime and thus keeps management happy. MySQL replication has a number of things which can not be configured dynamically. Having to temporarily stop replication to perform these tasks is in itself not an issue. Having to stop the db server where the replication is running is. This has an impact on the business and affects the DBA and also system administrators who may need to take action if the database instance goes out of service.
So what can’t be done dynamically?
As far as I’m aware the configuration of which databases should be replicated, the configuration of the different specific tables to include or exclude from replication. While this information does not change frequently if it does the impact on provoking downtime is significant. If Sun made these configuration options dynamic then we would not need to restart the mysqld server and therefore business could continue as usual. Most applications don’t notice or mind if there’s a small delay in replication and they can determine if this is the case and take appropriate action if needed. Howerver if the databse server goes away that’s much more of an issue.
Replication is part of the mysqld server, not a separate process.
I mentioned this before in http://blog.wl0.org/?p=7. Having the replication process in the same mysqld process makes it impossible without design changes to support anything except 1:N replication. Even without changing existing
functionality if you could build this as a separate process then you would be able to build N:1 replication, even if that required ‘N’ separate processes.
Also replication changes and new functionality could be added independently of the mysqld version. That would allow the groups of people within Sun who manage the mysqld and those that look at replication specific issues to do their own thing and not be hindered by each other. It would probably also allow improvements to be made to the replication process more quickly.
It’s not easy to configure table replication (config issue)
This is probably not considered an issue by people using mysql but that’s also probably because it’s not easily done there. Sometimes it’s convenient to have in a replicated database to have various types of tables: tables which are replication in the sense typical in MySQL, but many other tables which are local to the specific instance (perhaps instance configuration tables for
example). In MySQL’s current design managing a long list (hundreds) of specific tables to be replicated or to be excluded from replication is not very easy. Also this list is not dynamic. Thus improving the design here would help a lot.
It’s not easy to filter the replication commands
On some servers it might be nice to use it as a historic server containing all data, but on the master you might need to delete older data. Having a way of manipulating/filtering certain commands so that they can be eliminated or modified would allow this type of behaviour. For example filtering (and removing) commands (or the results of commands such as DELETE, UPDATE, TRUNCATE TABLE) could provide this sort of functionality. Again if the replication process were separate from mysqld this would be much easier to implement.
Initialisation of the databases when you first setup replication is difficult
Creating a new slave can be done in many ways. However it’s quite hard with a busy server especially when making the first slave to do so without stopping the master. That’s especially true as normal mysql dumps don’t guarantee a consistent database is generated nor is it easy to work out exactly the point in replication to use a dump from. What would be nice would be to setup a database as a copy of the master but have the mysql replication process fill the slave’s tables directly from the master with no extra help. At a certain size this may not be practical but for smaller database sizes this avoids a lot of extra complexity.
Replication configuration information is not in the database
Various parts of the replication configuration are outside of the database proper. The configuration is stored in separate text files (master.info relay-log.info) and not in the database itself. Where there are issues with MySQL’s mixed storage engines and thus how replication might work after recovery from a crash it would still be nicer if the data were stored in the database and thus hopefully the recovery after a crash would be consistent.
Quite a long time has passed and little has changed. One thing I did not add in my original post was a list of mysql bug reports which request the type of functionality mentioned above. Let me know of others I might have missed.
- http://bugs.mysql.com/35611 please make all slave replication options dynamic
- http://bugs.mysql.com/57560 please make all binary log options dynamically settable
- http://bugs.mysql.com/62825 Option to replicate at the database level instead of the server level