{"id":7438,"library":"mysql-replication","title":"MySQL Replication Client","description":"mysql-replication is a pure Python implementation of the MySQL replication protocol, built on top of PyMYSQL. It enables real-time streaming of binlog events from a MySQL server, facilitating tasks such as change data capture (CDC), data synchronization, and auditing. The current version is 1.0.15, and the project maintains an active release cadence, addressing bugs and adding support for newer Python versions.","status":"active","version":"1.0.15","language":"en","source_language":"en","source_url":"https://github.com/julien-duponchelle/python-mysql-replication","tags":["database","mysql","replication","binlog","cdc","realtime"],"install":[{"cmd":"pip install mysql-replication","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Core database driver used for establishing the connection to the MySQL server.","package":"PyMySQL","optional":false}],"imports":[{"symbol":"BinLogStreamReader","correct":"from mysql.replication import BinLogStreamReader"},{"symbol":"UpdateRowsEvent","correct":"from mysql.replication.row_event import UpdateRowsEvent"},{"symbol":"DeleteRowsEvent","correct":"from mysql.replication.row_event import DeleteRowsEvent"},{"symbol":"WriteRowsEvent","correct":"from mysql.replication.row_event import WriteRowsEvent"},{"symbol":"QueryEvent","correct":"from mysql.replication.event import QueryEvent"}],"quickstart":{"code":"import os\nfrom mysql.replication import BinLogStreamReader\nfrom mysql.replication.row_event import UpdateRowsEvent, DeleteRowsEvent, WriteRowsEvent\n\n# Configure your MySQL connection details. Use environment variables for production.\nMYSQL_HOST = os.environ.get('MYSQL_HOST', 'localhost')\nMYSQL_PORT = int(os.environ.get('MYSQL_PORT', 3306))\nMYSQL_USER = os.environ.get('MYSQL_USER', 'replication_user')\nMYSQL_PASSWORD = os.environ.get('MYSQL_PASSWORD', 'password') # Ensure user has REPLICATION SLAVE/CLIENT\n\ntry:\n    stream = BinLogStreamReader(\n        connection_settings={\n            \"host\": MYSQL_HOST,\n            \"port\": MYSQL_PORT,\n            \"user\": MYSQL_USER,\n            \"passwd\": MYSQL_PASSWORD,\n            \"charset\": \"utf8\"\n        },\n        server_id=101, # IMPORTANT: Must be a unique ID > 0 within your replication topology\n        blocking=True, # Wait for new events if no more are available\n        resume_stream=True, # Resume from last position if possible\n        log_file=None, # Start from current position\n        log_pos=None   # Start from current position\n    )\n\n    print(f\"Connected to MySQL binlog stream on {MYSQL_HOST}:{MYSQL_PORT}. Listening for events...\")\n\n    for event in stream:\n        if isinstance(event, (UpdateRowsEvent, DeleteRowsEvent, WriteRowsEvent)):\n            print(f\"\\n---\")\n            print(f\"[{type(event).__name__}] Database: {event.schema}, Table: {event.table}\")\n            for row in event.rows:\n                if isinstance(event, WriteRowsEvent):\n                    print(f\"  Inserted: {row['values']}\")\n                elif isinstance(event, UpdateRowsEvent):\n                    print(f\"  Before: {row['before_values']}\")\n                    print(f\"  After:  {row['after_values']}\")\n                elif isinstance(event, DeleteRowsEvent):\n                    print(f\"  Deleted: {row['values']}\")\n        # Add more event types (e.g., QueryEvent, RotateEvent, XidEvent) as needed\n\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")\nfinally:\n    if 'stream' in locals() and stream:\n        stream.close()\n        print(\"\\nBinLogStreamReader closed.\")","lang":"python","description":"This quickstart demonstrates how to connect to a MySQL server's binlog stream and process various row-based replication events (INSERT, UPDATE, DELETE). Ensure your MySQL server is configured for replication, and the provided user has sufficient privileges (`REPLICATION SLAVE`, `REPLICATION CLIENT`). Remember to set a unique `server_id`."},"warnings":[{"fix":"Upgrade to version `1.0.15` or newer. Always review your logging configuration in production environments to prevent exposure of sensitive data.","message":"Older versions (<=1.0.14) of `mysql-replication` could potentially log sensitive connection information (e.g., passwords) if logging was not carefully configured, especially in verbose modes.","severity":"gotcha","affected_versions":"<=1.0.14"},{"fix":"Ensure `server_id` is unique for each client connecting to the binlog. A common practice is to pick a high, unused number like 101 or 1000+.","message":"The `server_id` parameter in `BinLogStreamReader` must be a unique integer greater than 0 within your entire MySQL replication topology. Using a duplicate `server_id` can lead to unexpected behavior or replication conflicts.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Configure `my.cnf` with `log_bin`, `server-id`, `binlog_format=ROW`. Grant necessary permissions: `GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'replication_user'@'%' IDENTIFIED BY 'password';` then `FLUSH PRIVILEGES;`.","message":"Your MySQL server must be explicitly configured for binlog replication. This includes enabling `log_bin`, setting a unique `server-id` in `my.cnf`, and typically `binlog_format = ROW` to get detailed row changes. The replication user also needs `REPLICATION SLAVE` and `REPLICATION CLIENT` privileges.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Verify `MYSQL_HOST` and `MYSQL_PORT` in your connection settings. Check that the MySQL server is running and accessible from the client machine. Temporarily disable firewalls or ensure port 3306 (or your custom port) is open.","cause":"The Python client could not establish a connection to the MySQL server. This typically indicates an incorrect host/port, firewall blocking, or the MySQL server not running.","error":"pymysql.err.OperationalError: (2003, \"Can't connect to MySQL server on 'hostname' (111 Connection refused)\")"},{"fix":"Double-check `MYSQL_USER` and `MYSQL_PASSWORD`. Ensure the user has the required `REPLICATION SLAVE` and `REPLICATION CLIENT` grants for binlog access: `GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'replication_user'@'%' IDENTIFIED BY 'password'; FLUSH PRIVILEGES;`","cause":"The provided username or password for the MySQL connection is incorrect, or the user lacks the necessary `REPLICATION SLAVE` and `REPLICATION CLIENT` privileges.","error":"pymysql.err.OperationalError: (1045, \"Access denied for user 'replication_user'@'%' (using password: YES)\")"},{"fix":"Edit your `my.cnf` (or equivalent MySQL configuration file) and ensure `server-id = <unique_id>` is present under the `[mysqld]` section. Restart the MySQL server after making changes.","cause":"The MySQL server's `my.cnf` configuration is missing critical parameters for replication, specifically `server-id`.","error":"pymysql.err.OperationalError: (1290, \"The MySQL server is running with the --log-bin option, but is not configured for replication.\")"},{"fix":"Ensure you are using the latest version of `mysql-replication`. Check if your MySQL server version is explicitly supported. If it's a known event type that's not handled, you might need to extend the library or contribute a fix. For unknown low-level errors, check MySQL server logs for issues.","cause":"The library encountered an unsupported or malformed binlog event. This can sometimes happen with very new MySQL versions, custom event types, or corruption.","error":"mysql.replication.exceptions.BinLogStreamReaderException: Unknown event type: X (or similar low-level protocol error)"}]}