Recently we faced an issue when Nagios reported significant amount of “Locked” queries.
To investigate and debug the issue we needed to get more insight about the state of MySQL and the OS at the time the locks occurred.
This is how we got the much needed information:
wget http://bit.ly/1ltoZtk -O pt-stalk
chmod +x pt-stalk
mkdir -p /tmp/pt-stalk
sudo pt-stalk --daemonize --notify-by-email <EMAIL> --log /tmp/pt-stalk/pt-stalk.log --user root --dest /tmp/pt-stalk --function processlist --variable State --match Locked --threshold 20 --cycles=10 --sleep=15 --run-time=15
In this case we are running pt-stalk utility in background and trying to catch the case when there are 20+ Locked queries in processlist. When a match occurs, pt-stalk will collect a lot of OS and MySQL info and notify us by email.