Jumpy, jittery, convoluted video and audio
For a long time, Myth did well. It did everything we expected it to and even a little more. Then shows began to degrade. It may have been a driver or software upgrade or a settings change – these things don’t “just happen” – but I could not trace the cause. Random shows would end up with jumpy audio and video skips. I found a similar description on the web: It is like the first 200ms of each second is missing. Try watching that. Ben came up with a three-level rating system for the severity of unwatchability. Level One was annoying, but watchable and often went away. Level Two neared pure noise and at Level Three you lost all hope.
I turned debugging on, collecting volumes of log files. At one point, I filled up my /var filesystem (how did I forget to turn cron on? Oh yeah, on most other operating systems it’s running by default!) which does not make Myth happy when it’s trying to write logs there.
I tweaked settings, trolled for clues in the detritus, searched for and requested help from the Internet. No improvement. No message in the data. No suggestion availed me.
Recordings are big files
1.3G Apr 4 00:00 1047_20060403230000.mpg
1.3G Apr 5 00:00 1047_20060404230000.mpg
1.3G Apr 6 00:00 1047_20060405230000.mpg
627M Mar 30 23:30 1063_20060330230000.mpg
1.3G Mar 23 02:00 1048_20060323010000.mpg
Behind the scenes the operating system breaks up each file into manageable blocks and writes the chunks to the disk. It tries very hard to put all the blocks in a row so it can read the whole file much like you are reading this text – word by word. When the OS deletes a file it makes the file’s blocks available for other files that may come along. This creates a “space” between two files on disk. Often, new files are bigger than the space created so the OS fills the space with part of the file and then uses new space for the remainder. This is called file fragementation. Fragmentation is bad because it slows down file reads as the OS has to look all over the disk to get each piece of a file. UFS, a widely-used filesystem for unix machines, somehow keeps fragementation from becoming a problem. However, I was using XFS to store recordings.
That made me wonder
I wondered if XFS was as good as keeping fragments out of the filesystem and if fragmentation could cause my A/V problems. I fired up the xfs_db utility to check the situation.
mythical # xfs_db -f /dev/mapper/myth_vg-myth_lv
actual 14468, ideal 402, fragmentation factor 97.22%
This seemed bad to me. The XFS defrag’er only works for two hours at a time, so I set up a little script in screen to do the defrag’ing while I slept.
mythical # while true
while> echo done!
while> sleep 300
When the work was done several hours later, new recordings improved in quality. I was happy with myself and posted the fix to the mailing list.