Versatile, portable, and efficient file system profiling.

摘要

File systems are complex and their behavior depends on many factors. Source code, if available, does not directly help understand the file system's behavior, as the behavior depends on actual workloads and external inputs. Runtime profiling is a key technique for understanding the behavior and mutual-influence of modern OS components. Such profiling is useful to prove new concepts, debug problems, and optimize the performance of existing file systems. Unfortunately, existing profiling methods are lacking in important areas: they do not provide much of the necessary information about the file system's behavior, they require OS modification and therefore are not portable, or they exact high overheads thus perturbing the profiled file system.;We developed a direct, real-time file system profiling method based on the analysis of latency distributions. Our method is versatile: a suitable workload can be used to profile virtually any OS component. Our method is portable because we can intercept operations and measure file system behavior from the user level or from inside the kernel without requiring source code. Our method is efficient: it has small overheads (less than 4% of the CPU time). Moreover, if the source code is available, we can use it to reduce overheads even further.;In this dissertation we describe our profiling method, the theory behind it, and the automation of the profile analysis. We demonstrate the usefulness of our method through a series of profiles conducted on Linux, FreeBSD, and Windows, including client/server scenarios. We discovered and investigated a number of interesting interactions, including scheduler behavior, multi-modal I/O distributions, and a previously unknown lock contention, which we fixed. We use our profiling method for performance analysis of a complex RAID-like fan-out stackable file system called RAIF that we have developed.